# Pilot Analytics Dashboard — API Design Reference **Version**: 1.0 — Phase 1 (2026-04-29) **Scope**: Backend API contract for the Pilot Analytics Dashboard. This document is the single source of truth for **both backend and frontend/client** development. --- ## Table of Contents - [1 Overview](#1-overview) - [2 Authentication](#2-authentication) - [3 Status Constants](#3-status-constants) - [4 Timezone Handling](#4-timezone-handling) - [5 Endpoints](#5-endpoints) - [5 1 KPI Cards](#5-1-kpi-cards) - [5 2 Daily Summary](#5-2-daily-summary) - [5 3 Trend Charts](#5-3-trend-charts) - [5 4 Active Jobs Panel](#5-4-active-jobs-panel) - [5 5 Performance Gauges](#5-5-performance-gauges) - [5 6 Mark Job as Completed](#5-6-mark-job-as-completed) - [6 Error Responses](#6-error-responses) - [7 Data Model Notes](#7-data-model-notes) - [8 Frontend Integration Guide](#8-frontend-integration-guide) - [9 Backend Architecture Notes](#9-backend-architecture-notes) - [9 1 Pilot Scope Data Flow Diagram](#9-1-pilot-scope-data-flow-diagram) - [9 2 Endpoint Interaction Diagram](#9-2-endpoint-interaction-diagram) - [9 3 Performance Query Safety Diagram](#9-3-performance-query-safety-diagram) - [10 Open Decisions](#10-open-decisions) --- ## 1 Overview The Pilot Dashboard is a read-only analytics surface (plus one write action) scoped to the **currently authenticated Pilot user**. All endpoints derive the pilot identity from the JWT token — no client-side user ID parameter is accepted or trusted. **Base path**: `/api/dashboard/pilot` **Implemented in**: - `controllers/dashboard.js` - `routes/dashboard.js` - `routes/job.js` (for the complete action) **Testing resources**: - Postman collection: `docs/Pilot_Dashboard_API.postman_collection.json` - Node.js test script: `tests/test_pilot_dashboard_api.js` **All dashboard endpoints share the same scoping logic**: ``` pilot → Job.operator = req.uid → jobIds → Application.jobId IN jobIds ``` --- ## 2 Authentication All routes require a valid JWT bearer token. The `checkUser` middleware is applied globally in `server.js` before any route is mounted. ``` Authorization: Bearer ``` The token payload sets `req.uid` (user ID string) and `req.userInfo` on every request. Dashboard endpoints check `req.uid` and throw `401 Not Authorized` if absent. The dashboard route group (`/api/dashboard`) does **not** require a subscription package (`checkRqPkgSubscription` is not applied there). The complete-job endpoint lives under `/api/jobs` which **does** apply that middleware. --- ## 3 Status Constants ### Job Status (`helpers/job_constants.js`) | Constant | Value | Meaning | |--------------|-------|----------------------------------------------| | `NEW` | `0` | Job created, not yet prepared | | `READY` | `1` | Prepared and available for aircraft download | | `DOWNLOADED` | `2` | Downloaded to aircraft | | `SPRAYED` | `3` | Application data uploaded, not yet reviewed | | `COMPLETED` | `4` | Reviewed and marked done by Applicator | | `INVOICED` | `5` | Invoice issued | | `ARCHIVED` | `9` | Archived | ### Display Status (frontend label mapping) The `activeJobs` endpoint returns a `displayStatus` string derived from the raw numeric status: | `status` value(s) | `displayStatus` | Suggested UI treatment | |--------------------|-----------------|-----------------------------------| | `0` (NEW) | `"NEW"` | Grey badge, no progress bar | | `1`, `2`, `3` | `"IN_PROGRESS"` | Blue/active badge, show progress | | `4` (COMPLETED) | `"COMPLETED"` | Green badge, full bar | > INVOICED (`5`) and ARCHIVED (`9`) are excluded from the active jobs panel entirely. ### Application Status Only `status: 3` (processed) applications are counted in all aggregations. Uploading (`1`), in-progress (`2`), and error (`0`) applications are excluded automatically. --- ## 4 Timezone Handling All time-windowed endpoints accept an optional `?tz=` query parameter. - **Default**: `UTC` - **Example**: `?tz=America/Sao_Paulo`, `?tz=America/Chicago`, `?tz=Australia/Brisbane` - **Invalid values** silently fall back to `UTC` — the frontend should always send a valid tz. **What timezone affects**: - "Today" and "yesterday" day boundaries - Current week (Mon–Sun) boundaries - Current month and year boundaries - Date labels in the trend chart (`YYYY-MM-DD`) **What timezone does NOT affect**: - Raw UTC timestamps stored in MongoDB - Job `createdAt` (always UTC) **Recommended frontend behaviour**: read the browser's timezone once (`Intl.DateTimeFormat().resolvedOptions().timeZone`) and pass it on every dashboard request. --- ## 5 Endpoints ### 5 1 KPI Cards Returns top-level KPI cards and historical breakdowns for the authenticated pilot. #### Query Parameters | Parameter | Type | Default | Description | |-----------|--------|---------|------------------------------------| | `tz` | String | `UTC` | IANA timezone for period windows | #### Response `200 OK` ```json { "assignedJobs": 42, "assignedHectares": 1850.40, "sprayedToday": 120.50, "flightHoursToday": 3.25, "operations": { "distanceKm": 345.80, "sprayVolumeLiters": 2400.00 }, "historical": { "jobs": { "year": 38, "month": 12, "week": 4, "day": 42 }, "hectares": { "year": 1600.00, "month": 420.00, "week": 180.00, "day": 120.50 }, "flightHours": { "year": 210.50, "month": 52.75, "week": 18.25, "day": 3.25 } } } ``` #### Field Notes | Field | Unit | Notes | |-------------------------------|-----------|-------------------------------------------------| | `assignedJobs` | count | All-time total jobs where `Job.operator = uid` | | `assignedHectares` | ha | Sum of `Job.ttSprArea` across all assigned jobs | | `sprayedToday` | ha | Sum of `Application.totalSprayed` today | | `flightHoursToday` | hours | Sum of `Application.totalFlightTime` / 3600 | | `operations.distanceKm` | km | Sum of `Application.totalSprLength` / 1000 | | `operations.sprayVolumeLiters`| litres | Sum of `Application.totalSprayMat` | | `historical.jobs.day` | count | Same as `assignedJobs` (all-time) | | `historical.jobs.week/month/year` | count | Jobs created within the period (by `createdAt`) | All numeric values are rounded to 2 decimal places. --- ### 5 2 Daily Summary Returns today vs. yesterday operational metrics with percentage change deltas. #### Query Parameters | Parameter | Type | Default | Description | |-----------|--------|---------|----------------------------------| | `tz` | String | `UTC` | IANA timezone for day boundaries | #### Response `200 OK` ```json { "today": { "hectares": 120.50, "flightHours": 3.25, "haPerHour": 37.08, "avgSpeedKmh": 28.50, "sprayVolumeLiters": 2400.00 }, "yesterday": { "hectares": 95.00, "flightHours": 2.80, "haPerHour": 33.93, "avgSpeedKmh": 26.10, "sprayVolumeLiters": 1900.00 }, "deltas": { "hectaresPct": 27, "flightHoursPct": 16, "haPerHourPct": 9, "avgSpeedPct": 9, "sprayVolumePct": 26 } } ``` #### Field Notes | Field | Unit | Notes | |---------------------------------|---------|-----------------------------------------------------------| | `today.haPerHour` | ha/hr | Derived: `hectares / flightHours` (0 if no flight time) | | `today.avgSpeedKmh` | km/h | `Application.avgSpraySpeed` (m/s) × 3.6, averaged | | `deltas.*Pct` | % | `round((today − yesterday) / yesterday × 100)` | | `deltas.*Pct` = `null` | — | yesterday value was 0 (division by zero avoided) | Positive delta = improvement today vs. yesterday. Negative = drop. --- ### 5 3 Trend Charts Returns daily hours flown and hectares sprayed for a date range. Default range is the current calendar week (Monday–Sunday) in the given timezone. #### Query Parameters | Parameter | Type | Default | Description | |-------------|--------|--------------------|--------------------------------------------------| | `tz` | String | `UTC` | IANA timezone for day grouping and boundaries | | `startDate` | String | Monday of this week| Start date inclusive. Format: `YYYY-MM-DD` | | `endDate` | String | Sunday of this week| End date inclusive. Format: `YYYY-MM-DD` | - Maximum range: **90 days**. Returns `409` if exceeded. - Both `startDate` and `endDate` must be provided together, or both omitted. #### Response `200 OK` ```json { "labels": ["2026-04-27", "2026-04-28", "2026-04-29", "2026-04-30", "2026-05-01", "2026-05-02", "2026-05-03"], "hoursFlown": [2.50, 3.25, 0, 1.80, 0, 0, 0], "hectaresPerDay":[80.0, 120.5, 0, 65.0, 0, 0, 0] } ``` #### Field Notes | Field | Type | Notes | |-----------------|-----------------|----------------------------------------------------------| | `labels` | `String[]` | One entry per calendar day in `YYYY-MM-DD`, tz-adjusted | | `hoursFlown` | `Number[]` | Parallel array. Days with no activity = `0` | | `hectaresPerDay`| `Number[]` | Parallel array. Days with no activity = `0` | **Array contract**: all three arrays are always the same length and in the same order. Frontend can zip them: `labels[i]` ↔ `hoursFlown[i]` ↔ `hectaresPerDay[i]`. --- ### 5 4 Active Jobs Panel Returns the pilot's active-status jobs with per-job progress and applied totals. - Statuses included: `NEW (0)`, `READY (1)`, `DOWNLOADED (2)`, `SPRAYED (3)`, `COMPLETED (4)` - Statuses excluded: `INVOICED (5)`, `ARCHIVED (9)` - Maximum 50 most recent jobs returned (sorted by `createdAt` descending) #### Query Parameters None. #### Response `200 OK` ```json { "jobs": [ { "jobId": 1042, "name": "North Block — Canola", "clientName": "Sunrise Farms Ltd.", "aircraftReg": "C-FABM", "status": 3, "displayStatus": "IN_PROGRESS", "haTotal": 250.00, "haSprayed": 187.50, "progressPct": 75, "volumeAppliedLiters": 3750.00 } ] } ``` #### Field Notes | Field | Type | Notes | |-----------------------|---------|--------------------------------------------------------------------| | `jobId` | Number | Numeric job ID (`Job._id`, not a MongoDB ObjectId) | | `name` | String | Job name. Empty string if not set | | `clientName` | String | From linked `Client` user record. Empty if not linked | | `aircraftReg` | String | `Vehicle.tailNumber` or `Vehicle.unitId`. Empty if not linked | | `status` | Number | Raw numeric status (see constants table) | | `displayStatus` | String | `"NEW"` \| `"IN_PROGRESS"` \| `"COMPLETED"` | | `haTotal` | Number | `Job.ttSprArea` — planned area in ha. `0` if not set | | `haSprayed` | Number | Sum of `Application.totalSprayed` for this job (processed only) | | `progressPct` | Number | `0–100`. `0` if `haTotal` is 0 or no applications | | `volumeAppliedLiters` | Number | Sum of `Application.totalSprayMat` for this job | **`progressPct` formula**: `min(100, max(0, round(haSprayed / haTotal × 100)))` This can exceed 100% theoretically if `haSprayed > haTotal`; the backend caps it at 100. --- ### 5 5 Performance Gauges Returns average XT cross-track error and spray altitude gauges based on the pilot's last 10 processed application files. > **Note**: ApplicationDetail is a billion-row collection. Queries are strictly > bounded to a set of `fileId` values to use the collection's primary index. > Never query this collection without a `fileId` filter. #### Query Parameters None. #### Response `200 OK` — data available ```json { "avgXtErrorMeters": 1.85, "xtThreshold": { "good": 1.0, "monitor": 3.0 }, "hasXtData": true, "avgSprayAltitudeMeters": 3.62, "altitudeSource": "sprayHeight", "altThreshold": { "target": 3.7, "goodBand": 0.15, "monitorBand": 0.46 }, "hasAltitudeData": true, "sampleSize": 24 } ``` #### Response `200 OK` — no data (new pilot, no apps uploaded) ```json { "avgXtErrorMeters": null, "xtThreshold": { "good": 1.0, "monitor": 3.0 }, "hasXtData": false, "avgSprayAltitudeMeters": null, "altitudeSource": null, "altThreshold": { "target": 3.7, "goodBand": 0.15, "monitorBand": 0.46 }, "hasAltitudeData": false, "sampleSize": 0 } ``` #### Field Notes | Field | Unit | Notes | |---------------------------|--------|--------------------------------------------------------------------| | `avgXtErrorMeters` | metres | Average of `abs(ApplicationDetail.xTrack)`. `null` if no XT data | | `xtThreshold.good` | metres | Below this → green zone | | `xtThreshold.monitor` | metres | Above this → red zone; between good and monitor → yellow | | `hasXtData` | bool | `false` = no valid xTrack readings in sample (show "No data") | | `avgSprayAltitudeMeters` | metres | Average spray height from best available sensor. `null` if none | | `altitudeSource` | String | `"sprayHeight"` (FM dedicated sensor) or `"radarAlt"` (AGL). `null` if no data | | `altThreshold.target` | metres | Ideal spray height (~3.7 m) | | `altThreshold.goodBand` | metres | ±0.15 m of target → green | | `altThreshold.monitorBand`| metres | ±0.46 m of target → yellow; outside → red | | `hasAltitudeData` | bool | `false` = aircraft has no height sensor (show "No data") | | `sampleSize` | count | Number of AppFile records in the analysis window (up to ~20) | #### Threshold Gauge Logic (frontend) ``` XT Cross-track error: value < good → green (acceptable precision) good ≤ value < monitor → yellow (monitor) value ≥ monitor → red (needs attention) Spray Altitude: |value - target| < goodBand → green |value - target| < monitorBand → yellow |value - target| ≥ monitorBand → red ``` #### Altitude Source Priority The backend selects the best available altitude sensor in this order: 1. **`sprayHeight`** — dedicated FM spray height sensor (most accurate) 2. **`radarAlt`** — radar altimeter (AGL, good fallback) 3. **No data** — `hasAltitudeData: false`, `avgSprayAltitudeMeters: null` > `gpsAlt` (AMSL) is **not** used for spray height — it measures altitude above sea level, > not above the crop canopy. `xTrack = 0` values are excluded from the XT average because `0` means "no reading", not "perfectly on track". --- ### 5 6 Mark Job as Completed Transitions a job from **SPRAYED (3)** to **COMPLETED (4)**. This endpoint lives under `/api/jobs` (not `/api/dashboard`) and requires an active subscription package (enforced by `checkRqPkgSubscription` middleware). #### URL Parameters | Parameter | Type | Description | |-----------|--------|----------------------------------------| | `job_id` | Number | Numeric job ID | #### Request Body None. #### Response `200 OK` Returns the full updated job document (with `client`, `operator`, and `vehicle` populated). ```json { "_id": 1042, "name": "North Block — Canola", "status": 4, "client": { "_id": "...", "name": "Sunrise Farms Ltd." }, "operator": { "_id": "...", "name": "Jane Pilot" }, "vehicle": { "_id": "...", "name": "Agri-One", "tailNumber": "C-FABM" }, "..." } ``` #### Authorization Only the **Applicator** who owns the job (`Job.byPuid === req.uid`) may complete it. The Pilot (`Job.operator`) cannot call this endpoint. #### Error Cases | Condition | Status | Error tag | |----------------------------------------------|--------|-----------------------| | `job_id` is not a valid positive number | `409` | `invalid_param` | | Job not found | `409` | `job_not_found` | | Caller is not the job owner (`byPuid`) | `401` | `not_authorized` | | Job status is not `SPRAYED (3)` | `409` | `status_job_invalid` | --- ## 6 Error Responses All errors follow the standard AgMission error format: ```json { "error": { ".tag": "error_constant_value", "message": "Human-readable detail (development mode only)" } } ``` | HTTP Status | `.tag` value | When it occurs | |-------------|------------------------|------------------------------------------------------| | `401` | `not_authorized` | Missing/invalid JWT, or caller not the job owner | | `409` | `invalid_param` | Bad date format, range > 90 days, invalid job_id | | `409` | `job_not_found` | Job does not exist | | `409` | `status_job_invalid` | Job is not in required status for transition | --- ## 7 Data Model Notes ### Pilot Scoping All Application metrics are scoped via the Job's `operator` field, not `byUser` on Application: ``` Job.operator = pilotId → collect jobIds → Application.jobId IN jobIds ``` `Application.byUser` is the master Applicator account — it is **not** the pilot. Never use it to scope pilot metrics. ### Application.createdDate Time-window filtering uses `Application.createdDate` (the upload timestamp, a real MongoDB Date), **not** `Application.endDateTime`. The `endDateTime` field is a raw device string with no guaranteed format and is unreliable for date arithmetic. ### Job._id is a Number `Job._id` is an auto-incrementing Number (via `mongoose-sequence`), not a MongoDB ObjectId. Frontend must treat `jobId` values as integers, not hex strings. ### Vehicle and Client are User discriminators Both `Vehicle` (kind=`DEVICE`) and `Client` (kind=`CLIENT`) are stored in the `users` MongoDB collection. The `$lookup` in `activeJobs` joins against `users` for both. ### Models and Collections | Model name in code | Collection | Key dashboard fields | |--------------------|-----------------------|-------------------------------------------------------------| | `Job` | `jobs` | `operator`, `byPuid`, `ttSprArea`, `status`, `client`, `vehicle` | | `App` (Application)| `applications` | `jobId`, `status`, `createdDate`, `totalSprayed`, `totalFlightTime`, `totalSprLength`, `totalSprayMat`, `avgSpraySpeed` | | `AppFile` | `appfiles` | `appId` | | `AppDetail` | `application_details` | `fileId`, `xTrack`, `sprayHeight`, `radarAlt` | --- ## 8 Frontend Integration Guide ### Suggested Fetch Strategy Load the dashboard in two tiers to keep the initial paint fast: **Tier 1 — above the fold (load in parallel on mount)** ``` GET /api/dashboard/pilot/kpi?tz=... GET /api/dashboard/pilot/summary?tz=... GET /api/dashboard/pilot/activeJobs ``` **Tier 2 — charts and gauges (load after Tier 1 resolves or in parallel)** ``` GET /api/dashboard/pilot/trend?tz=...&startDate=...&endDate=... GET /api/dashboard/pilot/performance ``` ### Timezone Snippet ```javascript // Read once and reuse const tz = Intl.DateTimeFormat().resolvedOptions().timeZone; // e.g. "America/Sao_Paulo" const params = new URLSearchParams({ tz }); fetch(`/api/dashboard/pilot/kpi?${params}`, { headers: { Authorization: `Bearer ${token}` } }); ``` ### Empty State Handling Every endpoint returns meaningful zero-filled data when the pilot has no jobs: - Number fields return `0` - Null fields (`avgXtErrorMeters`, `altitudeSource`) return `null` - Boolean guard fields (`hasXtData`, `hasAltitudeData`) return `false` - Array fields (`labels`, `hoursFlown`, `hectaresPerDay`) return empty `[]` The frontend should check `hasXtData` and `hasAltitudeData` before rendering gauges, and render a "No data yet" placeholder instead of a gauge at `null`. ### Progress Bar Colour Logic Use `displayStatus` from `activeJobs` for badge colour, and `progressPct` for bar width: ``` displayStatus = "NEW" → grey badge, hide progress bar displayStatus = "IN_PROGRESS" → blue badge, show progress bar at progressPct% displayStatus = "COMPLETED" → green badge, show bar at 100% ``` ### Mark as Completed Button Visibility The **Complete** action is restricted to the **Applicator** (`Job.byPuid`), not the Pilot. If the frontend is used by an Applicator who is reviewing a pilot's job, show the button only when the raw `status === 3` (SPRAYED). After a successful `PATCH`, update the job in local state to `status = 4`, `displayStatus = "COMPLETED"`. ### Trend Chart Date Pickers - Pass `startDate`/`endDate` as `YYYY-MM-DD` strings in the user's local timezone. - Maximum selectable range: 90 days. - Default range on first render: current week (omit both parameters and let the API default). ### Gauge Rendering Reference ``` XT Error gauge (lower is better): 0 m ──────── 1.0 m ──────────── 3.0 m ──────────→ green yellow red ↑ good ↑ monitor Altitude gauge (target 3.7 m): ← red ── 3.24 ── yellow ── 3.55 ── [3.7] ── 3.85 ── yellow ── 4.16 ── red → ↑ -0.46 ↑ -0.15 target ↑ +0.15 ↑ +0.46 ``` --- ## 9 Backend Architecture Notes ### 9 1 Pilot Scope Data Flow Diagram ```mermaid flowchart LR A[Authenticated User req uid] --> B[Job Query operator equals uid] B --> C[Collect Job IDs] C --> D[Application Query jobId in Job IDs and status 3] D --> E[KPI Summary Trend Active Jobs Aggregates] ``` ### 9 2 Endpoint Interaction Diagram ```mermaid flowchart TD FE[Frontend Pilot Dashboard] --> K[GET pilot kpi] FE --> S[GET pilot summary] FE --> T[GET pilot trend] FE --> A[GET pilot activeJobs] FE --> P[GET pilot performance] FE --> C[PATCH jobs job_id complete] K --> J[(jobs)] K --> AP[(applications)] S --> AP T --> AP A --> J A --> AP P --> AP P --> AF[(appfiles)] P --> AD[(application_details)] C --> J ``` ### 9 3 Performance Query Safety Diagram ```mermaid flowchart LR A[Scoped Job IDs] --> B[Recent Processed Applications limit 10] B --> C[AppFile Lookup by appId] C --> D[ApplicationDetail Match by fileId in list] D --> E[Aggregate XT and Altitude Metrics] ``` ### Why Job.operator and not Application.byUser `Application.byUser` is the master Applicator account that uploaded the file — not the pilot assigned to fly the job. Using it would mix data from all jobs the Applicator manages, not just those assigned to this pilot. The correct field is `Job.operator` (set when a pilot is assigned to a job). ### Why Application.createdDate and not endDateTime `Application.endDateTime` is a raw string as received from the aircraft firmware. Its format varies by firmware version and is not reliable for MongoDB date range queries. `createdDate` is a `Date` field set by the server at upload time and is consistent across all records. ### ApplicationDetail Query Safety `application_details` is a very large collection (potentially billions of rows). The only index available for dashboard use is `{ fileId: 1 }`. All performance queries follow a three-step pattern to avoid collection scans: ``` 1. App.find({ jobId: { $in: jobIds } }).limit(10) → recent appIds 2. AppFile.find({ appId: { $in: appIds } }) → fileIds 3. AppDetail.aggregate([{ $match: { fileId: { $in: fileIds } } }, ...]) ``` Never add additional `ApplicationDetail` queries without scoping by `fileId` first. ### Unit Conversions (done server-side) | Raw storage | Conversion | API field | |------------------|--------------------|------------------------------| | `totalFlightTime`| seconds → hours ÷3600 | `flightHours`, `hoursFlown` | | `totalSprLength` | metres → km ÷1000 | `operations.distanceKm` | | `avgSpraySpeed` | m/s → km/h ×3.6 | `avgSpeedKmh` | All other fields (`totalSprayed`, `totalSprayMat`, `xTrack`, `sprayHeight`, `radarAlt`) are stored and returned in their natural units (ha, L, m). --- ## 10 Open Decisions These items affect backend behaviour but have not been finalised by the Product Owner. They are Phase 2 scope and the current implementation uses the defaults noted. | ID | Question | Current default | |--------|--------------------------------------------------------------------------|----------------------------------------------| | KPI-1 | Does "Assigned Jobs" count all-time or just open/current-season jobs? | All-time | | ACT-1 | Should INVOICED (5) jobs appear in the Active Jobs panel? | Excluded | | Q11 | Can the Pilot trigger the complete action, or Applicator-only? | Applicator (`byPuid`) only | | PERF-1 | When no altitude sensor exists, should the gauge show N/A or be hidden? | Returns `hasAltitudeData: false`, value null | | JOBS-1 | Are jobs with `operator = null` (unassigned) relevant to any view? | Not included in any pilot-scoped query |