957 lines
49 KiB
Markdown
957 lines
49 KiB
Markdown
# Data Export API — Design & Implementation Guide
|
||
|
||
Single Source of Truth: This is the canonical document for Data Export API design, implementation status, and next steps in this branch.
|
||
|
||
**Branch:** `data-export-api`
|
||
**Date:** April 10, 2026
|
||
**Status:** Phase A complete — Phase B in progress
|
||
|
||
## Change Log
|
||
|
||
| Date | Update |
|
||
|---|---|
|
||
| 2026-04-14 | Added `ApiKeyServices` and `ExportUnits` frozen constants to `helpers/constants.js`; wired throughout models and controllers. Added `service` field to `ApiKey`, `units` field to `ExportJob`, US unit conversion support to async export. |
|
||
| 2026-04-10 | Marked this file as the single source of truth for Data Export API design and implementation tracking. |
|
||
| 2026-04-10 | Consolidated documentation into this file and removed duplicate summary document. |
|
||
|
||
---
|
||
|
||
## 1. Overview
|
||
|
||
The Data Export API allows authorised external systems (data warehouses, Power BI, ArcGIS) to pull mission data from AgMission on demand or on a scheduled basis. It exposes the same data already shown in the web application's **Data Playback** screen, served through a versioned REST API authenticated with API keys.
|
||
|
||
Two functional areas:
|
||
|
||
1. **REST API** (`/api/v1/`) — session summaries, per-point GPS trace, spray-area polygons, async bulk export
|
||
2. **UI enhancement** — improved Job List filter controls (order number, date range) and an API Key management screen in the web app settings
|
||
|
||
---
|
||
|
||
## 2. Architecture
|
||
|
||
### 2.1 Request Flow & Authentication Architecture
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant External as External System
|
||
participant API as Express Server
|
||
participant Auth as checkApiKey Middleware
|
||
participant DB as ApiKey DB
|
||
participant Handler as Route Handler
|
||
|
||
External->>API: GET /api/v1/jobs/:id/sessions
|
||
External->>API: Header X-API-Key
|
||
API->>Auth: req.headers x-api-key
|
||
Auth->>Auth: Extract prefix first 8 chars
|
||
Auth->>DB: Find by prefix active=true
|
||
DB-->>Auth: ApiKey candidates
|
||
Auth->>Auth: bcrypt.compare plainKey vs keyHash
|
||
Auth->>Auth: On match set req.uid
|
||
Auth->>Handler: next with req.uid set
|
||
Handler->>Handler: ownerJob verify req.uid
|
||
Handler-->>External: JSON response
|
||
```
|
||
|
||
**Web UI caller (JWT-authenticated, unchanged):**
|
||
|
||
```mermaid
|
||
graph LR
|
||
A[Web App] -->|Bearer token| B[checkUser Middleware]
|
||
B -->|Verify JWT| C[req.uid set]
|
||
C -->|api/keys routes| D[Key Management]
|
||
D -->|CRUD ops| E[ApiKey Model]
|
||
```
|
||
|
||
### 2.2 Data Model Hierarchy
|
||
|
||
```mermaid
|
||
graph TD
|
||
Job[Job model]
|
||
App[App - Session data]
|
||
AppFile[AppFile - Metadata]
|
||
AppDetail[AppDetail - GPS points]
|
||
ExportJob[ExportJob - Export tracker]
|
||
ApiKey[ApiKey - Authentication]
|
||
|
||
Job -->|has many| App
|
||
App -->|has many| AppFile
|
||
AppFile -->|has many| AppDetail
|
||
Job -.->|triggers| ExportJob
|
||
Job -.->|auth via| ApiKey
|
||
App -.->|derived from| AppDetail
|
||
style Job fill:#e1f5ff
|
||
style App fill:#f3e5f5
|
||
style AppFile fill:#fff3e0
|
||
style AppDetail fill:#fce4ec
|
||
style ExportJob fill:#e8f5e9
|
||
style ApiKey fill:#f1f8e9
|
||
```
|
||
|
||
**Fields summary:**
|
||
- **Job**: jobId, byPuid, rptOp, weatherInfo, sprayAreas
|
||
- **App**: avgSpraySpeed, totalSprayed, totalSprayTime, totalFlightTime
|
||
- **AppFile**: meta (operator, appRate, fcName, sprOnLag), totalSprayed, totalSprayTime
|
||
- **AppDetail**: gpsTime, lat, lon, grSpeed, lminApp, swath, sprayStat, windSpd, temp, humid
|
||
- **ExportJob**: owner, jobId, format, status, filePath, expiresAt
|
||
- **ApiKey**: owner, keyHash, prefix, active, lastUsedAt
|
||
|
||
### 2.3 Route Prefix Strategy
|
||
|
||
| Prefix | Auth | Purpose |
|
||
|---|---|---|
|
||
| `/api/v1/` | `X-API-Key` header (new `checkApiKey`) | Public data export endpoints |
|
||
| `/api/keys` | `Authorization: Bearer` (existing `checkUser`) | Key management for web UI |
|
||
| All other `/api/...` | `Authorization: Bearer` (existing `checkUser`) | Existing application routes — unchanged |
|
||
|
||
The `/api/v1/` path is added to the `checkUser` bypass whitelist (in `isSecuredRoute()`) so the existing JWT middleware skips these routes.
|
||
|
||
### 2.4 Async Export Lifecycle
|
||
|
||
```mermaid
|
||
stateDiagram-v2
|
||
[*] --> pending: POST /export
|
||
pending --> processing: async generate
|
||
processing --> ready: success
|
||
processing --> error: fail I/O error
|
||
ready --> pending: download cleanup
|
||
error --> [*]: TTL expiry
|
||
ready --> [*]: 24h TTL
|
||
pending --> [*]: TTL index
|
||
```
|
||
|
||
**Lifecycle details:**
|
||
- **pending**: ExportJob created and returned to caller; caller polls GET /exports/:id
|
||
- **processing**: Streams AppDetail cursor to CSV or JSON format (memory-efficient)
|
||
- **ready**: File written to disk at filePath, TTL (expiresAt) set and ready for download
|
||
- **error**: Error message recorded, awaits manual retry via queue or TTL cleanup
|
||
- **Cleanup**: After file download completes, filePath cleared and status reset to pending for potential re-download
|
||
|
||
---
|
||
|
||
## 3. New Files
|
||
|
||
### Backend
|
||
|
||
| File | Status | Purpose |
|
||
|---|---|---|
|
||
| `model/api_key.js` | ✅ Done | ApiKey Mongoose model |
|
||
| `model/export_job.js` | ✅ Done | ExportJob tracking model |
|
||
| `middlewares/app_validator.js` | ✅ Done | Added `checkApiKey` function + whitelist entry |
|
||
| `routes/api_pub.js` | ✅ Done | `/api/v1/` route definitions |
|
||
| `routes/api_keys.js` | ✅ Done | `/api/keys` route definitions |
|
||
| `routes/index.js` | ✅ Done | Registers `api_pub` and `api_keys` |
|
||
| `controllers/api_key.js` | ✅ Done | `createKey`, `listKeys`, `revokeKey` |
|
||
| `controllers/api_pub.js` | ✅ Done | `getSessions`, `getSessionRecords`, `getAreas` |
|
||
| `controllers/api_export.js` | ✅ Done | `triggerExport`, `getExportStatus`, `downloadExport` |
|
||
| `scripts/migrate_avg_spray_speed.js` | ✅ Done | One-time back-fill for existing jobs |
|
||
|
||
### Modified Files (existing)
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `model/application.js` | Added `avgSpraySpeed: Number` field |
|
||
| `workers/job_worker.js` | Accumulates `avgSpraySpeed` during file import at lines ~528, ~944–1090, ~1309–1381 |
|
||
|
||
### Frontend (pending)
|
||
|
||
| File | Status | Purpose |
|
||
|---|---|---|
|
||
| `job-list.component.ts/.html` | ⬜ Pending | Add `orderNumber` filter input |
|
||
| `src/app/settings/api-keys/` | ⬜ Pending | API Key management feature module |
|
||
|
||
---
|
||
|
||
## 4. Model Designs
|
||
|
||
### 4.1 ApiKey (`model/api_key.js`)
|
||
|
||
| Field | Type | Notes |
|
||
|---|---|---|
|
||
| `owner` | ObjectId → User | The applicator this key authorises |
|
||
| `label` | String | Human-readable name (max 100 chars) |
|
||
| `prefix` | String | First 8 chars of plain key — stored clear-text for O(1) candidate lookup |
|
||
| `keyHash` | String | `bcryptjs` hash of the full plain key — plain key never stored |
|
||
| `service` | `ApiKeyServices` | Which service the key grants access to: `'data_export'` (default) or `'partner_api'` |
|
||
| `active` | Boolean | Revoke by setting `false` |
|
||
| `managedBy` | `'owner'` \| `'admin'` | Who created the key |
|
||
| `createdAt` | Date | |
|
||
| `lastUsedAt` | Date | Updated async (fire-and-forget) — no added request latency |
|
||
|
||
**Key lookup flow:** `prefix` → find candidates → `bcrypt.compare(incomingKey, candidate.keyHash)` → match → set `req.uid = key.owner`.
|
||
|
||
**Limit:** 10 active keys per owner (enforced in `createKey`).
|
||
|
||
### 4.2 ExportJob (`model/export_job.js`)
|
||
|
||
| Field | Type | Notes |
|
||
|---|---|---|
|
||
| `owner` | ObjectId → User | Scoped to requesting applicator |
|
||
| `jobId` | Number | AgMission job ID |
|
||
| `format` | `'csv'` \| `'json'` | Requested output format |
|
||
| `interval` | Number \| null | GPS point thinning in seconds; `null` = all points |
|
||
| `units` | `ExportUnits` | Output measurement system: `'metric'` (default) or `'us'` |
|
||
| `status` | `'pending'` \| `'processing'` \| `'ready'` \| `'error'` | Lifecycle state |
|
||
| `filePath` | String | Absolute path on disk (set when ready) |
|
||
| `errorMsg` | String | Populated on error |
|
||
| `createdAt` | Date | |
|
||
| `expiresAt` | Date | MongoDB TTL index — document auto-deleted after expiry |
|
||
|
||
Files are written to `env.TEMP_DIR`. TTL defaults to 24 hours (`EXPORT_TTL_HOURS` env var).
|
||
|
||
---
|
||
|
||
## 5. API Endpoint Reference
|
||
|
||
### 5.1 Authentication
|
||
|
||
All `/api/v1/` requests require:
|
||
|
||
```
|
||
X-API-Key: <full 64-char hex key>
|
||
```
|
||
|
||
No `Authorization` header needed. On failure the middleware returns `401`.
|
||
|
||
---
|
||
|
||
### 5.2 `GET /api/v1/jobs/:jobId/sessions`
|
||
|
||
Returns one summary record per uploaded application file ("session") for the job.
|
||
|
||
**Response shape:**
|
||
|
||
```json
|
||
{
|
||
"jobId": 12345,
|
||
"clientId": "664f1a...",
|
||
"clientName": "Fazenda São Paulo Ltda",
|
||
"assignedPilotId": "664f1b...",
|
||
"assignedPilotName": "Carlos Mendes",
|
||
"assignedAircraftId": "664f1a...",
|
||
"assignedAircraftName": "Agrinova 01",
|
||
"assignedAircraftTailNumber": "PR-XYZ",
|
||
"planAircraftName": "Agrinova 01",
|
||
"planAircraftTailNumber": "PR-XYZ",
|
||
"assignedDate": "2025-07-13T18:00:00Z",
|
||
"mappedArea_ha": 50.0,
|
||
"reportConfirmed": true,
|
||
"areaSize_ha": 50.0,
|
||
"coverage_ha": 48.3,
|
||
"overSprayed_pct": -3.40,
|
||
"appRate": 2.5,
|
||
"appRateUnit": "lit/ha",
|
||
"appRateConfirmed": 2.5,
|
||
"sprayVolume": 120.75,
|
||
"volumeUnit": "lit",
|
||
"useConfirmedVolume": false,
|
||
"actualSprayVolume": 118.42,
|
||
"confirmedActualVolume": 120.75,
|
||
"effectiveVolume": 120.75,
|
||
"useCustomWeather": false,
|
||
"weather": null,
|
||
"data": [
|
||
{
|
||
"sessionId": "...",
|
||
"fileName": "2507140724SatlocG4.log",
|
||
"startDateTime": "2025-07-14T10:24:00Z",
|
||
"endDateTime": "2025-07-14T11:05:42Z",
|
||
"totalFlightTime_s": 2462,
|
||
"totalSprayTime_s": 1840,
|
||
"totalTurnTime_s": 622,
|
||
"totalSprayed_ha": 48.3,
|
||
"totalSprayMat": 120.5,
|
||
"totalSprayMatUnit": "lit",
|
||
"avgSpraySpeed_ms": 14.2,
|
||
"sprayZoneName": "Field A North",
|
||
"sprayZoneArea_ha": 25.0,
|
||
"appRate": 2.5,
|
||
"appRateUnit": "lit/ha",
|
||
"flowController": "SatLoc G4",
|
||
"sprayOnLag_s": 0.2,
|
||
"sprayOffLag_s": 0.15,
|
||
"pulsesPerLiter": 1800,
|
||
"files": [{ "fileId": "...", "name": "2507140724SatlocG4.log" }],
|
||
"sessionPilotName": "João Silva"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
Output field definitions (sessions endpoint)
|
||
|
||
Response envelope fields:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `jobId` | number | ✓ | Numeric job identifier from the URL path. |
|
||
| `clientId` | string \| null | — | Client account ObjectId (the applicator's customer this job was performed for). |
|
||
| `clientName` | string \| null | — | Client account name. |
|
||
| `assignedPilotId` | string \| null | — | Assigned pilot ObjectId from the job operator relation. |
|
||
| `assignedPilotName` | string \| null | — | Assigned pilot name from the job operator relation. |
|
||
| `assignedAircraftId` | string \| null | — | Assigned aircraft ObjectId from latest live JobAssign when assignment user is `DEVICE`; otherwise null. |
|
||
| `assignedAircraftName` | string \| null | — | Assigned aircraft display name from latest live JobAssign when assignment user is `DEVICE`; otherwise null. |
|
||
| `assignedAircraftTailNumber` | string \| null | — | Assigned aircraft tail number from latest live JobAssign when assignment user is `DEVICE`; otherwise null. |
|
||
| `planAircraftName` | string \| null | — | Planned aircraft name from `Job.vehicle.name`. Always from the job plan, regardless of live assignment. |
|
||
| `planAircraftTailNumber` | string \| null | — | Planned aircraft tail number from `Job.vehicle.tailNumber`. Always from the job plan. |
|
||
| `assignedDate` | string \| null | — | Latest job assignment timestamp (ISO 8601 UTC). |
|
||
| `mappedArea_ha` | number \| null | — | Job mapped area in hectares from `Job.rptOp.areaSize`, falling back to `Job.ttSprArea`. **2 dp.** |
|
||
| `reportConfirmed` | boolean | ✓ | True when report settings are confirmed (`rptOp.coverage != null`). |
|
||
| `areaSize_ha` | number \| null | — | Confirmed area size, or fallback mapped area when not confirmed. **2 dp.** |
|
||
| `coverage_ha` | number \| null | — | Confirmed coverage, or fallback total sprayed area across sessions. **2 dp.** |
|
||
| `overSprayed_pct` | number \| null | — | `(coverage_ha − areaSize_ha) / areaSize_ha × 100`. **2 dp.** |
|
||
| `appRate` | number \| null | — | Confirmed app rate, or first-session fallback app rate. |
|
||
| `appRateUnit` | string \| null | — | App rate unit label from job setting. |
|
||
| `appRateConfirmed` | number \| null | — | Confirmed app rate only; null when not confirmed. |
|
||
| `sprayVolume` | number \| null | — | Planned/estimated spray volume: `coverage_ha × appRate`, converted by `Job.measureUnit`. **3 dp.** |
|
||
| `volumeUnit` | string \| null | — | Volume unit derived from `Job.measureUnit` and material type: `"lit"` / `"gal"` for liquid, `"kg"` / `"lb"` for solid (dry). |
|
||
| `useConfirmedVolume` | boolean | ✓ | True when applicator selected confirmed actual volume override in Report Settings. |
|
||
| `actualSprayVolume` | number \| null | — | Actual spray volume calculated from applications: `SUM(App.totalSprayMat)` normalized to metric base, then converted by `Job.measureUnit`. **3 dp.** |
|
||
| `confirmedActualVolume` | number \| null | — | Confirmed actual spray volume from `rptOp.actualVol` (stored in metric base: L/Kg), converted by `Job.measureUnit`. **3 dp.** |
|
||
| `effectiveVolume` | number \| null | — | Authoritative volume: `confirmedActualVolume` when `useConfirmedVolume=true`; otherwise `actualSprayVolume`. **3 dp.** |
|
||
| `useCustomWeather` | boolean | ✓ | True when custom weather was manually entered. |
|
||
| `weather` | object \| null | — | Weather block when custom weather exists; otherwise null. |
|
||
| `data` | array | ✓ | Array of per-session summary records. |
|
||
|
||
Per-session fields in `data[]`:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|
|
||
| `sessionId` | string | ✓ | Session identifier (`App._id`). |
|
||
| `fileName` | string \| null | — | Session file name from `App.fileName`. |
|
||
| `startDateTime` | string \| null | — | Session start datetime (ISO 8601 UTC). |
|
||
| `endDateTime` | string \| null | — | Session end datetime (ISO 8601 UTC). |
|
||
| `totalFlightTime_s` | number \| null | — | Total flight time in seconds. **3 dp.** |
|
||
| `totalSprayTime_s` | number \| null | — | Total spray time in seconds. **3 dp.** |
|
||
| `totalTurnTime_s` | number \| null | — | Total turn time in seconds. **3 dp.** |
|
||
| `totalSprayed_ha` | number \| null | — | Total sprayed area in hectares. **2 dp.** |
|
||
| `totalSprayMat` | number \| null | — | Total sprayed material amount. **3 dp.** |
|
||
| `totalSprayMatUnit` | string \| null | — | Spray material unit label (e.g. `"lit"`, `"kg"`) — decoded from raw code via `rateUnitString()`. |
|
||
| `avgSpraySpeed_ms` | number \| null | — | Average spray speed in m/s. **2 dp.** |
|
||
| `sprayZoneName` | string \| null | — | Zone/area name from `AppFile.meta.areaOrZone`. |
|
||
| `sprayZoneArea_ha` | number \| null | — | Zone area in hectares from `AppFile.meta.sprCoverage[1]`. **2 dp.** |
|
||
| `appRate` | number \| null | — | Session target app rate from file metadata. |
|
||
| `appRateUnit` | string \| null | — | App rate unit label from job setting (canonical, matches top-level). |
|
||
| `flowController` | string | — | Flow controller name from file metadata. `'No FC'` when absent or when the value is `'none'` (case-insensitive), matching the playback display. |
|
||
| `sprayOnLag_s` | number \| null | — | Spray-on lag in seconds. |
|
||
| `sprayOffLag_s` | number \| null | — | Spray-off lag in seconds. |
|
||
| `pulsesPerLiter` | number \| null | — | Pulses-per-liter. |
|
||
| `files` | array | ✓ | Session file list: `[{ fileId, name }]`. |
|
||
| `sessionPilotName` | string \| null | — | Pilot name recorded inside the imported data file. May differ from the job-assigned pilot. |
|
||
|
||
**`reportConfirmed` Fallback Logic Diagram:**
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A{Is rptOp.coverage<br/>defined?}
|
||
A -->|Yes| B["reportConfirmed=true"]
|
||
A -->|No| C["reportConfirmed=false"]
|
||
|
||
B --> D["Use Report Settings<br/>values"]
|
||
C --> E["Compute from raw<br/>data"]
|
||
|
||
D --> F{useActualVol?}
|
||
E --> G{useActualVol?}
|
||
|
||
F -->|Yes| H["effective=actual"]
|
||
F -->|No| I["effective=coverage*rate"]
|
||
|
||
G -->|Yes| J["effective=computed"]
|
||
G -->|No| K["effective=computed"]
|
||
|
||
H --> L["Confirmed block"]
|
||
I --> L
|
||
J --> M["Fallback block"]
|
||
K --> M
|
||
```
|
||
|
||
| Field | `reportConfirmed: true` | `reportConfirmed: false` |
|
||
|---|---|---|
|
||
| `areaSize_ha` | `Job.rptOp.areaSize` | `Job.ttSprArea` |
|
||
| `coverage_ha` | `Job.rptOp.coverage` | Sum of `App.totalSprayed` |
|
||
| `appRate` | `Job.rptOp.appRate` | `AppFile.meta.appRate` (first session) |
|
||
| `sprayVolume` | planned `coverage × appRate` | same formula using fallback coverage/appRate values |
|
||
| `effectiveVolume` | `actualVol` if `useActualVol`, else calculated from applications | calculated from applications |
|
||
| weather fields | `Job.weatherInfo.*` when `useCustWI=true` | omitted |
|
||
|
||
> When `reportConfirmed: false`, re-fetch this record after the applicator confirms in Report Settings.
|
||
|
||
---
|
||
|
||
### 5.3 `GET /api/v1/jobs/:jobId/sessions/:fileId/records`
|
||
|
||
Per-point GPS trace records, cursor-paginated. Uses the same `paginateWithCursor` helper as the existing `filesdata_post`.
|
||
|
||
**Query parameters:**
|
||
|
||
| Param | Default | Description |
|
||
|---|---|---|
|
||
| `after` | — | Cursor (`_id` of last record received) — preferred by customer requirements |
|
||
| `startingAfter` | — | Cursor (`_id` of last record received) |
|
||
| `limit` | 500 | Max records per page (hard cap: 2000) |
|
||
| `interval` | — | Return one record per N seconds of GPS time (e.g. `1`, `5`, `10`). Records where `sprayStat` changes are always kept. |
|
||
| `interval` | `null` or `0` | Set `interval=0` (or omit it) to disable interval thinning for full-fidelity results. |
|
||
| `fm` | `false` | Set `fm=true` to include Flight Master/AgDisp FM fields (see below). Off by default — only for customers with FM-enabled equipment. |
|
||
|
||
**Field groups per record:**
|
||
|
||
*GPS Data*: `timeUtc`, `lat`, `lon`, `utmX`, `utmY`, `alt`, `grSpeed`, `heading`, `xTrack`, `lockedLine`, `hdop`, `satsIn`, `tslu`, `calcodeFreq`, `sprayStat`
|
||
|
||
*Application Info*: `flowRateApplied`, `flowRateRequired`, `appRateRequired`, `appRateApplied`*, `swathWidth`, `boomPressure_psi`, `sprayOnLag_s`†, `sprayOffLag_s`†, `pulsesPerLiter`†, `rpm[]`
|
||
|
||
*MET*: `windSpeed_kt`, `windDir_deg`, `temp_c`, `humidity_pct`
|
||
|
||
Compatibility aliases returned by implementation for existing consumers:
|
||
- None — aliases were removed; this is a new API with no existing consumers.
|
||
|
||
Output field definitions (records endpoint)
|
||
|
||
Response envelope fields:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `data` | array | ✓ | Array of per-point records after pagination and optional interval thinning. |
|
||
| `hasMore` | boolean | ✓ | True when additional pages exist. |
|
||
| `startingAfter` | string \| undefined | — | Last record `_id` — pass as `?startingAfter=` to fetch the next page. Present whenever `data` is non-empty. |
|
||
| `endingBefore` | string \| undefined | — | First record `_id` — pass as `?endingBefore=` to fetch the previous page. Present whenever `data` is non-empty. |
|
||
|
||
Per-record fields in `data[]`:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `timeUtc` | string \| null | — | GPS timestamp formatted as ISO 8601 UTC. |
|
||
| `gpsTime` | number \| null | — | Raw GPS epoch seconds. |
|
||
| `lat` | number \| null | — | Latitude (WGS84 decimal degrees). |
|
||
| `lon` | number \| null | — | Longitude (WGS84 decimal degrees). |
|
||
| `utmX` | number \| null | — | UTM X coordinate in meters. |
|
||
| `utmY` | number \| null | — | UTM Y coordinate in meters. |
|
||
| `alt` | number \| null | — | Altitude in meters. |
|
||
| `grSpeed` | number \| null | — | Ground speed in m/s. |
|
||
| `heading` | number \| null | — | Aircraft heading in degrees. |
|
||
| `xTrack` | number \| null | — | Cross-track error in meters. |
|
||
| `lockedLine` | number \| null | — | Locked line index from guidance data. |
|
||
| `hdop` | number \| null | — | Horizontal dilution of precision. |
|
||
| `satsIn` | number \| null | — | Raw satellite/inside-area composite from **AgNav native NT binary**. Encoding is satellite count with inside offset: `0..99` = outside area (satellites = value), `100..199` = inside area (satellites = `value - 100`). |
|
||
| `tslu` | number \| null | — | Raw "time since last update" in seconds for GPS differential correction. |
|
||
| `calcodeFreq` | number \| null | — | Raw calibration/frequency field. 30000-60000 indicates frequency/RPM (true RPM = value - 30000). Also used for spray offset in decimeter: <20000 positive offset; >60000 negative offset where stored value is `65536 - abs(offset)`. |
|
||
| `sprayStat` | number \| null | — | Spray state from source data (returned as-is). **0** = OFF. **1** = ON, inside area. **3** = ON, first point of new spray line (start-of-line marker; boom IS open). **10** = ON, outside area. Any non-zero value = boom open. |
|
||
| `flowRateApplied` | number \| null | — | Applied flow rate (L/min). |
|
||
| `flowRateRequired` | number \| null | — | Required flow rate (L/min). |
|
||
| `appRateRequired` | number \| null | — | Required app rate from source data. |
|
||
| `appRateApplied` | number \| null | — | Playback-aligned app rate applied; null when spray is off. |
|
||
| `swathWidth` | number \| null | — | Swath width in meters. |
|
||
| `boomPressure_psi` | number \| null | — | Boom pressure in PSI. |
|
||
| `sprayOnLag_s` | number \| null | — | Session constant, repeated per record. |
|
||
| `sprayOffLag_s` | number \| null | — | Session constant, repeated per record. |
|
||
| `pulsesPerLiter` | number \| null | — | Session constant. |
|
||
| `rpm` | array \| null | — | RPM array from raw data. |
|
||
| `windSpeed_kt` | number \| null | — | Wind speed in knots (converted from m/s on output to match playback display). |
|
||
| `windDir_deg` | number \| null | — | Wind direction in degrees. |
|
||
| `temp_c` | number \| null | — | Temperature in Celsius. |
|
||
| `humidity_pct` | number \| null | — | Relative humidity percentage. |
|
||
|
||
> Interval thinning rule: keep first record in window, then keep records at least `interval` seconds after last kept record; always keep records where `sprayStat` changes. Use `interval=0` (or omit interval) to bypass thinning.
|
||
> † Session constants from `AppFile.meta` — same value repeated on every record for flat-file consumers.
|
||
|
||
**FM fields** (included only when `?fm=true` is set):
|
||
|
||
| Field | Type | DB source | Description |
|
||
|---|---|---|---|
|
||
| `sprayHeight_m` | number \| null | `sprayHeight` | Target spray height in metres (AgDisp). |
|
||
| `driftX_m` | number \| null | `driftX` | Lateral drift offset X in metres (AgDisp). |
|
||
| `driftY_m` | number \| null | `driftY` | Lateral drift offset Y in metres (AgDisp). |
|
||
| `depositX_m` | number \| null | `depositX` | Deposit offset X in metres (AgDisp). |
|
||
| `depositY_m` | number \| null | `depositY` | Deposit offset Y in metres (AgDisp). |
|
||
| `radarAlt_m` | number \| null | `radarAlt` | Radar altimeter reading in metres. |
|
||
| `laserAlt_m` | number \| null | `raserAlt` ¹ | Laser altimeter reading in metres. |
|
||
|
||
> ¹ The source DB field is named `raserAlt` (schema typo). The API exposes it as `laserAlt_m` with the correct name.
|
||
|
||
**Record Decoding Transformation Pipeline:**
|
||
|
||
```mermaid
|
||
graph LR
|
||
A[Raw AppDetail] --> B[Interval Thinning]
|
||
B --> C[Decode GPS Fields]
|
||
C --> D[Compute appRateApplied]
|
||
D --> E[Inject Session Meta]
|
||
E --> F[Format ISO 8601 UTC]
|
||
F --> G[Return API Record]
|
||
|
||
style A fill:#fce4ec
|
||
style B fill:#f3e5f5
|
||
style C fill:#e8eaf6
|
||
style D fill:#f3e5f5
|
||
style E fill:#e0f2f1
|
||
style F fill:#fff9c4
|
||
style G fill:#c8e6c9
|
||
```
|
||
|
||
**Raw quality-field semantics:**
|
||
- `satsIn`: AgNav native NT binary encoding uses inside offset: `inside = (value >= 100)`, satellites = `inside ? value - 100 : value`
|
||
- `tslu`: time since last update in seconds for GPS differential correction
|
||
- `calcodeFreq`: 30000-60000 indicates frequency/RPM (true RPM = `calcodeFreq - 30000`); also used for spray offset in decimeter (`<20000` positive, `>60000` negative with stored value `65536 - abs(offset)`)
|
||
- `sprayStat` values are returned as stored in source data (no filtering): 0=OFF, 1=ON inside, 3=ON first-point-of-line, 10=ON outside
|
||
- `appRateApplied` = `lminApp / (grSpeed × swath) × 10000`; null when grSpeed or swath = 0
|
||
|
||
---
|
||
|
||
### 5.4 Public API Endpoint Architecture
|
||
|
||
```mermaid
|
||
graph LR
|
||
subgraph External[External Callers]
|
||
PBI[Power BI]
|
||
ARCGIS[ArcGIS]
|
||
DW[Data Warehouse]
|
||
end
|
||
|
||
subgraph PublicAPI[Public API /api/v1]
|
||
SESSIONS[GET /sessions]
|
||
RECORDS[GET /records]
|
||
AREAS[GET /areas]
|
||
TRIGEXP[POST /export]
|
||
POLLEXP[GET /export-status]
|
||
DOWNLOAD[GET /download]
|
||
end
|
||
|
||
subgraph Internal[Backend Models]
|
||
APP[(App)]
|
||
APPFILE[(AppFile)]
|
||
APPDETAIL[(AppDetail)]
|
||
EXPORTJOB[(ExportJob)]
|
||
end
|
||
|
||
PBI --> SESSIONS
|
||
ARCGIS --> AREAS
|
||
DW --> DOWNLOAD
|
||
|
||
SESSIONS --> APP
|
||
RECORDS --> APPDETAIL
|
||
AREAS --> APP
|
||
TRIGEXP --> EXPORTJOB
|
||
POLLEXP --> EXPORTJOB
|
||
DOWNLOAD --> EXPORTJOB
|
||
|
||
SESSIONS --> APPFILE
|
||
RECORDS --> APPFILE
|
||
|
||
style External fill:#e3f2fd
|
||
style PublicAPI fill:#f3e5f5
|
||
style Internal fill:#e8f5e9
|
||
```
|
||
|
||
### 5.5 `GET /api/v1/jobs/:jobId/areas`
|
||
|
||
Returns the planned spray-area polygons as a GeoJSON `FeatureCollection`.
|
||
|
||
Output field definitions (areas endpoint)
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `type` | string | ✓ | Always `FeatureCollection`. |
|
||
| `jobId` | number | ✓ | Numeric job identifier from path. |
|
||
| `features` | array | ✓ | Array of polygon features from planned spray areas. |
|
||
|
||
Per-feature fields in `features[]`:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `type` | string | ✓ | Always `Feature`. |
|
||
| `properties.name` | string \| null | — | Spray area name. |
|
||
| `properties.appRate` | number \| null | — | Planned app rate for the area. |
|
||
| `properties.area_ha` | number \| null | — | Planned area size in hectares. |
|
||
| `properties.type` | string \| null | — | Area type metadata when present. |
|
||
| `geometry` | object \| null | — | GeoJSON polygon geometry copied from `job.sprayAreas`. |
|
||
|
||
> Only implement / expose once customer confirms this is needed for ArcGIS layer import (pending).
|
||
|
||
---
|
||
|
||
### 5.6 Async Export
|
||
|
||
**Trigger:**
|
||
```
|
||
POST /api/v1/jobs/:jobId/export
|
||
Body: { "format": "csv", "interval": 1, "units": "us" }
|
||
→ 202 { "exportId": "...", "status": "pending", "units": "us" }
|
||
```
|
||
|
||
Body parameters:
|
||
| Parameter | Required | Values | Default |
|
||
|---|---|---|---|
|
||
| `format` | Yes | `'csv'`, `'json'` | — |
|
||
| `interval` | No | seconds (e.g. `1`, `5`) | `null` (all points) |
|
||
| `units` | No | `'metric'` (`ExportUnits.METRIC`), `'us'` (`ExportUnits.US`) | `'metric'` |
|
||
| `fm` | No | `true` / `false` | `false` — include Flight Master/AgDisp FM fields |
|
||
|
||
Bulk export interval behavior:
|
||
- Records are read in stable `_id` ascending order per file.
|
||
- With `interval` set, records are thinned by GPS time window.
|
||
- Records where `sprayStat` changes are always included (not thinned out).
|
||
- Thinning is applied per file stream (not across a global merged timeline).
|
||
- For bulk export, omit `interval` (or set `interval=0`) to export all points.
|
||
|
||
**Poll:**
|
||
```
|
||
GET /api/v1/exports/:exportId
|
||
→ { "status": "processing" } (repeat)
|
||
→ { "status": "ready", "downloadUrl": "/api/v1/exports/:id/download" }
|
||
→ { "status": "error", "errorMsg": "..." }
|
||
```
|
||
|
||
**Download:**
|
||
```
|
||
GET /api/v1/exports/:exportId/download
|
||
→ streams file with Content-Disposition: attachment
|
||
```
|
||
|
||
Output field definitions (export endpoints)
|
||
|
||
`POST /api/v1/jobs/:jobId/export` response fields (HTTP 202):
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `exportId` | string | ✓ | Export tracker identifier. |
|
||
| `status` | string | ✓ | Initial export status (`pending`). |
|
||
| `format` | string | ✓ | Selected format (`csv` or `json`). |
|
||
| `units` | string | ✓ | Selected units (`metric` or `us`). |
|
||
| `createdAt` | string | ✓ | Export tracker creation timestamp (ISO 8601 UTC). |
|
||
|
||
`GET /api/v1/exports/:exportId` response fields:
|
||
|
||
| Field | Type | Required | Description |
|
||
|---|---|---|---|
|
||
| `exportId` | string | ✓ | Export tracker identifier. |
|
||
| `status` | string | ✓ | `pending`, `processing`, `ready`, or `error`. |
|
||
| `format` | string | ✓ | Export format. |
|
||
| `units` | string | ✓ | Export units mode. |
|
||
| `createdAt` | string | ✓ | Creation timestamp. |
|
||
| `expiresAt` | string \| null | — | Expiry timestamp for downloaded file cleanup. |
|
||
| `error` | string \| null | — | Error message when generation fails. |
|
||
| `downloadUrl` | string \| undefined | — | Present only when status is `ready`. |
|
||
|
||
`GET /api/v1/exports/:exportId/download` response:
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Body | Streamed file content (CSV or JSON). |
|
||
| `Content-Type` | `text/csv` or `application/geo+json`. |
|
||
| `Content-Disposition` | Attachment filename with format extension. |
|
||
|
||
**CSV structure:** one row per `AppDetail` record. All raw trace fields plus job/session header columns (`jobId`, `orderNumber`, `jobName`, `clientId`, `clientName`, `sessionId`, `fileName`, `pilotName`) repeated on every row — no joins required for Power BI or data warehouse import. Column headers include unit suffix when `units='us'` (e.g. `groundSpeed_mph` vs `groundSpeed_ms`, `temp_f` vs `temp_c`).
|
||
|
||
**US unit conversions** (`units='us'`):
|
||
|
||
| Metric field | US field | Factor |
|
||
|---|---|---|
|
||
| `alt_m` | `alt_ft` | × 3.28084 |
|
||
| `groundSpeed_ms` | `groundSpeed_mph` | × 2.23694 |
|
||
| `crossTrackError_m` | `crossTrackError_ft` | × 3.28084 |
|
||
| `swathWidth_m` | `swathWidth_ft` | × 3.28084 |
|
||
| `flowRateApplied_Lmin` | `flowRateApplied_galMin` | × 0.264172 |
|
||
| `flowRateRequired_Lmin` | `flowRateRequired_galMin` | × 0.264172 |
|
||
| `appRateRequired_Lha` | `appRateRequired_galAc` | × 0.10694 |
|
||
| `appRateApplied_Lha` | `appRateApplied_galAc` | × 0.10694 |
|
||
| `windSpeed_kt` | `windSpeed_mph` | × 1.15078 (kt → mph) |
|
||
| `temp_c` | `temp_f` | × 9/5 + 32 |
|
||
| `boomPressure_psi` | `boomPressure_psi` | already PSI — no conversion |
|
||
|
||
**Implementation:** Node.js `Transform` stream over `AppDetail` cursor (sorted by `_id: 1`) → writes to `env.TEMP_DIR`. Keeps memory flat regardless of file size. `interval` thinning preserves spray-state transition points.
|
||
|
||
---
|
||
|
||
### 5.7 Key Management Endpoints (Web UI, JWT-authenticated)
|
||
|
||
| Method | Path | Description |
|
||
|---|---|---|
|
||
| `GET` | `/api/keys` | List active keys for the signed-in applicator |
|
||
| `POST` | `/api/keys` | Create a key — returns full plain key **once** in the response |
|
||
| `DELETE` | `/api/keys/:keyId` | Revoke a key (sets `active: false`) |
|
||
|
||
**Key management body (`POST /api/keys`):**
|
||
```json
|
||
{ "label": "Power BI Prod", "service": "data_export" }
|
||
```
|
||
`service` is optional and defaults to `'data_export'`. Valid values are defined in `ApiKeyServices` in `helpers/constants.js`.
|
||
|
||
Admin users may append `?ownerId=<ObjectId>` or include `ownerId` in the POST body to manage keys for another account.
|
||
|
||
---
|
||
|
||
## 6. `avgSpraySpeed` — Storage Strategy
|
||
|
||
Rather than computing average spray speed on demand (which would require scanning all `AppDetail` records for every session summary request), it is computed once at **import time** and stored in `App.avgSpraySpeed`.
|
||
|
||
**Accumulation logic in `job_worker.js`:**
|
||
```javascript
|
||
// Per GPS point during file parsing (in importDataFiles, per-file pass):
|
||
// sprayStat=3 is the start-of-line spray marker (boom IS open, but it records
|
||
// the position anchor for area calculation). Excluded from speed averaging
|
||
// because it may capture the low-speed moment of spray transition, which would
|
||
// skew the average spray speed downward.
|
||
if (record.sprayStat !== 3 && record.sprayStat > 0 && utils.isNumber(record.grSpeed)) {
|
||
totalSpeedAcc += record.grSpeed;
|
||
spraySpeedCount++;
|
||
}
|
||
// At end of file:
|
||
importInfo.avgSpraySpeed = spraySpeedCount > 0 ? totalSpeedAcc / spraySpeedCount : null; // m/s
|
||
```
|
||
|
||
**One-time back-fill:** `scripts/migrate_avg_spray_speed.js` — iterates existing `App` docs via cursor, re-scans their `AppDetail` records (`sprayStat > 0`, `grSpeed !== 0`) and bulk-writes the value. It targets apps with missing/null/zero `avgSpraySpeed`. Safe to run on production (cursor-based, low memory, progress logging every 100 docs).
|
||
|
||
---
|
||
|
||
## 7. Frontend Design
|
||
|
||
### 7.1 Job List Filter Enhancement (Step 3 — pending)
|
||
|
||
**File:** `src/app/job/job-list/job-list.component.ts`
|
||
|
||
Add an `orderNumber` text filter control to the existing filter bar alongside client, status, and date pickers. Wire into the existing `Job.Fetch()` NgRx action that calls `jobService.loadJobs()`. Minor backend check: ensure `searchJobs_post` / `getJobs_get` accepts `orderNumber` as a partial-match filter.
|
||
|
||
### 7.2 API Key Management UI (Step 8 — pending)
|
||
|
||
New lazy-loaded feature module following the same NgRx pattern as `PartnerListComponent` / `ClientListComponent`.
|
||
|
||
**Structure:**
|
||
```
|
||
src/app/settings/api-keys/
|
||
api-keys.module.ts
|
||
api-keys-routing.module.ts
|
||
api-keys-list/
|
||
api-keys-list.component.ts
|
||
api-keys-list.component.html
|
||
store/
|
||
api-key.actions.ts
|
||
api-key.reducer.ts
|
||
api-key.effects.ts
|
||
services/
|
||
api-key.service.ts
|
||
```
|
||
|
||
**UX flow:**
|
||
1. PrimeNG `p-table` listing keys — columns: Label, Prefix, Created, Last Used, Status
|
||
2. "Generate Key" button → calls `POST /api/keys` → shows full key in a `p-dialog` with copy-to-clipboard — key masked after dialog is closed, never retrievable again
|
||
3. "Revoke" button per row → `p-confirmDialog` → calls `DELETE /api/keys/:id`
|
||
4. Admin view: additional applicator selector (`p-dropdown`) to manage keys on behalf of any account
|
||
|
||
---
|
||
|
||
## 8. Implementation Status
|
||
|
||
| Step | Feature | Status | Notes |
|
||
|---|---|---|---|
|
||
| 1 | `App.avgSpraySpeed` — model field + import worker + migration script | ✅ Done | `model/application.js`, `workers/job_worker.js`, `scripts/migrate_avg_spray_speed.js` |
|
||
| 2 | `ApiKey` model + `checkApiKey` middleware + `/api/keys` CRUD | ✅ Done | `model/api_key.js`, `middlewares/app_validator.js`, `routes/api_keys.js`, `controllers/api_key.js`. `ApiKeyServices` frozen constant controls valid `service` values. |
|
||
| 3 | Job List UI filter enhancements | ⬜ Pending | Frontend only — `job-list.component` |
|
||
| 4 | `GET /api/v1/jobs/:id/sessions` — session summary | ✅ Done | `controllers/api_pub.js` `getSessions` |
|
||
| 5 | `GET /api/v1/jobs/:id/sessions/:fid/records` — raw trace | ✅ Done | `controllers/api_pub.js` `getSessionRecords` |
|
||
| 6 | `GET /api/v1/jobs/:id/areas` — spray-area GeoJSON | ✅ Done | `controllers/api_pub.js` `getAreas` — awaiting customer confirmation to expose |
|
||
| 7 | Async export (`POST /export`, `GET /exports/:id`, download) | ✅ Done | `model/export_job.js`, `controllers/api_export.js`. `ExportUnits` frozen constant controls valid `units` values; US unit conversions applied at output time. |
|
||
| 8 | API Key management UI (Angular) | ⬜ Pending | New `settings/api-keys` feature module |
|
||
| 9 | Sandbox seeding script | ⬜ Pending | `scripts/seed_sandbox.js` |
|
||
| — | Tests | ⬜ Pending | `checkApiKey` unit tests, session summary integration tests |
|
||
|
||
---
|
||
|
||
## 9. Key Design Decisions
|
||
|
||
| Decision | Rationale |
|
||
|---|---|
|
||
| Separate `checkApiKey` middleware (not extending `checkUser`) | Zero risk to existing JWT-protected routes; `req.uid` set identically so all ownership filters work unchanged |
|
||
| `prefix` stored clear-text in `ApiKey` | O(1) candidate row lookup before expensive `bcrypt.compare`; prefix alone is not usable as a key |
|
||
| `ApiKeyServices` frozen constant for `service` enum | Single source of truth in `helpers/constants.js`; adding a new service type requires editing the constant only — model and controller stay in sync via `Object.values()` |
|
||
| `ExportUnits` frozen constant for `units` enum | Same principle — consistent with project convention for all enumeric text constants |
|
||
| `avgSpraySpeed` stored at import, not computed on demand | Session summary endpoint must never touch `AppDetail` (billion-scale collection); O(1) read from `App` model |
|
||
| Cursor pagination on `AppDetail._id` | Consistent with existing `filesdata_post` pattern; no skip-based offset that degrades on large collections |
|
||
| `interval` thinning on both records endpoint and export | Consistent behaviour; reduces Power BI payload for overview queries; daily batch export at 17:00 can use `interval=1` to shrink CSV size significantly |
|
||
| `reportConfirmed` boolean + always-populated fallback | Consumer's data warehouse always has a usable record; can upsert when field flips to `true` |
|
||
| Async export with TTL (`ExportJob.expiresAt` + MongoDB TTL index) | Files self-clean after 24 hours; no manual housekeeping job needed |
|
||
| CSV columns include job/session header repeated per row | Direct Power BI / warehouse import without requiring a separate join step |
|
||
| Unit conversion at output time, not at storage | Raw data stored in metric throughout; conversion applied in `recordToRow()` with unit-labelled column headers so output is self-documenting |
|
||
|
||
---
|
||
|
||
## 10. Constraints & Notes
|
||
|
||
- All API responses use **metric units by default** (ha, m/s, L/min, L/ha, Kg/ha, °C, metres). Callers may request US customary output via `units: 'us'` on the export endpoint — see Section 5.6.
|
||
- All dates/times are **ISO 8601 UTC strings**.
|
||
- Coordinates are **WGS84 decimal degrees** (EPSG:4326) — numerically equivalent to SIRGAS 2000 (EPSG:4674) for Brazil.
|
||
- `AppDetail.sprayStat === 3` is the **first sprayed point of a new spray line** — the boom IS open at this record. It is written when the spray transitions from OFF to ON (or at the start of a new save target). It also serves as an area anchor (records UTM X/Y, swath, line number) that the import worker uses to compute spray segment coverage. Do NOT exclude `sprayStat = 3` from coverage calculations.
|
||
- `AppDetail.raserAlt` (typo in source schema) is exposed as `laserAlt_m` in the API.
|
||
- `rpm[]` array semantics differ between liquid and dry material types.
|
||
- **Pending:** customer confirmation on whether `GET /api/v1/jobs/:id/areas` (spray-area GeoJSON) is required for their ArcGIS workflow — endpoint is implemented but not yet scheduled for release.
|
||
|
||
---
|
||
|
||
## 11. Canonical Field Reference — DB Source Map
|
||
|
||
Every public `/api/v1` output field mapped to its exact MongoDB source. Use this table as the authoritative reference when debugging a field returning wrong or null values.
|
||
|
||
### 11.1 `/sessions` Envelope Fields
|
||
|
||
| API field | DB model | DB field path | Notes |
|
||
|---|---|---|---|
|
||
| `jobId` | — | URL param `Job.id` (integer) | |
|
||
| `clientId` | `Job` | `client._id` (populated) | |
|
||
| `clientName` | `Job` | `client.name` (populated) | |
|
||
| `mappedArea_ha` | `Job` | `rptOp.areaSize` → fallback `ttSprArea` | `getJobMappedAreaHa()` helper; **2 dp** |
|
||
| `reportConfirmed` | `Job` | `rptOp.coverage != null` | |
|
||
| `areaSize_ha` | `Job` | `rptOp.areaSize` (confirmed) / `ttSprArea` (fallback) | **2 dp** |
|
||
| `coverage_ha` | `Job` / `App[]` | `rptOp.coverage` (confirmed) / `SUM(App.totalSprayed)` (fallback) | **2 dp** |
|
||
| `overSprayed_pct` | derived | `(coverage_ha - areaSize_ha) / areaSize_ha × 100` | null when either area is 0 or null; **2 dp** |
|
||
| `appRate` | `Job` / `AppFile` | `rptOp.appRate` (confirmed) / `AppFile.meta.appRate` first session (fallback) | |
|
||
| `appRateUnit` | `Job` | `appRateUnit` code → `rateUnitString(code, true)` | |
|
||
| `appRateConfirmed` | `Job` | `rptOp.appRate` | null when not confirmed |
|
||
| `sprayVolume` | derived | `coverage_ha × appRate` converted by `Job.measureUnit` | planned estimate; **3 dp** |
|
||
| `volumeUnit` | derived | material type + `Job.measureUnit` → `"lit"/"gal"` (liquid) or `"kg"/"lb"` (solid) | liquid default when `appRateUnit` unset |
|
||
| `useConfirmedVolume` | `Job` | `rptOp.useActualVol` | false when not confirmed |
|
||
| `actualSprayVolume` | `App[]` | `SUM(App.totalSprayMat)` normalized to metric base, then converted by `Job.measureUnit` | null when no application totals; **3 dp** |
|
||
| `confirmedActualVolume` | `Job` | `rptOp.actualVol` converted by `Job.measureUnit` | null when not confirmed or not set; **3 dp** |
|
||
| `effectiveVolume` | derived | `confirmedActualVolume` (if `useConfirmedVolume`) else `actualSprayVolume` | **3 dp** |
|
||
| `useCustomWeather` | `Job` | `useCustWI` | |
|
||
| `weather.windSpeed_kt` | `Job` | `weatherInfo.windSpd` | only when `useCustWI=true` |
|
||
| `weather.windDir` | `Job` | `weatherInfo.windDir` | only when `useCustWI=true` |
|
||
| `weather.temp_c` | `Job` | `weatherInfo.temp` | only when `useCustWI=true` |
|
||
| `weather.humidity_pct` | `Job` | `weatherInfo.humid` | only when `useCustWI=true` |
|
||
| `assignedPilotId` | `Job` | `operator._id` (populated) | |
|
||
| `assignedPilotName` | `Job` | `operator.name` (populated) | |
|
||
| `assignedAircraftId` | `JobAssign` | `user._id` (latest, when `user.kind=DEVICE`) | live workflow assignment traceability |
|
||
| `assignedAircraftName` | `JobAssign` / `Job` | `user.name` when live-assigned; fallback `vehicle.name` | |
|
||
| `assignedAircraftTailNumber` | `JobAssign` / `Job` | `user.tailNumber` when live-assigned; fallback `vehicle.tailNumber` | |
|
||
| `planAircraftName` | `Job` | `vehicle.name` (populated) | always the job-plan aircraft regardless of live assignment |
|
||
| `planAircraftTailNumber` | `Job` | `vehicle.tailNumber` (populated) | always the job-plan aircraft regardless of live assignment |
|
||
| `assignedDate` | `JobAssign` | `date` | latest assignment; sorted by `date desc` |
|
||
|
||
### 11.2 `/sessions` Per-Session `data[]` Fields
|
||
|
||
| API field | DB model | DB field path | Notes |
|
||
|---|---|---|---|
|
||
| `sessionId` | `App` | `_id` | |
|
||
| `fileName` | `App` | `fileName` | |
|
||
| `startDateTime` | `App` | `startDateTime` | ISO 8601 UTC |
|
||
| `endDateTime` | `App` | `endDateTime` | ISO 8601 UTC |
|
||
| `totalFlightTime_s` | `App` | `totalFlightTime` | **3 dp** |
|
||
| `totalSprayTime_s` | `App` | `totalSprayTime` | **3 dp** |
|
||
| `totalTurnTime_s` | `App` | `totalTurnTime` | **3 dp** |
|
||
| `totalSprayed_ha` | `App` | `totalSprayed` | **2 dp** |
|
||
| `totalSprayMat` | `App` | `totalSprayMat` | **3 dp** |
|
||
| `totalSprayMatUnit` | `App` | `totalSprayMatUnit` code → `rateUnitString(code, true, 1)` | decoded to string e.g. `"lit"` |
|
||
| `avgSpraySpeed_ms` | `App` | `avgSpraySpeed` | stored at import time; m/s; **2 dp** |
|
||
| `sprayZoneName` | `AppFile` | `meta.areaOrZone` | first file |
|
||
| `sprayZoneArea_ha` | `AppFile` | `meta.sprCoverage[1]` | first file; **2 dp** |
|
||
| `appRate` | `AppFile` | `meta.appRate` | first file |
|
||
| `appRateUnit` | `Job` | see envelope `appRateUnit` | |
|
||
| `flowController` | `AppFile` | `meta.fcName` | `'No FC'` when absent or `"none"` |
|
||
| `sprayOnLag_s` | `AppFile` | `meta.sprOnLag` | first file |
|
||
| `sprayOffLag_s` | `AppFile` | `meta.sprOffLag` | first file |
|
||
| `pulsesPerLiter` | `AppFile` | `meta.pulsesPerLit` | first file |
|
||
| `files` | `AppFile[]` | `[{ fileId: _id, name }]` | all files for this session |
|
||
| `sessionPilotName` | `AppFile` | `meta.operator` | name as written in the data file; may differ from job-assigned pilot |
|
||
|
||
### 11.3 `/records` Per-Record Fields
|
||
|
||
| API field | DB model | DB field path | Transform |
|
||
|---|---|---|---|
|
||
| `timeUtc` | `AppDetail` | `gpsTime` | epoch-s → ISO 8601 UTC (`toRecordTimeUtc`) |
|
||
| `gpsTime` | `AppDetail` | `gpsTime` | raw |
|
||
| `lat` | `AppDetail` | `lat` | **7 dp** |
|
||
| `lon` | `AppDetail` | `lon` | **7 dp** |
|
||
| `utmX` | `AppDetail` | `utmX` | **1 dp** |
|
||
| `utmY` | `AppDetail` | `utmY` | **1 dp** |
|
||
| `alt` | `AppDetail` | `alt` | m; US: × 3.28084 → ft; **2 dp** |
|
||
| `grSpeed` | `AppDetail` | `grSpeed` | m/s; US: × 2.23694 → mph; **2 dp** |
|
||
| `heading` | `AppDetail` | `head` | degrees; **2 dp** |
|
||
| `xTrack` | `AppDetail` | `xTrack` | m; US: × 3.28084 → ft; **2 dp** |
|
||
| `lockedLine` | `AppDetail` | `llnum` | |
|
||
| `hdop` | `AppDetail` | `stdHdop` | **2 dp** |
|
||
| `satsIn` | `AppDetail` | `satsIn` | raw NT value: `0..99` outside area, `100..199` inside area; satellites = `value` (outside) or `value-100` (inside) |
|
||
| `tslu` | `AppDetail` | `tslu` | raw time since last GPS differential correction update (seconds) |
|
||
| `calcodeFreq` | `AppDetail` | `calcodeFreq` | raw frequency/calibration field; see semantics in section 5.3 |
|
||
| `sprayStat` | `AppDetail` | `sprayStat` | raw (0/1/3/10) |
|
||
| `flowRateApplied` | `AppDetail` | `lminApp` | L/min; US: × 0.264172 → gal/min; **3 dp** |
|
||
| `flowRateRequired` | `AppDetail` | `lminReq` | L/min; US: × 0.264172 → gal/min; **3 dp** |
|
||
| `appRateRequired` | `AppDetail` | `lhaReq` | L/ha; US: × 0.10694 → gal/ac; **2 dp** |
|
||
| `appRateApplied` | derived | `lminApp / (grSpeed × swath) × 10000` | null on zero-division; US: × 0.10694; **2 dp** |
|
||
| `swathWidth` | `AppDetail` | `swath` | m; US: × 3.28084 → ft |
|
||
| `boomPressure_psi` | `AppDetail` | `psi` | already PSI; **2 dp** |
|
||
| `flowController` | `AppFile` | `meta.fcName` | session constant; `'No FC'` when absent |
|
||
| `sprayOnLag_s` | `AppFile` | `meta.sprOnLag` | session constant |
|
||
| `sprayOffLag_s` | `AppFile` | `meta.sprOffLag` | session constant |
|
||
| `pulsesPerLiter` | `AppFile` | `meta.pulsesPerLit` | session constant |
|
||
| `rpm` | `AppDetail` | `rpm` | raw array |
|
||
| `windSpeed_kt` | `AppDetail` | `windSpd` | m/s × 1.94384 → kt; US: m/s × 2.23694 → mph; **2 dp** |
|
||
| `windDir_deg` | `AppDetail` | `windDir` | **1 dp** |
|
||
| `temp_c` | `AppDetail` | `temp` | °C; US: × 9/5 + 32 → °F; **1 dp** |
|
||
| `humidity_pct` | `AppDetail` | `humid` | **1 dp** |
|
||
| `sprayHeight_m` *(fm)* | `AppDetail` | `sprayHeight` | `?fm=true` only |
|
||
| `driftX_m` *(fm)* | `AppDetail` | `driftX` | `?fm=true` only |
|
||
| `driftY_m` *(fm)* | `AppDetail` | `driftY` | `?fm=true` only |
|
||
| `depositX_m` *(fm)* | `AppDetail` | `depositX` | `?fm=true` only |
|
||
| `depositY_m` *(fm)* | `AppDetail` | `depositY` | `?fm=true` only |
|
||
| `radarAlt_m` *(fm)* | `AppDetail` | `radarAlt` | `?fm=true` only |
|
||
| `laserAlt_m` *(fm)* | `AppDetail` | `laserAlt` → `raserAlt` (typo fallback) | `?fm=true` only |
|
||
|
||
### 11.4 `/areas` Feature Properties
|
||
|
||
| API field | DB model | DB field path | Notes |
|
||
|---|---|---|---|
|
||
| `name` | `Job` | `sprayAreas[i].properties.name` | |
|
||
| `appRate` | `Job` | `sprayAreas[i].properties.appRate` | |
|
||
| `area_ha` | `Job` | `sprayAreas[i].properties.area` | polygon-level metadata only; NOT used for session area totals |
|
||
| `type` | `Job` | `sprayAreas[i].properties.type` | e.g. `"area"`, `"xcl"` |
|
||
| `appRateUnit` | `Job` | `appRateUnit` code → `rateUnitString` | |
|
||
| `fallbackAreaHa` | `Job` | `rptOp.areaSize` → `ttSprArea` | envelope-level fallback, not per-polygon |
|
||
| `geometry` | `Job` | `sprayAreas[i].geometry` | copied verbatim |
|
||
|
||
### 11.5 CSV Export Column → DB Source
|
||
|
||
| CSV column (metric) | CSV column (US) | DB model | DB field path | Transform |
|
||
|---|---|---|---|---|
|
||
| `jobId` | same | `ExportJob` | `jobId` | |
|
||
| `orderNumber` | same | `Job` | `orderNumber` | |
|
||
| `jobName` | same | `Job` | `name` | |
|
||
| `clientId` | same | `Job` | `client._id` | |
|
||
| `clientName` | same | `Job` | `client.name` | |
|
||
| `sessionId` | same | `App` | `_id` | |
|
||
| `fileName` | same | `App` | `fileName` | |
|
||
| `pilotName` | same | `AppFile` | `meta.operator` | |
|
||
| `timeUtc` | same | `AppDetail` | `gpsTime` | epoch-s → ISO 8601 UTC |
|
||
| `gpsTime` | same | `AppDetail` | `gpsTime` | raw |
|
||
| `lat` | same | `AppDetail` | `lat` | |
|
||
| `lon` | same | `AppDetail` | `lon` | |
|
||
| `utmX` | same | `AppDetail` | `utmX` | |
|
||
| `utmY` | same | `AppDetail` | `utmY` | |
|
||
| `alt_m` | `alt_ft` | `AppDetail` | `alt` | US: × 3.28084 |
|
||
| `groundSpeed_ms` | `groundSpeed_mph` | `AppDetail` | `grSpeed` | US: × 2.23694 |
|
||
| `heading` | same | `AppDetail` | `head` | |
|
||
| `crossTrackError_m` | `crossTrackError_ft` | `AppDetail` | `xTrack` | US: × 3.28084 |
|
||
| `lockedLine` | same | `AppDetail` | `llnum` | |
|
||
| `hdop` | same | `AppDetail` | `stdHdop` | |
|
||
| `satsIn` | same | `AppDetail` | `satsIn` | raw |
|
||
| `tslu` | same | `AppDetail` | `tslu` | raw |
|
||
| `calcodeFreq` | same | `AppDetail` | `calcodeFreq` | raw |
|
||
| `sprayStat` | same | `AppDetail` | `sprayStat` | raw |
|
||
| `flowRateApplied_Lmin` | `flowRateApplied_galMin` | `AppDetail` | `lminApp` | US: × 0.264172 |
|
||
| `flowRateRequired_Lmin` | `flowRateRequired_galMin` | `AppDetail` | `lminReq` | US: × 0.264172 |
|
||
| `appRateRequired_Lha` | `appRateRequired_galAc` | `AppDetail` | `lhaReq` | US: × 0.10694 |
|
||
| `appRateApplied_Lha` | `appRateApplied_galAc` | derived | `lminApp / (grSpeed × swath) × 10000` | US: × 0.10694 |
|
||
| `swathWidth_m` | `swathWidth_ft` | `AppDetail` | `swath` | US: × 3.28084 |
|
||
| `boomPressure_psi` | same | `AppDetail` | `psi` | |
|
||
| `flowController` | same | `AppFile` | `meta.fcName` | `'No FC'` fallback |
|
||
| `sprayOnLag_s` | same | `AppFile` | `meta.sprOnLag` | |
|
||
| `sprayOffLag_s` | same | `AppFile` | `meta.sprOffLag` | |
|
||
| `pulsesPerLiter` | same | `AppFile` | `meta.pulsesPerLit` | |
|
||
| `rpm` | same | `AppDetail` | `rpm` | JSON-serialised array |
|
||
| `windSpeed_kt` | `windSpeed_mph` | `AppDetail` | `windSpd` | m/s × 1.94384 (kt); US: m/s × 2.23694 (mph) |
|
||
| `windDir_deg` | same | `AppDetail` | `windDir` | |
|
||
| `temp_c` | `temp_f` | `AppDetail` | `temp` | US: × 9/5 + 32 |
|
||
| `humidity_pct` | same | `AppDetail` | `humid` | |
|
||
| `sprayHeight_m` *(fm)* | same | `AppDetail` | `sprayHeight` | |
|
||
| `driftX_m` *(fm)* | same | `AppDetail` | `driftX` | |
|
||
| `driftY_m` *(fm)* | same | `AppDetail` | `driftY` | |
|
||
| `depositX_m` *(fm)* | same | `AppDetail` | `depositX` | |
|
||
| `depositY_m` *(fm)* | same | `AppDetail` | `depositY` | |
|
||
| `radarAlt_m` *(fm)* | same | `AppDetail` | `radarAlt` | |
|
||
| `laserAlt_m` *(fm)* | same | `AppDetail` | `laserAlt` → `raserAlt` | schema typo fallback |
|