agmission/client/docs/ADVANCED_REPORTS_API.md

461 lines
19 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Advanced Reports — API Design Reference
**Version:** 1.0
**Date:** July 9, 2026
**Status:** Draft — contract for Phase 1 implementation (endpoint not yet implemented)
**Scope:** Backend API contract for the Advanced Application Report. This document is the single source of truth for **both backend and frontend/client** development.
**Related Documents:** `ADVANCED_REPORTS_FEASIBILITY.md`, `ADVANCED_REPORTS_FUNCTIONAL.md`, `ADVANCED_REPORTS_NON_FUNCTIONAL.md`, `ADVANCED_REPORTS_IMPLEMENTATION_PLAN.md`, `ADVANCED_REPORTS_PROPOSAL.md`
---
## Table of Contents
- [1 Overview](#1-overview)
- [2 Authentication](#2-authentication)
- [3 Report Generation Flow](#3-report-generation-flow)
- [4 Endpoints](#4-endpoints)
- [4.1 Generate Advanced Report](#41-generate-advanced-report)
- [4.2 Report Options (existing, reused)](#42-report-options-existing-reused)
- [4.3 Save Report Template (existing, reused)](#43-save-report-template-existing-reused)
- [5 Generated Artifacts](#5-generated-artifacts)
- [6 Datasource Contract (`rptDS.json`)](#6-datasource-contract-rptdsjson)
- [7 Error Responses](#7-error-responses)
- [8 Data Model Notes](#8-data-model-notes)
- [9 Frontend Integration Guide](#9-frontend-integration-guide)
- [10 Backend Architecture Notes](#10-backend-architecture-notes)
- [10.1 Generation Data Flow Diagram](#101-generation-data-flow-diagram)
- [10.2 Component Interaction Diagram](#102-component-interaction-diagram)
- [10.3 Map Capture Decision Diagram](#103-map-capture-decision-diagram)
- [10.4 Report Generation Sequence](#104-report-generation-sequence)
- [11 Open Decisions](#11-open-decisions)
- [12 Changelog](#12-changelog)
---
## 1 Overview
The Advanced Application Report is a mission-level, multi-page report (Mission Overview,
Mission Coverage, Zone Detail per zone) generated for a single completed job ("mission").
The server computes all analytics, renders map images, and writes a JSON datasource;
the **client Stimulsoft viewer** renders and exports the report — identical to the legacy
report contract.
**Base path**: `/api/jobs`
**To be implemented in**:
- `controllers/advanced_report.js` (new)
- `helpers/report_util.js` (new — analytics engine)
- `routes/job.js` (new route)
- `public/sprayMap.html` (map page variants)
- `reports/app_advanced.mrt` (authored manually in the embedded Stimulsoft designer)
The legacy endpoints (`/preAppReport`, `/preLoadReport`) are unchanged.
---
## 2 Authentication
Same as the legacy report endpoints. All routes require a valid JWT bearer token; the
`checkUser` middleware is applied globally in `server.js`.
```
Authorization: Bearer <jwt>
```
The `/api/jobs` route group applies the subscription middleware (`checkRqPkgSubscription`),
so the caller must hold an active package. The job must belong to the caller's customer
scope; otherwise `401 not_authorized`.
---
## 3 Report Generation Flow
```
Client (Report Settings dialog)
│ POST /api/jobs/preAdvancedReport { jobId, rptOp, reportContents, ... }
Server
1. Load job + populated refs (client, operator, vehicle, products, crop)
2. Persist report settings onto the job (rptOp incl. reportContents)
3. Stream ApplicationDetail (by the job's fileIds, projected fields)
4. Analytics engine: per-line → per-zone → mission aggregates (one pass over the data)
5. Render map images (one Chromium instance: mission map, zone maps, thumbnails)
6. Write REPORT_DIR/dat/<genFolder>/rptDS.json + map images
7. Select template: app_advanced_<applicatorId>.mrt else app_advanced.mrt
│ 200 { rid, path, c }
Client (Stimulsoft viewer)
GET /reports/<rid>.mrt
GET /reports/dat/<path>/rptDS.json (+ map images referenced within)
→ render, print, export PDF (client-side)
```
Generation is synchronous within the HTTP request. Budget: ~35 s per 10 zones,
~15 s for a typical 3-zone job (NFR-1.1). Repeat exports from an open viewer are
client-side and cost nothing.
---
## 4 Endpoints
### 4.1 Generate Advanced Report
```
POST /api/jobs/preAdvancedReport
```
#### Request Body (JSON)
| Field | Type | Required | Description |
|---|---|---|---|
| `jobId` | number | yes | The job/mission to report on (`Job._id` — numeric auto-increment id) |
| `lang` | string | no | Report language: `en` (default), `pt`, `es` |
| `rptOp` | object | no | Report settings (persisted onto the job, legacy shape) |
| `rptOp.printArea` | boolean | no | Print the planned area size |
| `rptOp.areaSize` | number | no | Planned area (job units; acres converted to ha server-side when `measureUnit` is US) |
| `rptOp.coverage` | number | no | Sprayed area (job units) |
| `rptOp.appRate` | number | no | Application rate override |
| `rptOp.actualVol` | number | no | Actual spray volume |
| `rptOp.useActualVol` | boolean | no | Use `actualVol` instead of computed volume |
| `reportContents` | object | no | **New** — Report Contents selections (persisted with `rptOp`) |
| `reportContents.includeZoneDetail` | boolean | no | Include Zone Detail pages. Default `true` |
| `reportContents.sprayedZonesOnly` | boolean | no | Zone Detail pages only for zones with spray data. Default `false`; ignored when `includeZoneDetail` is `false` |
| `reportContents.includeFlightLineStats` | boolean | no | Include the flight-line table on Zone Detail pages. Default `true` |
| `reportContents.hideMapBackground` | boolean | no | Render all report maps on a plain dark-green background (mockup styling) instead of satellite imagery — smaller files, faster capture. Default `false` |
| `useCustWI` | boolean | no | Use manually entered weather instead of logged averages |
| `weatherInfo` | object | no | Manual weather: `{ windSpd, windDir, temp, humid }` |
#### Example Request Body
```json
{
"jobId": 10234,
"lang": "en",
"rptOp": { "printArea": true, "areaSize": 6681.1, "coverage": 6201.3, "appRate": 10.0, "useActualVol": false },
"reportContents": { "includeZoneDetail": true, "sprayedZonesOnly": false, "includeFlightLineStats": true, "hideMapBackground": false },
"useCustWI": false
}
```
#### Response `200 OK`
```json
{
"rid": "app_advanced",
"path": "appadv_10234_1720537200000",
"c": 0
}
```
| Field | Type | Description |
|---|---|---|
| `rid` | string | Template id — `app_advanced` (default) or `app_advanced_<applicatorId>` (customer-customized) |
| `path` | string | Generated-artifact folder under `REPORT_DIR/dat/` |
| `c` | number | `1` when a customer-customized template was selected, else `0` |
This is the exact `{ rid, path, c }` contract of the legacy `/preAppReport`, so the
existing viewer flow needs no changes beyond calling the new endpoint.
### 4.2 Report Options (existing, reused)
```
POST /api/jobs/reportOps { jobId }
```
Unchanged. Returns coverage / actual volume / area size defaults for pre-filling the
Report Settings dialog (values in ha; client converts per `measureUnit`).
### 4.3 Save Report Template (existing, reused)
```
POST /api/jobs/saveReport
```
Unchanged. The in-product report designer saves an edited template as `<rid>.mrt`;
saving under `app_advanced_<applicatorId>` creates the per-customer override (FR-7.1).
---
## 5 Generated Artifacts
Written to `REPORT_DIR/dat/<path>/` and served from the same static `/reports` path as
legacy report artifacts (hosting sits outside this Express app; non-guessable folder
names are the effective access control, as with legacy reports — see NFR-4.3):
| Artifact | Description |
|---|---|
| `rptDS.json` | Full report datasource (section 6) |
| `map.jpg` | Mission overview map (single-viewport or locator mode, FR-2.3) |
| `zone_<n>.jpg` | Zone Detail map, one per included zone |
| `thumb_<n>.jpg` | Coverage-grid thumbnail; **absent** when zone count > 12 (compact layout, FR-3.5) or cropped from `map.jpg` in single-viewport missions |
Folder names are server-generated and non-guessable; artifacts are retained and later
removed by the separate maintainer app's periodic cleanup (legacy pattern). A repeat
request regenerates into a fresh folder.
---
## 6 Datasource Contract (`rptDS.json`)
All display values are **pre-localized, pre-formatted strings** (units, locale numbers,
local times) — the template renders them exactly as written, with no further processing. Missing/unavailable values are the
em-dash string `""`. Optional sections are suppressed via **empty datasets**, never
empty objects.
```json
{
"reports": { "type": 2 },
"mission": [{
"jobId": 10234,
"name": "Spring Fertilizer 2026",
"jobType": "Fertilizer Application",
"crop": "Corn",
"planDates": "May 22, 2026 - May 22, 2026",
"actualDates": "May 22, 2026, 10:15 AM - 3:57 PM",
"duration": "5h 42m",
"customer": "Greenfield Farms",
"customerAddress": "12 Harvie Road, Barrie, ON",
"pilot": "John Smith",
"licence": "AG-48213-ON",
"aircraft": "Air Tractor AT-802",
"flightNumber": "C-GNAV",
"applicator": "AgMission Aerial Services",
"applicatorAddress": "45 Airport Road, Barrie, ON",
"mapfile": "https://<host>/reports/dat/<path>/map.jpg",
"coveragePct": "95.7%",
"avgSpeed": "143.6 mph",
"avgHeight": "12.3 ft",
"avgXtError": "2.07 ft",
"totalVolume": "12,845 gal",
"zonesSprayed": "5 / 9",
"plannedArea": "6,681.1 ac",
"sprayedArea": "6,201.3 ac",
"totalFlightTime": "5h 42m",
"totalSprayTime": "4h 31m",
"ferryTime": "1h 11m",
"totalDistance": "1,245.2 mi",
"sprayDistance": "903.4 mi",
"ferryDistance": "341.8 mi",
"avgAppRate": "0.50 gal/ac",
"avgFlowRate": "46.8 GPM",
"swathWidth": "60.0 ft",
"remark": "Light crosswind after 14:00; zones 6, 8 and 9 deferred.",
"createdDate": "Jul 9, 2026"
}],
"coverageCards": [{
"zoneNum": 1, "name": "North 40",
"sprayedPlanned": "299.3 / 312.4 ac", "coveragePct": "95.8%",
"thumbFile": "https://<host>/reports/dat/<path>/thumb_1.jpg"
}],
"zones": [{
"zoneNum": 1, "name": "North 40", "crop": "Corn",
"plannedArea": "312.4 ac", "sprayedArea": "299.3 ac", "coveragePct": "95.8%",
"volumeApplied": "625 gal", "avgAppRate": "0.50 gal/ac",
"flightTime": "26m", "sprayTime": "23m", "avgTurnTime": "17.4 s",
"avgSpeed": "145.1 mph", "avgHeight": "12.2 ft",
"avgFlowRate": "46.5 GPM", "avgXtError": "1.90 ft",
"mapfile": "https://<host>/reports/dat/<path>/zone_1.jpg",
"zoneIndexLabel": "Zone 1 of 9"
}],
"lines": [{
"zoneNum": 1, "lineNum": 1, "startTime": "09:15:00",
"sprayTime": "97.2 s", "sprayLength": "4,085 ft", "avgSpeed": "146.1 mph",
"areaCovered": "22.85 ac", "appRate": "0.50 gal/ac",
"avgXtError": "1.80 ft", "turnTime": "17.1 s"
}],
"products": [{
"name": "28-0-0 UAN Blend", "restricted": "No", "epaReg": "",
"rateStr": "0.50 gal/ac", "totalRateStr": "12,845 gal", "count": 1
}],
"weather": [{
"windSpd": "8.6 mph", "windDir": "215° SW", "temp": "21.8°C", "humid": "56%"
}]
}
```
#### Field Notes
- `reports.type``0` planning, `1` legacy application report, **`2` advanced report**.
- `zones[]` is already filtered per `reportContents` (excluded zones don't appear);
`coverageCards[]` always contains **all** zones regardless of filtering.
- Unsprayed zones in `zones[]` carry `""` values, a boundary/ferry-only `mapfile`, and
exactly one `lines[]` placeholder row of `""` cells (FR-4.6).
- `lines[]` is empty when `includeFlightLineStats` is `false` (template band collapses).
- When zone count > 12, `coverageCards[].thumbFile` is `""` and the template renders the
compact text layout (FR-3.5).
- `mission.remark``job.remark` verbatim; `""` when the job has none (Remark line, FR-2.10).
- Temperature is always °C; every other quantity follows the job's `measureUnit` (FR-6.1).
- Mission totals are computed in the same data pass as the zone values — they always
reconcile (NFR-3.3).
---
## 7 Error Responses
Standard AgMission error format:
```json
{ "error": { ".tag": "error_constant_value", "message": "Detail (development mode only)" } }
```
| HTTP Status | `.tag` value | When it occurs |
|---|---|---|
| `401` | `not_authorized` | Missing/invalid JWT, or job not in caller's scope |
| `409` | `job_not_found` | Job does not exist |
| `409` | `invalid_param` | Malformed `jobId`, unknown `lang`, invalid option values |
| `409` | `report_limits_exceeded` *(new)* | Mission exceeds the supported limits: > 50 zones or > 2,000 flight lines (NFR-2.1) |
| `429` | `report_busy` *(new)* | Max concurrent generations (2 per process) reached — client should retry (NFR-2.2) |
| `500` | `report_generation_failed` *(new)* | Mission map capture failed or datasource write failed (zone-map failures degrade to placeholders instead, NFR-3.1) |
---
## 8 Data Model Notes
- **Mission = Job.** `Job._id` is a Number (auto-increment; there is no separate `jobId` field on Job); zones are the `job.sprayAreas` polygon array.
- **Report settings persistence**: `rptOp` (extended with `reportContents`) is saved onto
the job on every request, so the dialog restores the last-used selections per job.
Values arrive in job units and are stored metric (acre→ha conversion server-side when
`measureUnit` is US) — same as legacy.
- **Analytics granularity**: per-line and per-zone values are computed on the fly from
`ApplicationDetail` (read once via a streaming cursor, projected fields, queried by the job's
`fileId`s — the collection's only index). Nothing new is persisted by report generation.
- **Known data gaps** (render as `""`): `lminApp` flat 0 without a flow controller;
SatLoc-imported applications lack xTrack/turn statistics; devices without xTrack
recording have no XT error anywhere.
---
## 9 Frontend Integration Guide
1. Open Report Settings; pre-fill from `POST /reportOps` (existing behaviour).
2. Render the **Report Contents** panel (right side): Include All Zone Detail (default on),
nested Sprayed Zones Only (default off, disabled when parent off), Include Flight Line
Statistics (default on), each with an info tooltip (FR-7.4).
3. On **Preview**: `POST /preAdvancedReport` with the dialog state; show a progress
indicator sized to the NFR-1.1 budget (~1535+ s; consider zone count).
4. Hand `{ rid, path }` to the existing Stimulsoft viewer component unchanged; the viewer
loads the template and datasource and handles print/PDF export client-side.
5. On `report_busy`, offer retry; on `report_limits_exceeded`, surface the zone/line limits.
6. The viewer's `localizeReport()` cultures (en-US / pt-PT / es-ES) are guaranteed present
in `app_advanced.mrt` — no client change needed.
---
## 10 Backend Architecture Notes
### 10.1 Generation Data Flow Diagram
The `ApplicationDetail` records are read **once per report**, and the per-line, per-zone
and mission values are all computed during that single pass over the data. No dataset is
produced by a separate query or code path, so the values always agree with each other
(NFR-1.2, NFR-3.3).
```mermaid
flowchart LR
A["Job by jobId"] --> B["Applications and<br/>AppFiles of the job"]
B --> C["fileId list"]
C --> D["Read ApplicationDetail once<br/>streaming cursor,<br/>projected fields"]
D --> E["Line segmentation<br/>by llnum / sprayStat"]
E --> F["Zone assignment:<br/>point-in-polygon<br/>vs job.sprayAreas"]
F --> G["Per-line<br/>stats"]
G --> H["Zone<br/>roll-ups"]
H --> I["Mission<br/>totals"]
G --> J["rptDS<br/>lines dataset"]
H --> K["rptDS zones and<br/>coverageCards datasets"]
I --> L["rptDS<br/>mission dataset"]
```
### 10.2 Component Interaction Diagram
```mermaid
flowchart TD
FE["Frontend:<br/>Report Settings dialog"] --> EP["POST /api/jobs/<br/>preAdvancedReport"]
EP --> CTL["controllers/<br/>advanced_report.js"]
CTL --> RU["helpers/report_util.js<br/>analytics engine"]
CTL --> WU["helpers/web_util.js<br/>single shared Chromium"]
WU --> SM["public/sprayMap.html<br/>variants"]
CTL --> FS[("REPORT_DIR/dat/genFolder:<br/>rptDS.json + map images")]
CTL --> TPL{"customer template<br/>app_advanced_applicatorId.mrt<br/>exists?"}
TPL -->|yes| C1["rid = customized<br/>c = 1"]
TPL -->|no| C0["rid = app_advanced<br/>c = 0"]
RU --> J[("jobs")]
RU --> AP[("applications")]
RU --> AF[("application_files")]
RU --> AD[("application_details")]
FE2["Stimulsoft viewer"] --> MRT["GET /reports/<br/>rid.mrt"]
FE2 --> DS["GET /reports/dat/path/<br/>rptDS.json + images"]
```
### 10.3 Map Capture Decision Diagram
Capture count follows the effective page selection, not the zone count (NFR-1.3).
```mermaid
flowchart TD
A["Start captures:<br/>one shared browser"] --> B{"Zones fit legibly<br/>in one viewport?"}
B -->|yes| C["Mission map:<br/>full polygons"]
B -->|no| D["Mission map:<br/>locator badges<br/>(FR-2.3.2)"]
C --> E{"More than<br/>12 zones?"}
D --> E
E -->|yes| F["Skip thumbnails:<br/>compact layout<br/>(FR-3.5)"]
E -->|no| G{"Single<br/>viewport?"}
G -->|yes| H["Crop thumbnails from<br/>the mission capture"]
G -->|no| I["Per-zone<br/>thumbnail captures"]
F --> K{"includeZoneDetail?"}
H --> K
I --> K
K -->|yes| L["Zone map capture per<br/>included zone<br/>(sprayedZonesOnly filter)"]
K -->|no| M["No zone<br/>captures"]
```
### 10.4 Report Generation Sequence
```mermaid
sequenceDiagram
participant FE as Frontend
participant API as Jobs API
participant DB as MongoDB
participant CH as Shared Chromium
participant FS as REPORT_DIR
FE->>API: POST preAdvancedReport<br/>jobId, rptOp, reportContents
API->>DB: Load job and its related records,<br/>persist report settings
API->>DB: Read ApplicationDetail once<br/>by fileIds, streaming cursor
DB-->>API: Points aggregated to<br/>line, zone, mission values
API->>CH: Render mission map,<br/>thumbnails, zone maps (10.3)
CH-->>API: JPEG captures<br/>(zone-map failure = placeholder)
API->>FS: Write rptDS.json + images<br/>to dat/genFolder
API-->>FE: 200 rid, path, c
FE->>FS: GET template .mrt,<br/>rptDS.json + images
Note over FE: Viewer renders.<br/>Print and PDF export client-side
```
Concurrency: a simple in-process counter caps generation at 2 concurrent requests
(`429 report_busy` beyond that, NFR-2.2). The generation function is isolated from the
HTTP layer so it can later move behind the existing worker framework unchanged (NFR-2.3).
---
## 11 Open Decisions
| # | Decision | Status |
|---|---|---|
| 1 | Page orientation (portrait-only vs landscape variant) — affects template only, not this API | Awaiting PO (F-OQ-1) |
| 2 | Regeneration reuse/caching for unchanged repeat requests (same `{rid, path}` returned) | Deferred — out of Phase 1 scope; API shape already compatible |
| 3 | Exact `.tag` strings for the new error constants (`helpers/constants.js` naming review) | To be finalized during D2 implementation |
| 4 | Compact coverage layout threshold — exact rule (more than 12 vs 12 and above) and threshold value; affects when `coverageCards[].thumbFile` is empty | Awaiting PO (F-OQ-2) |
---
## 12 Changelog
| Version | Date | Notes |
|---|---|---|
| 1.0 | 2026-07-09 | Initial draft — contract derived from the approved Phase 1 planning set (feasibility, FR, NFR, implementation plan) |
| 1.1 | 2026-07-13 | `mission.remark` added to `mission[]` (Remark line on page 1, FR-2.10). Overview-map zone/field names (FR-2.3 rev.) — map rendering only, no contract impact |