agmission/Development/server/docs/PILOT_DASHBOARD_BACKEND_SUMMARY.md

# Pilot Dashboard Backend Summary

> **Superseded** — This document was the initial design analysis and planning doc.
> The definitive API specification (including frontend integration guide) is now in:
> **[PILOT_DASHBOARD_API.md](./PILOT_DASHBOARD_API.md)**
>
> The content below is retained for historical context (original requirements analysis,
> open decisions log, and delivery plan).

---

## Document Info
- Scope: Backend support for Pilot Analytics Dashboard (Phase 1)
- Source inputs: Product Owner brief v1.5 and Requirements Analysis v1.5
- Validation status: aligned with current server codebase (models, constants, routes, middleware)
- Date: 2026-04-29

## 1. Objective
Build backend APIs and aggregation logic for a pilot-only analytics dashboard, using existing AgMission data (Job, Application, Application_Detail), with strict role/data isolation and production-safe query patterns.

## 2. Current Backend Baseline (Verified)

### 2.1 What already exists
- Pilot CRUD/search routes exist at /api/pilots via routes/pilot.js and controllers/pilot.js.
- Global route registration follows function-based mounting in routes/index.js.
- Auth middleware checkUser is applied globally before route mounting in server.js.
- Standard centralized error handling is already in place.

### 2.2 What does not exist yet
- No dashboard analytics controller/routes for pilots.
- No endpoints currently serving KPI/summary/trend/active-jobs/performance payloads.
- No implemented status transition endpoint for Mark as Completed.

### 2.3 Key model and constant facts
- Job status constants exist in helpers/job_constants.js:
  - NEW 0, READY 1, DOWNLOADED 2, SPRAYED 3, COMPLETED 4, INVOICED 5, ARCHIVED 9.
- Job pilot assignment is by Job.operator (optional ObjectId).
- Job ownership is by Job.byPuid (Applicator account).
- Aircraft display source is Job.vehicle.
- Job_Assign is assignment workflow data and should not be used as pilot-job source of truth for dashboard metrics.
- Application links to job by numeric jobId.
- Application aggregates needed for dashboard already exist: totalSprayed, totalFlightTime, totalSprLength, totalSprayMat, avgSpraySpeed.
- Application_Detail contains xTrack, sprayHeight, radarAlt, gpsAlt and has primary index on fileId.

## 3. Confirmed Design Direction So Far

### 3.1 Pilot scoping model
The only reliable pilot scope for dashboard metrics is:
1. Resolve jobIds from Job where operator = current pilot user id.
2. Aggregate Application records where jobId is in that list.
3. For quality gauges, resolve recent app files and aggregate Application_Detail by fileId.

This avoids wrong assumptions around uploader identity and keeps logic aligned with how pilot assignment is represented.

### 3.2 Dashboard modules (backend perspective)
- KPI cards: assigned jobs/ha + sprayed today + flight hours today.
- Daily summary: today vs yesterday deltas for hectares, hours, ha/hr, speed, volume.
- Operations today: distance and spray volume totals.
- Active jobs panel: per-job progress and actual applied volume with status mapping.
- Trend: daily hours and hectares over date range (default current week).
- Performance indicators: average XT error and altitude, with threshold bands.
- Mark Job as Completed: manual SPRAYED to COMPLETED transition.

## 4. Proposed API Surface (Phase 1)

Suggested new route group:
- /api/dashboard/pilot

Suggested endpoints:
- GET /api/dashboard/pilot/kpi
- GET /api/dashboard/pilot/summary
- GET /api/dashboard/pilot/trend
- GET /api/dashboard/pilot/active-jobs
- GET /api/dashboard/pilot/performance

Job completion action:
- PATCH /api/jobs/:jobId/complete

Implementation notes:
- Keep endpoint naming camelCase where needed in path segments to match existing project conventions.
- Reuse existing auth middleware and error classes for consistency.

## 5. Query and Aggregation Approach

### 5.1 KPI and summary
- Base filter: Job.find({ operator: pilotId, markedDelete: { $ne: true } }).
- Read assigned hectares from Job.ttSprArea.
- Aggregate Applications by jobId with time-window filtering for today/yesterday.
- Convert units in backend responses:
  - hours = seconds / 3600
  - distance = meters / 1000
  - speed = m/s to km/h when needed

### 5.2 Active jobs
- Start from pilot-scoped jobs.
- Aggregate per job from Application:
  - haSprayed = sum totalSprayed
  - volumeApplied = sum totalSprayMat
- Compute progressPct = min(100, max(0, haSprayed / haTotal * 100)).
- Status display mapping:
  - NEW (0): no progress bar
  - READY/DOWNLOADED/SPRAYED: IN PROGRESS behavior
  - COMPLETED (4): full bar

### 5.3 Trend
- Accept startDate/endDate (max 90 days recommended).
- Default to current calendar week (Mon-Sun) in user timezone policy.
- Group by day and fill missing dates with zero values.

### 5.4 Performance
- Use recent application files only (for bounded cost).
- Fetch recent pilot-scoped Application records, then map to related fileIds.
- Query Application_Detail with fileId IN [...].
- Compute:
  - avgXtErrorMeters = avg(abs(xTrack))
  - altitude with source priority sprayHeight then radarAlt
- Return no-data state when sample is empty or sensor fields unavailable.

## 6. Authorization and Data Isolation

### 6.1 Pilot dashboard endpoints
- Must be pilot-only.
- Always derive pilotId from authenticated user context, never trust client-provided user ids.
- Never expose other pilots or global Applicator data.

### 6.2 Mark as Completed endpoint
- Allowed transition: SPRAYED (3) to COMPLETED (4) only.
- Recommended permission for Phase 1: Applicator owner only (Job.byPuid match).
- Return 409 for invalid state transition.
- Return 403 when caller is not authorized.

## 7. Performance and Scalability Considerations

- Application_Detail is large-scale; avoid broad scans.
- Always drive quality queries by fileId-scoped subsets.
- Keep performance endpoint sample bounded (for example, last 10 files as proposed).
- Ensure lean reads for read-only queries where practical.
- Add/validate indexes only where query plans prove necessary after measurement.

## 8. Key Product Decisions Still Open (Affect Backend)

- Active Jobs cutoff status set (exact statuses to include).
- Assigned Jobs KPI business meaning (all-time vs open vs season).
- Altitude fallback policy when sprayHeight/radarAlt is absent.
- Whether pilot can trigger completion action or Applicator-only.
- Final timezone policy for Today/Yesterday windows (requirements currently point to browser timezone).

## 9. Recommended Delivery Plan

### Phase A - Foundation
- Add dashboard route/controller/service skeleton.
- Add shared pilot scope resolver (pilot to jobIds).
- Add consistent response DTO contracts.

### Phase B - Core endpoints
- Implement kpi, summary, trend, active-jobs.
- Add input validation for date range/timezone params.
- Add empty-state and zero-safe calculations.

### Phase C - Performance indicators
- Implement performance endpoint with bounded recent-file strategy.
- Add explicit source label and no-data states.

### Phase D - Job completion workflow
- Implement PATCH complete endpoint with strict state and auth checks.
- Add/update JSDoc for API docs.

### Phase E - Verification
- Create tests scripts in tests/ for each endpoint and transition scenarios.
- Validate role isolation, response shapes, and edge cases (no jobs, no apps, missing altitude).
- Run scripts and record execution output.

## 10. Risks and Mitigations

- Risk: Missing Job.operator on some jobs leads to undercount.
  - Mitigation: surface this as known data-quality dependency in API/docs.

- Risk: Inconsistent device coverage for altitude metrics.
  - Mitigation: deterministic source priority + explicit no-data response.

- Risk: Heavy Application_Detail scans.
  - Mitigation: strict fileId-scoped querying and bounded sample windows.

- Risk: Frontend status mismatch for COMPLETED/INVOICED labels.
  - Mitigation: coordinate backend constants and frontend enum alignment before rollout.

## 11. Definition of Done for Backend

- Pilot dashboard endpoints implemented and documented.
- Mark Complete transition endpoint implemented with auth and state guards.
- APIs return stable contracts for all normal and empty-data scenarios.
- Endpoint behavior validated via executed test scripts.
- Relevant docs updated in docs/ and JSDoc included for apidoc generation.