agmission/server/docs/DASHBOARD_SNAPSHOT_DESIGN.md

# Dashboard Snapshot Design & Custom Params Pattern

## Overview

The **`GET /api/dashboard/pilot/snapshot`** endpoint solves the **N+1 API problem** on the frontend: instead of calling 5 separate dashboard endpoints, the frontend calls one endpoint and gets a composite response with selected modules.

## Endpoint Signature

```http
GET /api/dashboard/pilot/snapshot?include=kpi,summary,activeJobs,performance,trend&tz=UTC
```

## Query Parameters

### `include` (optional, default: all modules)
Comma-separated list of modules to fetch. Valid values:
- `kpi` — KPI card data (operations + periods)
- `summary` — Today vs yesterday deltas
- `activeJobs` — Job progress panel
- `performance` — XT error and altitude gauges
- `trend` — Trend chart data (hours + hectares over date range)

**Default behavior:** When omitted, returns **all available modules**. Recommended for initial page load.

**Example usage:**
```javascript
// Get only KPI and performance (skip activeJobs/trend)
GET /api/dashboard/pilot/snapshot?include=kpi,performance

// Get all available modules (recommended for initial load)
GET /api/dashboard/pilot/snapshot

// Get trend data for custom date range
GET /api/dashboard/pilot/snapshot?include=trend&startDate=2026-05-01&endDate=2026-05-31
```

### `tz` (optional, default: `UTC`)
IANA timezone string for date calculations (KPI periods, trend dates, etc).

### `startDate`, `endDate` (optional)
Date range for `trend` and `performance` modules (format: `YYYY-MM-DD`).

## Response Shape

Each module is optional in the response based on `include` parameter:

```json
{
  "kpi": {
    "operations": {
      "missionsFlown": 2,
      "distanceTravelledKm": 45.2,
      "distanceSprayedKm": 32.1,
      "sprayEfficiencyPct": 68.40,
      "ferryTimePct": 31.60,
      "flowAccuracyPct": 97.50,
      "avgHdop": 1.20
    },
    "periods": {
      "day": {
        "assignedJobs": 3,
        "assignedHectares": 45.5,
        "sprayedHectares": 32.1,
        "flightHours": 1.53,
        "sprayEfficiencyPct": 68.40,
        "ferryTimePct": 31.60,
        "flowAccuracyPct": 97.50,
        "avgHdop": 1.20,
        "jobCounts": { "new": 0, "inProgress": 1, "completed": 2 }
      },
      "week": { "assignedJobs": 5, "assignedHectares": 120.3, "sprayedHectares": 98.7, "flightHours": 6.15, "sprayEfficiencyPct": 71.00, "ferryTimePct": 29.00, "flowAccuracyPct": 96.25, "avgHdop": 1.15, "jobCounts": {...} },
      "month": { "assignedJobs": 12, "assignedHectares": 450.2, "sprayedHectares": 380.5, "flightHours": 19.73, "sprayEfficiencyPct": 70.50, "ferryTimePct": 29.50, "flowAccuracyPct": 98.10, "avgHdop": 1.08, "jobCounts": {...} },
      "year": { "assignedJobs": 45, "assignedHectares": 1800.5, "sprayedHectares": 1520.2, "flightHours": 82.3, "sprayEfficiencyPct": 69.80, "ferryTimePct": 30.20, "flowAccuracyPct": 97.50, "avgHdop": 1.18, "jobCounts": {...} },
      "all": { "assignedJobs": 120, "assignedHectares": 5200.1, "sprayedHectares": 4850.3, "flightHours": 245.1, "sprayEfficiencyPct": 69.80, "ferryTimePct": 30.20, "flowAccuracyPct": 97.50, "avgHdop": 1.18, "jobCounts": {...} }
    }
  },
  "summary": {
    "today": { "hectares": 32.1, "flightHours": 1.53, "haPerHour": 21.0, "avgSpeedKmh": 45.2, "sprayVolumeLiters": 256 },
    "yesterday": {...},
    "deltas": { "hectaresPct": 15, "flightHoursPct": 10, ... }
  },
  "activeJobs": {
    "jobs": [
      { "jobId": 42, "name": "North Block", "status": 3, "displayStatus": "IN_PROGRESS", "progressPct": 75, ... }
    ]
  },
  "performance": {
    "startDate": "2026-05-19",
    "endDate": "2026-05-25",
    "avgXtError": 2.82,
    "hasXtData": true,
    "xtThreshold": { "good": 1.0, "monitor": 3.0 },
    "avgSprayAltitudeMeters": 3.62,
    "altitudeSource": "sprayHeight",
    "altThreshold": {...},
    "hasAltitudeData": true,
    "sampleSize": 4
  },
  "trend": {
    "labels": ["2026-05-19", "2026-05-20", ...],
    "hoursFlown": [1.53, 2.1, ...],
    "hectaresPerDay": [32.1, 45.5, ...]
  }
}
```

---

## Custom Params Pattern — Best Practice

### Problem
Each dashboard endpoint (`/kpi`, `/summary`, `/trend`, `/performance`) has **independent query params**:
- All accept `tz` (timezone)
- Only `trend` and `performance` accept `startDate`/`endDate`
- Parameters are **not centrally validated** — validation logic lives in each endpoint function

### Solution: No Centralized Param Validator

**Why?** Each endpoint has different requirements:

| Endpoint | Required Params | Optional Params | Logic |
|---|---|---|---|
| `/kpi` | (none) | `tz` | None — period windows are always relative (today, week, month, etc) |
| `/summary` | (none) | `tz` | Compares today vs yesterday |
| `/trend` | (none) | `tz`, `startDate`, `endDate` | Validates date range ≤ 90 days |
| `/performance` | (none) | `tz`, `startDate`, `endDate` | Validates date range ≤ 90 days; defaults to current week |
| `/snapshot` | (none) | `include`, `tz`, `startDate`, `endDate` | Routes params to appropriate sub-modules |

### Implementation Pattern in `/snapshot`

**Each module in snapshot reuses its own validation logic:**

```javascript
async function getSnapshot(req, res) {
  // 1. Parse module list
  const include = parseIncludeList(req.query.include);

  // 2. Fetch shared data once (job/app filter)
  const jobs = await fetchPilotJobs(req.uid);
  const base = appMatch(jobs.map(j => j._id));

  const snapshot = {};

  // 3. For each module, apply its own param validation & logic
  if (include.has('kpi')) {
    const tz = validateTz(req.query.tz);
    // KPI doesn't need startDate/endDate
    snapshot.kpi = buildKpiModule(jobs, base, tz);
  }

  if (include.has('trend')) {
    const tz = validateTz(req.query.tz);
    // Trend DOES need startDate/endDate — validates date range
    validateDateRange(req.query.startDate, req.query.endDate);
    snapshot.trend = buildTrendModule(jobs, base, tz, startDate, endDate);
  }

  // 4. Return only requested modules
  res.json(snapshot);
}
```

### Key Points

1. **Each endpoint owns its params:**
   - No shared validator (each endpoint's logic is self-contained)
   - Snapshot calls each module's validation inline

2. **Shared data fetches:**
   - Pilot's jobs fetched once, not 5 times
   - Application filter (`base`) reused
   - No N+1 database calls

3. **Error handling:**
   - Invalid `include` values → silently ignored (graceful degradation)
   - Invalid date ranges → throw 409 (consistency with individual endpoints)
   - Missing timezone → default to UTC (fallback in validateTz)

---

## Benefits

### Frontend Developer Experience

**Before snapshot:**
```javascript
// 5 separate requests (+ error handling for each)
const kpi        = await fetch('/api/dashboard/pilot/kpi');
const summary    = await fetch('/api/dashboard/pilot/summary');
const trend      = await fetch('/api/dashboard/pilot/trend');
const activeJobs = await fetch('/api/dashboard/pilot/activeJobs');
const perf       = await fetch('/api/dashboard/pilot/performance');

// Load states are complex: each endpoint loads independently
// Error recovery: fail gracefully per module?
// Latency: slowest endpoint dominates (serial or parallel?)
```

**After snapshot:**
```javascript
// 1 request, all modules (or just what you need)
const snapshot = await fetch('/api/dashboard/pilot/snapshot');
// or selective:
const snapshot = await fetch('/api/dashboard/pilot/snapshot?include=kpi,performance');

// Single load state, single error handler
// Latency: one network round-trip + internal parallelism
```

### Backend Performance

1. **Job fetch is 1x, not 5x**
2. **Aggregation queries parallelized** (Promise.all across periods)
3. **Database indexes reused** (same queries as individual endpoints)
4. **Bandwidth:** Omit unused modules with `?include=`

---

## Alternative Patterns (Rejected)

### ❌ Centralized Param Helper
```javascript
function parseCommonParams(req) {
  return { tz: ..., startDate: ..., endDate: ... };
}
```
**Problem:** Not all endpoints use all params. Adds confusion. Snapshot needs to handle missing params per-module anyway.

### ❌ GraphQL-style Query Language
```
POST /api/dashboard/pilot/query
{ query: "{ kpi { operations periods } performance { avgXtError } }" }
```
**Problem:** Overkill for 5 modules. Complexity not justified. REST query params sufficient.

### ❌ POST with JSON body for include list
```
POST /api/dashboard/pilot/snapshot
{ "include": ["kpi", "performance"] }
```
**Problem:** GET is idempotent & cacheable. POST is not. Query params are the right tool.

---

## Testing Snapshot

### Manual test via curl
```bash
# All modules (default)
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?tz=America/Toronto'

# Selective modules
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?include=kpi,performance&tz=UTC'

# Trend with custom date range
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?include=trend&startDate=2026-05-01&endDate=2026-05-31'
```

### Postman collection
Add snapshot test to `Pilot_Dashboard_API.postman_collection.json`:
```javascript
pm.test('snapshot includes requested modules', () => {
  const res = pm.response.json();
  const hasKpi = 'kpi' in res;
  const hasSummary = 'summary' in res;
  pm.expect(hasKpi && hasSummary).to.be.true;
});
```

---

## Future Enhancements

1. **Caching:** Cache snapshot responses per user per hour (dashboard is often static)
2. **Pagination in activeJobs:** Add `?limit=10&offset=0` to performance module
3. **Conditional module fields:** Omit fields when not needed (`?include=kpi:brief` for just operations)
4. **Batch snapshot:** `POST /api/dashboard/pilot/snapshots` with list of pilot IDs (admin view)