pujas 045ce34398 feat: 2026/08/21 - Advanced Reports added

2026-08-21 10:20:52 -04:00

9.6 KiB

Raw Blame History

Dashboard Snapshot Design & Custom Params Pattern

Overview

The GET /api/dashboard/pilot/snapshot endpoint solves the N+1 API problem on the frontend: instead of calling 5 separate dashboard endpoints, the frontend calls one endpoint and gets a composite response with selected modules.

Endpoint Signature

GET /api/dashboard/pilot/snapshot?include=kpi,summary,activeJobs,performance,trend&tz=UTC

Query Parameters

`include` (optional, default: all modules)

Comma-separated list of modules to fetch. Valid values:

kpi — KPI card data (operations + periods)
summary — Today vs yesterday deltas
activeJobs — Job progress panel
performance — XT error and altitude gauges
trend — Trend chart data (hours + hectares over date range)

Default behavior: When omitted, returns all available modules. Recommended for initial page load.

Example usage:

// Get only KPI and performance (skip activeJobs/trend)
GET /api/dashboard/pilot/snapshot?include=kpi,performance

// Get all available modules (recommended for initial load)
GET /api/dashboard/pilot/snapshot

// Get trend data for custom date range
GET /api/dashboard/pilot/snapshot?include=trend&startDate=2026-05-01&endDate=2026-05-31

`tz` (optional, default: `UTC`)

IANA timezone string for date calculations (KPI periods, trend dates, etc).

`startDate`, `endDate` (optional)

Date range for trend and performance modules (format: YYYY-MM-DD).

Response Shape

Each module is optional in the response based on include parameter:

{
  "kpi": {
    "operations": {
      "missionsFlown": 2,
      "distanceTravelledKm": 45.2,
      "distanceSprayedKm": 32.1,
      "sprayEfficiencyPct": 68.40,
      "ferryTimePct": 31.60,
      "flowAccuracyPct": 97.50,
      "avgHdop": 1.20
    },
    "periods": {
      "day": {
        "assignedJobs": 3,
        "assignedHectares": 45.5,
        "sprayedHectares": 32.1,
        "flightHours": 1.53,
        "sprayEfficiencyPct": 68.40,
        "ferryTimePct": 31.60,
        "flowAccuracyPct": 97.50,
        "avgHdop": 1.20,
        "jobCounts": { "new": 0, "inProgress": 1, "completed": 2 }
      },
      "week": { "assignedJobs": 5, "assignedHectares": 120.3, "sprayedHectares": 98.7, "flightHours": 6.15, "sprayEfficiencyPct": 71.00, "ferryTimePct": 29.00, "flowAccuracyPct": 96.25, "avgHdop": 1.15, "jobCounts": {...} },
      "month": { "assignedJobs": 12, "assignedHectares": 450.2, "sprayedHectares": 380.5, "flightHours": 19.73, "sprayEfficiencyPct": 70.50, "ferryTimePct": 29.50, "flowAccuracyPct": 98.10, "avgHdop": 1.08, "jobCounts": {...} },
      "year": { "assignedJobs": 45, "assignedHectares": 1800.5, "sprayedHectares": 1520.2, "flightHours": 82.3, "sprayEfficiencyPct": 69.80, "ferryTimePct": 30.20, "flowAccuracyPct": 97.50, "avgHdop": 1.18, "jobCounts": {...} },
      "all": { "assignedJobs": 120, "assignedHectares": 5200.1, "sprayedHectares": 4850.3, "flightHours": 245.1, "sprayEfficiencyPct": 69.80, "ferryTimePct": 30.20, "flowAccuracyPct": 97.50, "avgHdop": 1.18, "jobCounts": {...} }
    }
  },
  "summary": {
    "today": { "hectares": 32.1, "flightHours": 1.53, "haPerHour": 21.0, "avgSpeedKmh": 45.2, "sprayVolumeLiters": 256 },
    "yesterday": {...},
    "deltas": { "hectaresPct": 15, "flightHoursPct": 10, ... }
  },
  "activeJobs": {
    "jobs": [
      { "jobId": 42, "name": "North Block", "status": 3, "displayStatus": "IN_PROGRESS", "progressPct": 75, ... }
    ]
  },
  "performance": {
    "startDate": "2026-05-19",
    "endDate": "2026-05-25",
    "avgXtError": 2.82,
    "hasXtData": true,
    "xtThreshold": { "good": 1.0, "monitor": 3.0 },
    "avgSprayAltitudeMeters": 3.62,
    "altitudeSource": "sprayHeight",
    "altThreshold": {...},
    "hasAltitudeData": true,
    "sampleSize": 4
  },
  "trend": {
    "labels": ["2026-05-19", "2026-05-20", ...],
    "hoursFlown": [1.53, 2.1, ...],
    "hectaresPerDay": [32.1, 45.5, ...]
  }
}

Custom Params Pattern — Best Practice

Problem

Each dashboard endpoint (/kpi, /summary, /trend, /performance) has independent query params:

All accept tz (timezone)
Only trend and performance accept startDate/endDate
Parameters are not centrally validated — validation logic lives in each endpoint function

Solution: No Centralized Param Validator

Why? Each endpoint has different requirements:

Endpoint	Required Params	Optional Params	Logic
`/kpi`	(none)	`tz`	None — period windows are always relative (today, week, month, etc)
`/summary`	(none)	`tz`	Compares today vs yesterday
`/trend`	(none)	`tz`, `startDate`, `endDate`	Validates date range ≤ 90 days
`/performance`	(none)	`tz`, `startDate`, `endDate`	Validates date range ≤ 90 days; defaults to current week
`/snapshot`	(none)	`include`, `tz`, `startDate`, `endDate`	Routes params to appropriate sub-modules

Implementation Pattern in `/snapshot`

Each module in snapshot reuses its own validation logic:

async function getSnapshot(req, res) {
  // 1. Parse module list
  const include = parseIncludeList(req.query.include);

  // 2. Fetch shared data once (job/app filter)
  const jobs = await fetchPilotJobs(req.uid);
  const base = appMatch(jobs.map(j => j._id));

  const snapshot = {};

  // 3. For each module, apply its own param validation & logic
  if (include.has('kpi')) {
    const tz = validateTz(req.query.tz);
    // KPI doesn't need startDate/endDate
    snapshot.kpi = buildKpiModule(jobs, base, tz);
  }

  if (include.has('trend')) {
    const tz = validateTz(req.query.tz);
    // Trend DOES need startDate/endDate — validates date range
    validateDateRange(req.query.startDate, req.query.endDate);
    snapshot.trend = buildTrendModule(jobs, base, tz, startDate, endDate);
  }

  // 4. Return only requested modules
  res.json(snapshot);
}

Key Points

Each endpoint owns its params:
- No shared validator (each endpoint's logic is self-contained)
- Snapshot calls each module's validation inline
Shared data fetches:
- Pilot's jobs fetched once, not 5 times
- Application filter (base) reused
- No N+1 database calls
Error handling:
- Invalid include values → silently ignored (graceful degradation)
- Invalid date ranges → throw 409 (consistency with individual endpoints)
- Missing timezone → default to UTC (fallback in validateTz)

Benefits

Frontend Developer Experience

Before snapshot:

// 5 separate requests (+ error handling for each)
const kpi        = await fetch('/api/dashboard/pilot/kpi');
const summary    = await fetch('/api/dashboard/pilot/summary');
const trend      = await fetch('/api/dashboard/pilot/trend');
const activeJobs = await fetch('/api/dashboard/pilot/activeJobs');
const perf       = await fetch('/api/dashboard/pilot/performance');

// Load states are complex: each endpoint loads independently
// Error recovery: fail gracefully per module?
// Latency: slowest endpoint dominates (serial or parallel?)

After snapshot:

// 1 request, all modules (or just what you need)
const snapshot = await fetch('/api/dashboard/pilot/snapshot');
// or selective:
const snapshot = await fetch('/api/dashboard/pilot/snapshot?include=kpi,performance');

// Single load state, single error handler
// Latency: one network round-trip + internal parallelism

Backend Performance

Job fetch is 1x, not 5x
Aggregation queries parallelized (Promise.all across periods)
Database indexes reused (same queries as individual endpoints)
Bandwidth: Omit unused modules with ?include=

Alternative Patterns (Rejected)

❌ Centralized Param Helper

function parseCommonParams(req) {
  return { tz: ..., startDate: ..., endDate: ... };
}

Problem: Not all endpoints use all params. Adds confusion. Snapshot needs to handle missing params per-module anyway.

❌ GraphQL-style Query Language

POST /api/dashboard/pilot/query
{ query: "{ kpi { operations periods } performance { avgXtError } }" }

Problem: Overkill for 5 modules. Complexity not justified. REST query params sufficient.

❌ POST with JSON body for include list

POST /api/dashboard/pilot/snapshot
{ "include": ["kpi", "performance"] }

Problem: GET is idempotent & cacheable. POST is not. Query params are the right tool.

Testing Snapshot

Manual test via curl

# All modules (default)
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?tz=America/Toronto'

# Selective modules
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?include=kpi,performance&tz=UTC'

# Trend with custom date range
curl -H "Authorization: Bearer $TOKEN" \
  'https://localhost:4100/api/dashboard/pilot/snapshot?include=trend&startDate=2026-05-01&endDate=2026-05-31'

Postman collection

Add snapshot test to Pilot_Dashboard_API.postman_collection.json:

pm.test('snapshot includes requested modules', () => {
  const res = pm.response.json();
  const hasKpi = 'kpi' in res;
  const hasSummary = 'summary' in res;
  pm.expect(hasKpi && hasSummary).to.be.true;
});

Future Enhancements

Caching: Cache snapshot responses per user per hour (dashboard is often static)
Pagination in activeJobs: Add ?limit=10&offset=0 to performance module
Conditional module fields: Omit fields when not needed (?include=kpi:brief for just operations)
Batch snapshot: POST /api/dashboard/pilot/snapshots with list of pilot IDs (admin view)

9.6 KiB Raw Blame History