agmission/Development/server
Devin Major df31b2080d
All checks were successful
Server Tests / Mocha – Unit & Utility Tests (push) Successful in 42s
-(#3013) Data Export - Implement Data Export API BE (Cont.)
+ Added public data export API enhancements, tests, and customer documentation
  + Extended /api/v1 data export endpoints with richer session, records, area, and async export output
  + Added confirmed/fallback report values, client metadata, mapped area, over-spray, volume/apprate (string) units, and weather blocks
  + Normalized flowController to "No FC" and align record field names with playback output
  + Converted record wind speed output to knots, add Fligh Mater only record/export fields behind fm=true, and persist fm on export jobs
  + Added export status/area constants, HTTP 202 support, route-level API docs, and per-account export rate limiting support
  + Added comprehensive endpoint, format, and verification test coverage plus test-suite README
  + Added customer-facing data export design, integration, rate-limit, and documentation index guides
  + Updated README/DLQ docs and related documentation links to current HTTPS dashboard paths
2026-04-24 09:05:55 -04:00
..
.github copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
.vscode copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
apidoc first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
controllers -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
docs -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
emails first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
helpers -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
job-uploads copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
locales first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
middlewares copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
model -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
public first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
reports first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
routes -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
scripts -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
services copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
tests -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
workers copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
.eslintrc.json first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
.npmrc copy of data-export-api branch as of April 22 2026 2026-04-22 15:12:27 -04:00
apidoc.json first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
db-scripts.js first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
package-lock.json all changes from April 23 2026 2026-04-23 14:24:54 -04:00
package.json all changes from April 23 2026 2026-04-23 14:24:54 -04:00
PINO_MODULE_FILTERING_GUIDE.md first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
README_PARTNER_INTEGRATION.md first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
README.md -(#3013) Data Export - Implement Data Export API BE (Cont.) 2026-04-24 09:05:55 -04:00
server.js first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
setup_partners.js first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
start_workers.js first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00
test_satloc_pattern_brief.js first commit (copy of Trunk as of April 22 2026) 2026-04-22 15:00:02 -04:00

AgMission Server

📚 Documentation Index | 🔄 Partner Integration | 📝 DLQ System

Environment keys

Syntax Description
AGM_PORT BackEnd endpoint port
PRODUCTION Whether runnung in production mode
DEBUG debug pattern. E.g:
AGM_PORT BackEnd endpoint port DEBUG=agm:*
STRIPE_SECRET_KEY Stripe secret key
STRIPE_PUBLISHABLE_KEY Stripe publishable key
STRIPE_API_VERSION Stripe API Version
STRIPE_WH_SEC Stripe Webhooks Endpoint secret
[Package]_[Number] Stripe package in uppercased 3-first character and number. E.g: ESS_1 for essential package 1

Partner Integration Environment Keys

Syntax Description
SATLOC_API_ENDPOINT SatLoc API base URL
SATLOC_API_TIMEOUT Request timeout in milliseconds
SATLOC_RETRY_ATTEMPTS Number of API retry attempts
SATLOC_RATE_LIMIT API requests per second limit
PARTNER_SYNC_INTERVAL Partner data sync interval (ms)
PARTNER_HEALTH_CHECK_INTERVAL Health check interval (ms)
PARTNER_MAX_CONCURRENT_JOBS Max concurrent partner jobs
PARTNER_ENCRYPT_CREDENTIALS Whether to encrypt partner credentials
QUEUE_NAME_PARTNER Partner task queue name (auto-prefixes with 'dev_' in development, defaults to 'partner_tasks' in production)
PARTNER_MAX_RETRIES Maximum retry attempts before DLQ (default: 5)
DLQ_CHECK_INTERVAL DLQ monitoring interval (default: 5 min)

Note: Partner authentication now uses customer-specific credentials stored in PartnerSystemUser records, not global environment variables.

For complete partner integration setup, see README_PARTNER_INTEGRATION.md

Debug Stripe Webhook

Prerequisites

  1. Install Stripe CLI: https://stripe.com/docs/stripe-cli
  2. Make sure your server is running locally
  3. Have your Stripe account credentials ready

Setup Steps

  1. Login to Stripe CLI

    stripe login
    
  2. Forward webhook events to your local server

    # Forward to local development server (default port 3000)
    stripe listen --forward-to https://localhost:4100/stripe_webhooks --skip-verify
    
    # Or specify custom port if your AGM_PORT is different
    stripe listen --forward-to localhost:YOUR_AGM_PORT/webhook/stripe
    
  3. Get the webhook signing secret When you run the stripe listen command, it will output a webhook signing secret like:

    > Ready! Your webhook signing secret is whsec_1234567890abcdef...
    
  4. Update your environment variables Add or update the webhook secret in your .env file:

    STRIPE_WH_SEC=whsec_1234567890abcdef...
    
  5. Enable debug logging Set the DEBUG environment variable to see webhook processing logs:

    DEBUG=agm:* npm start
    # Or specifically for webhook debugging:
    DEBUG=agm:webhook,agm:stripe npm start
    

Testing Webhook Events

  1. Trigger test events from Stripe CLI

    # Test subscription created event
    stripe trigger invoice.payment_succeeded
    
    # Test subscription updated event
    stripe trigger customer.subscription.updated
    
    # Test payment failed event
    stripe trigger invoice.payment_failed
    
    # Test subscription cancelled event
    stripe trigger customer.subscription.deleted
    
  2. Create test events from Stripe Dashboard

    • Go to your Stripe Dashboard
    • Navigate to Developers > Webhooks
    • Click "Send test webhook"
    • Select the event type you want to test

Debug Output

When debugging is enabled, you should see output like:

agm:webhook Received Stripe webhook: invoice.payment_succeeded
agm:stripe Processing subscription payment for customer: cus_...
agm:db -> MongoDB connected - Main Application ready

Common Issues

  1. Webhook signature verification failed

    • Make sure STRIPE_WH_SEC matches the secret from stripe listen
    • Ensure the webhook endpoint path is correct (/webhook/stripe)
  2. Connection refused

    • Verify your server is running on the correct port
    • Check that the --forward-to URL matches your server address
  3. No webhook events received

    • Confirm stripe listen is still running
    • Check that events are being sent to the correct endpoint
    • Verify your webhook endpoint is accessible

Production Webhook Setup

For production, configure webhooks directly in your Stripe Dashboard:

  1. Go to Developers > Webhooks
  2. Click "Add endpoint"
  3. Set endpoint URL: https://yourdomain.com/webhook/stripe
  4. Select events to listen for
  5. Copy the signing secret to your production STRIPE_WH_SEC environment variable

DLQ (Dead Letter Queue) Monitoring

The DLQ system provides global queue-native operations for managing failed tasks across all queue types.

Documentation

Complete DLQ documentation: docs/DLQ_INDEX.md

Quick links:

Web Dashboard

https://localhost:4100/dlq-monitor.html

Features:

  • Real-time DLQ statistics for all queues
  • View failed messages with error details
  • One-click retry operations
  • Queue selection dropdown
  • Auto-refresh every 30 seconds

Quick API Examples

View DLQ Messages:

curl http://localhost:4100/api/dlq/partner_tasks/messages?limit=20 \
  -H "Authorization: Bearer $TOKEN"

Retry All Messages:

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxMessages": 50}'

Multiple Queue Support:

# Partner queue
POST /api/dlq/partner_tasks/retryAll

# Job queue
POST /api/dlq/dev_jobs/retryAll

# Future queues (no code changes needed!)
POST /api/dlq/notifications/retryAll

API Endpoints

Global DLQ endpoints (work for ANY queue):

  • GET /api/dlq/:queueName/messages - View DLQ messages
  • GET /api/dlq/:queueName/stats - Get DLQ statistics
  • POST /api/dlq/:queueName/retryAll - Retry all messages
  • POST /api/dlq/:queueName/retryByPosition - Retry by position range
  • POST /api/dlq/:queueName/retryByHeader - Retry by header match
  • DELETE /api/dlq/:queueName/purge - Purge queue ⚠️

For complete API documentation, see docs/DLQ_API_REFERENCE.md

Automated Processing

Use the web dashboard at https://localhost:4100/dlq-monitor.html or API endpoints:

# Retry all DLQ messages
curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxMessages": 100}'