agmission/Documents/DEPLOYMENT.md

# AgMission Deployment Guide

**Last updated:** April 2026

---

## Table of Contents

- [1 Prerequisites](#1-prerequisites)
- [2 Repository and Directory Layout](#2-repository-and-directory-layout)
- [3 Deployment Script Reference](#3-deployment-script-reference)
- [4 Configuration](#4-configuration)
- [5 Deployment Modes](#5-deployment-modes)
- [6 Step-by-Step Production Deployment](#6-step-by-step-production-deployment)
- [7 PM2 Process Management](#7-pm2-process-management)
- [8 Backup and Recovery](#8-backup-and-recovery)
- [9 Environment Variables Reference](#9-environment-variables-reference)

---

## 1 Prerequisites

**Developer machine (deploying from):**

- SSH access to the production server on port `22222`
- `rsync` installed
- SVN working copy at `~/work/AgMission`
- AGN private libraries at `~/work/@agn`
- Angular CLI for frontend builds (`npm install -g @angular/cli`)

**Production server:**

- Node.js 16.20.2 LTS
- PM2 (global): `npm install -g pm2`
- MongoDB 4.4.x replica set (`rs0`)
- RabbitMQ 3.10.7+
- Redis
- Nginx 1.10.x+

---

## 2 Repository and Directory Layout

### Local developer machine

```
~/work/AgMission/
├── trunk/Development/      ← main line source
├── branches/<branch-name>/ ← feature branches
└── tags/release-x.y.z/    ← release snapshots

~/work/@agn/                ← private AGN Node.js libraries
```

### Remote production server

```
/home/agm/apps/
├── agmission/              ← API server (server/)
├── client/dist/            ← Angular build output
├── gps-server/
├── track-server/
├── satloc/
├── maintainer/
├── pm2-apps/               ← PM2 JSON configs
└── @agn/                   ← private libs (rsync'd from ~/work/@agn)
```

---

## 3 Deployment Script Reference

**Script:** `trunk/Others/scripts/deploy/agm-deploy.sh`

### Usage

```bash
./agm-deploy.sh [run_mode] [branch_name] [fe_mode]
```

| Argument | Values | Default | Description |
|---|---|---|---|
| `run_mode` | `0`, `1`, `2` | `0` | Deployment scope (see table below) |
| `branch_name` | `trunk`, `main`, or branch name | `subscription-invoicing` | Source to deploy from |
| `fe_mode` | `run`, `dry-run` | `run` | Frontend deploy behaviour (mode 2 only) |

| `run_mode` | Backend | Frontend | Use case |
|---|---|---|---|
| `0` (or empty) | Dry run | Dry run | Verify what would be synced |
| `1` | Deploy | Skipped | Backend-only update |
| `2` | Deploy | Deploy | Full production deployment |

### Configuration file recommended

```bash
# 1. Copy template
cp trunk/Others/scripts/deploy/agm-deploy.conf.template ~/.agm-deploy.conf

# 2. Edit values
nano ~/.agm-deploy.conf

# 3. Use it
source ~/.agm-deploy.conf && ./agm-deploy.sh 1 trunk
```

### Environment variable overrides

| Variable | Default | Description |
|---|---|---|
| `AGM_BASE_DIR` | `~/work/AgMission` | Local AgMission SVN root |
| `AGN_LIBS_DIR` | `~/work/@agn` | Local AGN private libraries |
| `AGM_DEST_HOST` | `agm@agmission-1.agnav.com` | SSH target (`user@host`) |
| `AGM_DEST_PORT` | `22222` | SSH port |
| `AGM_DEST_PATH` | `/home/agm/apps` | Remote base directory |

---

## 4 Configuration

### Excluded files

`trunk/Others/scripts/deploy/excludes.txt` lists paths excluded from `rsync`. These typically include:

- `node_modules/`
- `.env` / `environment*.env` (never overwrite production secrets)
- `.tmp/`, `job-unzip/`, `job-uploads/`
- Log files and rlog crash reports

### Production environment files

Environment files are **not** deployed by the script. They must be maintained manually on the production server. See [9 Environment Variables Reference](#9-environment-variables-reference) for key variables.

---

## 5 Deployment Modes

### Dry run mode 0

```bash
./agm-deploy.sh 0 trunk
```

Shows all files that would be synced without touching the server. Always run this first when deploying after a significant code change.

### Backend only mode 1

```bash
./agm-deploy.sh 1 trunk
```

Deploys:
- `server/` → `/home/agm/apps/agmission/`
- `gps-server/` → `/home/agm/apps/gps-server/`
- `track-server/` → `/home/agm/apps/track-server/`
- `shared/` → `/home/agm/apps/shared/`
- `@agn` libraries → `/home/agm/apps/@agn/`

After sync, run `pm2 reload` on the server (see [7 PM2 Process Management](#7-pm2-process-management)).

### Full deployment mode 2

```bash
./agm-deploy.sh 2 trunk
```

Deploys everything in mode 1, plus the Angular frontend `client/dist/` to the Nginx webroot.

**Before running mode 2**, build the frontend locally:

```bash
cd trunk/Development/client
npm install
npm run build-prod     # builds all locales: en, pt, es
```

---

## 6 Step-by-Step Production Deployment

```mermaid
graph TD
    A["1. Update SVN working copy<br/>svn update trunk/"] --> B["2. Run tests<br/>cd server && npm test"]
    B --> C["3. Build frontend<br/>cd client && npm run build-prod"]
    C --> D["4. Dry run<br/>./agm-deploy.sh 0 trunk"]
    D --> E{"Review OK?"}
    E -- No --> F["Fix issues"]
    F --> D
    E -- Yes --> G["5. Deploy backend<br/>./agm-deploy.sh 1 trunk"]
    G --> H["6. SSH to server<br/>ssh agm@agmission-1.agnav.com -p 22222"]
    H --> I["7. Install dependencies<br/>cd /home/agm/apps/agmission && npm install --production"]
    I --> J["8. Reload PM2<br/>pm2 reload agmission-prod"]
    J --> K["9. Check logs<br/>pm2 logs agmission-prod --lines 50"]
    K --> L{"Errors?"}
    L -- Yes --> M["Rollback: pm2 reload with previous snapshot"]
    L -- No --> N["10. Deploy frontend (if needed)<br/>./agm-deploy.sh 2 trunk"]
```

### Post-deployment checklist

- [ ] `pm2 status` — all processes show `online`
- [ ] `pm2 logs agmission-prod --lines 20` — no crash errors
- [ ] Health check endpoint: `curl https://agmission-1.agnav.com/api/health`
- [ ] Login to the web UI and confirm dashboard loads
- [ ] If partner workers were updated: `pm2 reload partner_sync_worker partner_data_polling_worker`
- [ ] If worker code changed: check `pm2 logs job_worker --lines 20`

---

## 7 PM2 Process Management

### Start all services first time

```bash
# SSH into server
ssh agm@agmission-1.agnav.com -p 22222

# Start all configured apps
/home/agm/apps/pm2-apps/start_pm2_apps.sh
```

### Reload after deployment

```bash
# Reload individual app (zero-downtime)
pm2 reload agmission-prod

# Reload all at once
pm2 reload all

# Check status
pm2 status
pm2 logs agmission-prod --lines 30
```

### Process list

```bash
pm2 list
```

Expected processes:

| Name | Status |
|---|---|
| `agmission-prod` | online |
| `track_server` | online |
| `gps_server-agnav` | online |
| `gps_server-rap` | online |
| `job_worker` | online |
| `invoice_worker` | online |
| `cleanup_worker` | online |
| `partner_sync_worker` | online |
| `partner_data_polling_worker` | online |

### Restart a single worker

```bash
pm2 restart job_worker
pm2 restart partner_sync_worker
pm2 restart invoice_worker
```

### Save current PM2 process list

```bash
pm2 save
```

---

## 8 Backup and Recovery

### Automated backup

`trunk/Others/scripts/backup_agm.sh` runs via cron. It:

1. Dumps MongoDB to a gzip archive (`mongodump --gzip`)
2. Retains 11 days of database archives
3. Syncs archives to NAS via `rsync` (`rsync://rsync@data.agnav.com/agm/`)
4. Syncs uploaded job files separately
5. Syncs rsync log files

### Manual MongoDB backup

```bash
mongodump \
  --archive=/home/agm/backups/manual_$(date +%Y%m%d).gz \
  --gzip \
  --db agmission \
  --username agm \
  --authenticationDatabase agmission
```

### Restore from backup

```bash
mongorestore \
  --archive=/home/agm/backups/agmdb_YYYYMMDD.gz \
  --gzip \
  --db agmission \
  --username agm \
  --authenticationDatabase agmission \
  --drop
```

---

## 9 Environment Variables Reference

Environment files live on the production server at `/home/agm/apps/agmission/environment.env` and are **not** managed by the deployment script.

### Core server

| Variable | Description |
|---|---|
| `AGM_PORT` | HTTP port the API server listens on (e.g., `7000`) |
| `PRODUCTION` | `true` in production |
| `DB_USR` / `DB_PWD` / `DB_NAME` | MongoDB credentials |
| `DB_HOSTS` | MongoDB host list (replica set) |
| `DB_REPLSET` | Replica set name (`rs0`) |
| `REDIS_PWD` | Redis password |
| `JWT_SECRET` | JWT signing secret |
| `MAX_SESSION_SECS` | Session token lifetime (default: 28800) |

### Stripe billing

| Variable | Description |
|---|---|
| `STRIPE_SECRET_KEY` | Stripe secret key |
| `STRIPE_PUBLISHABLE_KEY` | Stripe publishable key |
| `STRIPE_API_VERSION` | Stripe API version |
| `STRIPE_WH_SEC` | Stripe webhook signing secret |
| `ESS_1` … `ESS_5` | Stripe price IDs for Essential plans |
| `ENT_1` … `ENT_4` | Stripe price IDs for Enterprise plans |
| `ADDON_1` | Stripe price ID for add-on |

### File storage

| Variable | Description |
|---|---|
| `UPLOAD_DIR` | Job upload directory |
| `UNZIP_DIR` | Temp extraction directory |
| `INV_UPLOAD_DIR` | Invoice image upload directory |
| `INV_IMG_VIR_DIR` | Virtual URL path for invoice images |
| `SATLOC_STORAGE_PATH` | Local storage for downloaded SatLoc log files |

### RabbitMQ

| Variable | Description |
|---|---|
| `QUEUE_HOST` | RabbitMQ host |
| `QUEUE_PORT` | RabbitMQ port (default: 5672) |
| `QUEUE_USR` / `QUEUE_PWD` | RabbitMQ credentials |
| `QUEUE_VHOST` | Virtual host |
| `QUEUE_NAME_JOBS` | Job processing queue name |
| `QUEUE_NAME_GDATA` | GPS data queue name |
| `QUEUE_NAME_PARTNER` | Partner tasks queue name |

### Partner integration

| Variable | Description |
|---|---|
| `SATLOC_API_ENDPOINT` | SatLoc Cloud base URL |
| `SATLOC_API_TIMEOUT` | HTTP request timeout (ms) |
| `PARTNER_SYNC_INTERVAL` | Partner sync interval (ms) |
| `PARTNER_HEALTH_CHECK_INTERVAL` | Health check interval (ms) |
| `PARTNER_MAX_RETRIES` | Max retry attempts before DLQ |
| `DLQ_CHECK_INTERVAL` | DLQ monitoring interval (ms) |

---

*See [ARCHITECTURE.md](ARCHITECTURE.md) for full system design documentation.*