agmission/Development/server/scripts/MIGRATION_QUICK_REFERENCE.md

# Customer Data Migration - Quick Reference

## Quick Start

### 1. Preview Migration (Always Do This First!)

```bash
# Using usernames (recommended - more readable)
node scripts/migrateCustomerData.js \
  --preview \
  --sources acme_aviation,regional_ag \
  --destination consolidated_ag

# Using IDs (also supported)
node scripts/migrateCustomerData.js \
  --preview \
  --sources <SOURCE_ID1>,<SOURCE_ID2> \
  --destination <DEST_ID>

# Mixed: usernames and IDs work together
node scripts/migrateCustomerData.js \
  --preview \
  --sources acme_aviation,507f1f77bcf86cd799439012 \
  --destination consolidated_ag
```

### 2. Execute After Reviewing Preview

```bash
# Using usernames
node scripts/migrateCustomerData.js \
  --sources acme_aviation,regional_ag \
  --destination consolidated_ag

# Using IDs
node scripts/migrateCustomerData.js \
  --sources <SOURCE_ID1>,<SOURCE_ID2> \
  --destination <DEST_ID>
```

## Common Scenarios

### Scenario 1: Simple Single Customer Migration

**Use Case**: Merge one customer into another

```bash
# Step 1: Preview (using usernames - easier to read)
node scripts/migrateCustomerData.js \
  --preview \
  --sources acme_aviation \
  --destination consolidated_ag

# Step 2: Execute
node scripts/migrateCustomerData.js \
  --sources acme_aviation \
  --destination consolidated_ag

# Alternative: Using IDs
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011 \
  --destination 507f191e810c19729de860ea
```

**Result**: 
- Source customer deactivated
- All data moved to destination
- Aborts if username conflicts detected

---

### Scenario 2: Consolidate Multiple Regional Accounts

**Use Case**: Merge multiple regional accounts into corporate account

```bash
# Using usernames (recommended)
node scripts/migrateCustomerData.js \
  --sources acme_west,acme_east,acme_south \
  --destination acme_corporate

# Using IDs
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011,507f1f77bcf86cd799439012,507f1f77bcf86cd799439013 \
  --destination 507f191e810c19729de860ea
```

**Result**:
- All 3 regional accounts deactivated
- All data consolidated under corporate account
- Regional managers can be converted to admins (see Scenario 4)

---

### Scenario 3: Merge with Known Conflicts

**Use Case**: Known username overlaps, want to merge accounts

```bash
# Preview first to see conflicts (using usernames)
node scripts/migrateCustomerData.js \
  --preview \
  --sources acme_aviation \
  --destination consolidated_ag

# Execute with merge if conflicts are acceptable
node scripts/migrateCustomerData.js \
  --sources acme_aviation \
  --destination consolidated_ag \
  --merge-conflicts
```

**Result**:
- Conflicting accounts merged (old accounts marked deleted)
- Job/data references updated to existing accounts
- Old account data preserved in history

⚠️ **WARNING**: Review conflicts carefully before merging!

---

### Scenario 4: Convert Customers to Admin Accounts

**Use Case**: Keep source customer as admin user under destination

```bash
# Using usernames
node scripts/migrateCustomerData.js \
  --sources acme_west,acme_east \
  --destination acme_corporate \
  --convert-to-admin

# Using IDs
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011,507f1f77bcf86cd799439012 \
  --destination 507f191e810c19729de860ea \
  --convert-to-admin
```

**Result**:
- Source customers become admin (kind='2') accounts under destination
- All sub-accounts and data moved to destination
- Original customer accounts deactivated but admin access preserved

**Perfect For**:
- Regional managers becoming corporate admins
- Acquired companies maintaining some autonomy
- Franchise consolidation

---

### Scenario 5: Data Only Migration (Skip Invoices)

**Use Case**: Migrate operational data but keep invoices separate

```bash
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011 \
  --destination 507f191e810c19729de860ea \
  --skip-invoices
```

**Result**:
- All data migrated except invoice customer references
- Useful for financial separation requirements
- Invoices remain linked to original customer

---

### Scenario 6: Custom History File Location

**Use Case**: Store migration logs in specific location

```bash
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011 \
  --destination 507f191e810c19729de860ea \
  --output-file /var/log/agmission/migrations/customer_migration_2025.json
```

**Result**:
- Migration history saved to custom location
- Useful for compliance/audit requirements

---

## Decision Tree

```
Do you want to migrate customer data?
│
├─ Single source customer?
│  ├─ Yes → Use Scenario 1
│  └─ No → Continue
│
├─ Multiple source customers?
│  ├─ Yes → Use Scenario 2
│  └─ No → Continue
│
├─ Known username conflicts?
│  ├─ Yes → Use Scenario 3 (with --merge-conflicts)
│  └─ No → Continue
│
├─ Keep source customers as admins?
│  ├─ Yes → Use Scenario 4 (with --convert-to-admin)
│  └─ No → Continue
│
├─ Need to preserve invoice separation?
│  ├─ Yes → Use Scenario 5 (with --skip-invoices)
│  └─ No → Use Scenario 1 or 2
│
└─ Always preview first with --preview!
```

## Conflict Resolution Strategies

### Strategy 1: Manual Resolution (RECOMMENDED)

**Before migration**:
1. Run preview to identify conflicts
2. Manually rename conflicting accounts in source OR destination
3. Run migration without conflicts

```bash
# Example: Rename client in source database
db.users.updateOne(
  { _id: ObjectId("507f1f77bcf86cd799439015") },
  { $set: { username: "john_farmer_regional" } }
)

# Then run migration
node scripts/migrateCustomerData.js --sources ... --destination ...
```

### Strategy 2: Automatic Merge

**During migration**:
- Use `--merge-conflicts` flag
- Old accounts marked as deleted
- References updated to existing accounts

```bash
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011 \
  --destination 507f191e810c19729de860ea \
  --merge-conflicts
```

### Strategy 3: Selective Migration

**Migrate in phases**:
1. Migrate non-conflicting customers first
2. Resolve conflicts for remaining customers
3. Migrate remaining customers

```bash
# Phase 1: No conflicts
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439011 \
  --destination 507f191e810c19729de860ea

# Phase 2: After resolving conflicts
node scripts/migrateCustomerData.js \
  --sources 507f1f77bcf86cd799439012 \
  --destination 507f191e810c19729de860ea
```

## Verification Steps

### After Migration

1. **Check Migration History**
   ```bash
   cat migration_history.json | jq '.[-1]'
   ```

2. **Verify Source Customer Deactivated**
   ```javascript
   db.users.findOne({ 
     _id: ObjectId("507f1f77bcf86cd799439011"),
     kind: "1"
   })
   // Should have: markedDelete: true, active: false
   ```

3. **Verify Job Migration**
   ```javascript
   db.jobs.countDocuments({ 
     byPuid: ObjectId("507f191e810c19729de860ea") 
   })
   // Should equal total jobs from source + existing destination jobs
   ```

4. **Verify Sub-Account Migration**
   ```javascript
   db.users.countDocuments({ 
     parent: ObjectId("507f191e810c19729de860ea"),
     kind: { $in: ["3", "5", "9"] }
   })
   // Should equal expected count
   ```

5. **Check for Errors in History**
   ```bash
   cat migration_history.json | jq '.[-1].stats.errors'
   ```

## Troubleshooting Common Issues

### Issue: "Source customer(s) not found"
```bash
# Verify customer exists
mongo
> use agmission
> db.users.findOne({ _id: ObjectId("YOUR_ID"), kind: "1" })
```

### Issue: "Migration aborted due to conflicts"
```bash
# View conflicts in detail
node scripts/migrateCustomerData.js --preview --sources ... --destination ...
# Then use Strategy 1 or 2 above
```

### Issue: Transaction timeout
```bash
# Reduce number of source customers per migration
# Instead of migrating 10 at once, do 2-3 at a time
```

### Issue: Need to rollback
```bash
# MongoDB transactions auto-rollback on error
# If completed but need to reverse:
# 1. Restore from backup
# OR
# 2. Manually reverse using migration_history.json details
```

## Environment Variables

```bash
# Enable detailed debugging
DEBUG=agm:migrate-customer-data,agm:mongo_enhanced \
  node scripts/migrateCustomerData.js ...

# Custom MongoDB connection
MONGODB_URI=mongodb://user:pass@host:port/dbname \
  node scripts/migrateCustomerData.js ...
```

## Data Validation Queries

### Check Job Distribution
```javascript
// Count jobs per customer
db.jobs.aggregate([
  { $match: { markedDelete: { $ne: true } } },
  { $group: { _id: "$byPuid", count: { $sum: 1 } } },
  { $sort: { count: -1 } }
])
```

### Check Sub-Account Distribution
```javascript
// Count sub-accounts per parent
db.users.aggregate([
  { $match: { kind: { $in: ["3", "5", "9"] }, markedDelete: { $ne: true } } },
  { $group: { _id: "$parent", count: { $sum: 1 } } },
  { $sort: { count: -1 } }
])
```

### Find Orphaned Data
```javascript
// Jobs without valid customer
db.jobs.aggregate([
  { $match: { markedDelete: { $ne: true } } },
  {
    $lookup: {
      from: "users",
      localField: "byPuid",
      foreignField: "_id",
      as: "customer"
    }
  },
  { $match: { customer: { $size: 0 } } },
  { $count: "orphaned_jobs" }
])
```

## Checklist

Before Migration:
- [ ] Database backup completed
- [ ] Source and destination customer IDs confirmed
- [ ] Preview completed and reviewed
- [ ] Conflicts identified and strategy decided
- [ ] Stakeholders notified
- [ ] Maintenance window scheduled (if applicable)

After Migration:
- [ ] Migration history file reviewed
- [ ] No errors in migration log
- [ ] Job counts verified
- [ ] Sub-account counts verified
- [ ] Invoice references checked (if not skipped)
- [ ] Source customer deactivated (if not converted to admin)
- [ ] Destination customer has all expected data
- [ ] Users notified of changes

## Support

For issues or questions:
1. Check `migration_history.json` for details
2. Enable debug logging: `DEBUG=agm:*`
3. Review this guide and main README
4. Contact database administrator

## Related Commands

```bash
# View migration history
cat migration_history.json | jq '.'

# View latest migration
cat migration_history.json | jq '.[-1]'

# View all failed migrations
cat migration_history.json | jq '.[] | select(.status == "failed")'

# View all completed migrations
cat migration_history.json | jq '.[] | select(.status == "completed")'

# Count migrations by status
cat migration_history.json | jq 'group_by(.status) | map({status: .[0].status, count: length})'
```