agmission/Development/server/docs/DEBUG_CONFIGURATION_GUIDE.md

# VS Code Debug Configuration for cleanOrphanedAppDetails.js

## Overview
This document explains how to use the VS Code debug configurations for the cleanOrphanedAppDetails.js script.

## Launch Configurations Added

### 1. **Debug Clean Orphaned AppDetails - Safe Test**
- **Purpose**: Safest option for testing and development
- **Environment**: Uses `environment.env`
- **Mode**: `--dry-run --check-only` (no actual deletions)
- **Scope**: Single year (2024) with estimation
- **Memory**: 2GB allocation
- **Best for**: Initial testing, development, debugging logic

### 2. **Debug Clean Orphaned AppDetails - Production Run**
- **Purpose**: Production-ready execution with optimizations
- **Environment**: Uses `environment_prod.env` 
- **Mode**: Real execution (will delete orphaned records)
- **Scope**: Full year (2024) with skip counting strategy
- **Memory**: 4GB allocation
- **Best for**: Production cleanup operations

### 3. **Debug Clean Orphaned AppDetails - Date Range**
- **Purpose**: Process specific date ranges
- **Environment**: Uses `environment.env`
- **Mode**: `--dry-run` with date range (Q1 2024)
- **Scope**: Custom date range with estimation
- **Best for**: Testing specific time periods, troubleshooting date ranges

### 4. **Debug Clean Orphaned AppDetails - Custom Env Vars**
- **Purpose**: Maximum flexibility using environment variables
- **Environment**: Uses `environment.env` + custom env vars in launch config
- **Mode**: Fully configurable via environment variables
- **Best for**: Custom configurations, parameter testing

## Environment Files

### Primary Files
- **`environment.env`**: Development/test database
- **`environment_prod.env`**: Production database  
- **`.env.debug`**: Custom debug file (created for you)

### Creating Your Debug Environment
1. **Copy existing environment file:**
   ```bash
   cp environment.env .env.debug
   ```

2. **Modify `.env.debug` for your needs:**
   ```bash
   # Safe defaults for debugging
   DRY_RUN=true
   CHECK_ONLY=true
   COUNTING_STRATEGY=estimate
   SPECIFIC_YEAR=2024
   DEBUG=agm:clean-orphaned-details
   ```

## How to Debug

### Step 1: Choose Configuration
1. Open VS Code
2. Go to Run and Debug (Ctrl+Shift+D)
3. Select one of the "Debug Clean Orphaned AppDetails" configurations

### Step 2: Set Breakpoints
- **Line 361**: Before counting strategy decision
- **Line 390**: In `estimateDocumentCount()` function
- **Line 450**: In batch processing loop
- **Line 520**: In `deleteBatch()` function

### Step 3: Start Debugging
1. Press F5 or click the play button
2. Monitor the integrated terminal for output
3. Use VS Code debugging features (variables, call stack, etc.)

## Configuration Details

### Command Line Arguments vs Environment Variables
```javascript
// Priority order (highest to lowest):
1. Command line arguments (--dry-run, --specific-year=2024)
2. Environment variables in launch config "env" section
3. Environment variables from envFile
4. Default values in script
```

### Memory Configuration
- **Development**: 2GB (`--max-old-space-size=2048`)
- **Production**: 4GB (`--max-old-space-size=4096`) 
- Adjust based on your dataset size

### Debug Output Filtering
```bash
# See only script output
DEBUG=agm:clean-orphaned-details

# See all AGM debug output  
DEBUG=agm:*

# See specific modules
DEBUG=agm:clean-orphaned-details,agm:db
```

## Debugging Scenarios

### Scenario 1: Test New Counting Strategy
```json
{
  "name": "Test Counting Strategy",
  "args": ["--counting-strategy=estimate", "--specific-year=2024", "--dry-run"]
}
```

### Scenario 2: Debug Memory Issues
```json
{
  "runtimeArgs": [
    "--expose-gc", 
    "--max-old-space-size=4096",
    "--trace-gc"
  ]
}
```

### Scenario 3: Debug Database Connection
```json
{
  "env": {
    "DEBUG": "agm:*",
    "DB_MAX_POOLSIZE": "4"
  }
}
```

## Troubleshooting

### Common Issues

1. **"DB_HOSTS environment variable is required"**
   - Check your envFile path
   - Ensure environment file exists and has DB_HOSTS

2. **Out of Memory Errors**
   - Increase `--max-old-space-size` 
   - Use smaller batch sizes: `--batch-size=500`

3. **Connection Timeouts**
   - Check database connectivity
   - Verify environment file has correct DB credentials

4. **No Debug Output**
   - Ensure `DEBUG=agm:clean-orphaned-details` is set
   - Check VS Code debug console vs integrated terminal

### Performance Debugging

1. **Set breakpoints at key functions:**
   - `loadAppFileIds()`: Memory loading
   - `findOrphanedAppDetailsForPeriod()`: Main processing
   - `estimateDocumentCount()`: Counting estimation

2. **Monitor variables:**
   - `existingFileIds.size`: AppFile cache size
   - `processed`: Records processed count
   - `orphanedRecords.length`: Orphaned records found

3. **Watch expressions:**
   - `process.memoryUsage()`
   - `Date.now() - periodStartTime.getTime()`
   - `Math.floor(processed / (elapsedSeconds || 1))`

## Example Debugging Session

1. **Start with Safe Test configuration**
2. **Set breakpoint at line 390** (counting strategy)
3. **Step through counting logic**
4. **Check `totalAppDetails` value**
5. **Verify progress reporting works**
6. **Test different counting strategies**

## Safety Notes

- **Always start with `--dry-run --check-only`**
- **Test on a subset first** (`--specific-year` or date range)
- **Monitor memory usage** during large operations
- **Use development database first**
- **Backup production data** before real cleanup

## Customization

You can create additional launch configurations by copying and modifying existing ones. Common modifications:

```json
{
  "name": "My Custom Debug Config",
  "args": ["--start-date=2024-06-01", "--end-date=2024-06-30"],
  "env": {
    "BATCH_SIZE": "2000",
    "COUNTING_STRATEGY": "full"
  }
}
```