agmission/Development/server/docs/DLQ_QUICKSTART.md

DLQ Quick Start Guide

Navigation: 📖 Index | 🚀 Quick Start | 📚 API Reference | 🔧 Operations | 🏗️ System Guide

---

Get started with Dead Letter Queue (DLQ) management in minutes.

What is the DLQ System?

The DLQ system provides queue-native tools for monitoring and managing failed tasks across all queue types (partner_tasks, jobs, etc.) without MongoDB coupling, supporting multiple queue types.

---

Quick Access

1. Web Dashboard

https://localhost:4100/dlq-monitor.html

• Real-time DLQ statistics
• View failed messages
• One-click retry operations
• Queue-native operations

2. API Access

View DLQ statistics:

curl http://localhost:4100/api/dlq/partner_tasks/stats \
  -H "Authorization: Bearer $TOKEN"

Retry all messages:

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxMessages": 100}'

3. Message Operations

curl http://localhost:4100/api/dlq/partner_tasks/stats \
  -H "Authorization: Bearer $TOKEN"

---

Common Operations

View DLQ Messages

curl -X GET "http://localhost:4100/api/dlq/partner_tasks/messages?limit=20" \
  -H "Authorization: Bearer $TOKEN"

Retry All Messages

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxMessages": 50}'

Retry by Position Range

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryByPosition \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startPosition": 0, "endPosition": 10}'

Retry by Header Match

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryByHeader \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"headerKey": "x-retry-count", "headerValue": "1"}'

---

API Endpoints Summary

Endpoint	Method	Purpose
/api/dlq/:queueName/messages	GET	View DLQ messages (peek mode)
/api/dlq/:queueName/stats	GET	Get DLQ statistics
/api/dlq/:queueName/retryAll	POST	Retry all messages (queue-native)
/api/dlq/:queueName/retryByPosition	POST	Retry by position range (queue-native)
/api/dlq/:queueName/retryByHeader	POST	Retry by header match (queue-native)
/api/dlq/:queueName/purge	DELETE	Purge all DLQ messages ⚠️

---

Multiple Queue Support

Partner Queue

curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll ...

Job Queue

curl -X POST http://localhost:4100/api/dlq/dev_jobs/retryAll ...

Any Future Queue

curl -X POST http://localhost:4100/api/dlq/notifications/retryAll ...

No code changes needed - just use the queue name!

---

Monitoring Workflow

1. Check DLQ Health

curl -s http://localhost:4100/api/dlq/partner_tasks/stats \
  -H "Authorization: Bearer $TOKEN" | jq '.dlq.messageCount'

2. View Failed Messages

curl -s http://localhost:4100/api/dlq/partner_tasks/messages?limit=5 \
  -H "Authorization: Bearer $TOKEN" | jq '.messages[].errorMessage'

3. Retry Failed Messages

# Retry all (up to 100)
curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"maxMessages": 100}'

---

Troubleshooting

High DLQ Count

1. Check recent error messages:

   curl http://localhost:4100/api/dlq/partner_tasks/messages?limit=10 \
     -H "Authorization: Bearer $TOKEN"
   

2. Review error patterns in logs:

   grep "DLQ" agm_server.rlog
   

3. Retry transient errors:

   curl -X POST http://localhost:4100/api/dlq/partner_tasks/retryAll \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"maxMessages": 50}'
   

Authentication Errors

Ensure your token is valid:

export TOKEN=$(curl -X POST http://localhost:4100/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"***"}' | jq -r '.token')

Queue Not Found

Verify queue name matches environment config:

• Development: dev_partner_tasks, dev_jobs
• Production: partner_tasks, jobs

---

Best Practices

1. Monitor Regularly: Check DLQ counts daily
2. Retry Transient Errors: Network timeouts often succeed on retry
3. Investigate Patterns: Multiple failures of same type indicate systemic issues
4. Use Position Retry: For targeted retry of specific message ranges
5. Archive Non-Recoverable: Purge after investigation (with confirm=true)

---

Next Steps

📚 Complete Documentation

• 📖 DLQ Index - Documentation overview and navigation
• 📚 API Reference - Complete API with examples
• 🔧 Operations Guide - Advanced operations and troubleshooting
• 🏗️ System Guide - Architecture and internals

🔗 Related Resources

• Global DLQ Refactoring - Architecture changes
• Partner Integration Guide - Partner system overview
• Main README - Server documentation