docs/architecture/async-queues.md

Asynchronous Queue Topology

This slice shows the queue-backed Lambda handoffs. Use this when debugging "what happens after the API call returns" or when tracing work that moves Lambda -> SQS -> Lambda, including the chained backfill path.

flowchart LR
    subgraph Upload["ICP Upload Evaluation"]
        S3["env-reef-a-matic-upload S3 event"] --> CONVERT["Lambda: upload-convert"]
        CONVERT -->|"one message per non-RO element"| UPLOADQ["SQS FIFO: env-upload-results.fifo"]
        UPLOADQ -->|"event source mapping batch_size=1"| EVAL["Lambda: corrections-eval-element-test-result"]
        EVAL --> TESTH["DynamoDB: env-test-history"]
        EVAL --> PROGH1["DynamoDB: env-program-history draft"]
        EVAL --> APPCFG["DynamoDB: env-app-config prompts"]
        EVAL --> BEDROCK["Amazon Bedrock Reef Review severity review + interpretation"]
        EVAL --> REEFREV["DynamoDB: env-reef-review"]
    end

    subgraph Correction["ICP Correction Application"]
        APICORR["POST /corrections"] --> APPLY["Lambda: corrections-apply-correction"]
        APPLY -->|"checked correction doses"| CORRQ1["SQS FIFO: env-icp-correction.fifo"]
        CORRQ1 -->|"event source mapping batch_size=1"| SAVESQS1["Lambda: save-dose-history-sqs"]
        SAVESQS1 --> DOSEH1["DynamoDB: env-dose-history"]
    end

    subgraph Backfill["Program Backfill Chain"]
        APIPROG["POST /program/history/save"] --> SAVEPROG["Lambda: dose-save-program-history"]
        SAVEPROG -->|"backdated active or inactive program"| BACKFILLQ["SQS: env-dose-generate-past-dose"]
        BACKFILLQ -->|"event source mapping batch_size=1"| GENPAST["Lambda: generate-past-program-history"]
        GENPAST -->|"generated dose records"| CORRQ2["SQS FIFO: env-icp-correction.fifo"]
        CORRQ2 -->|"event source mapping batch_size=1"| SAVESQS2["Lambda: save-dose-history-sqs"]
        SAVESQS2 --> DOSEH2["DynamoDB: env-dose-history"]
    end

    subgraph Daily["Scheduled Daily Dosing"]
        EB["EventBridge: apply-daily-dose at 01:00 UTC"] --> APPLYDAILY["Lambda: dose-apply-daily-dose"]
        APPLYDAILY -->|"due active-program elements"| DAILYQ["SQS: env-dose-save-daily-dose"]
        DAILYQ -->|"event source mapping batch_size=1"| SAVEDAILY["Lambda: dose-save-daily-dose"]
        SAVEDAILY --> DOSEH3["DynamoDB: env-dose-history"]
    end

    TESTH --> WST["WebSocket: ws-test-history-notify"]
    PROGH1 --> WSP["WebSocket: ws-program-history-notify"]
    DOSEH1 --> WSD1["WebSocket: ws-dose-history-notify"]
    DOSEH2 --> WSD2["WebSocket: ws-dose-history-notify"]
    DOSEH3 --> WSD3["WebSocket: ws-dose-history-notify"]

Queue Inventory

Queue	Producer	Consumer	Why it exists
`env-upload-results.fifo`	`upload-convert`	`corrections-eval-element-test-result`	Fans one extracted ICP PDF into deterministic per-element evaluation work while preserving FIFO/dedupe behavior.
`env-icp-correction.fifo`	`corrections-apply-correction`, `generate-past-program-history`	`save-dose-history-sqs`	Serializes dose-history writes generated from checked corrections and from program backfill.
`env-dose-generate-past-dose`	`dose-save-program-history`	`generate-past-program-history`	Moves potentially large retrospective program backfill outside the API request path.
`env-dose-save-daily-dose`	`dose-apply-daily-dose`	`dose-save-daily-dose`	Moves scheduled daily dose creation outside the EventBridge target Lambda.

Important Chains

ICP upload: upload-convert -> env-upload-results.fifo -> corrections-eval-element-test-result -> env-test-history / env-program-history / env-app-config prompts / Bedrock severity review and interpretation / env-reef-review.
Checked correction: corrections-apply-correction -> env-icp-correction.fifo -> save-dose-history-sqs -> env-dose-history.
Program backfill: dose-save-program-history -> env-dose-generate-past-dose -> generate-past-program-history -> env-icp-correction.fifo -> save-dose-history-sqs -> env-dose-history.
Scheduled daily dosing: EventBridge -> dose-apply-daily-dose -> env-dose-save-daily-dose -> dose-save-daily-dose -> env-dose-history.