Week 17 • Day 5Week 17
Day 5:Asynchronous Background Tasks & Webhook Event Queues
Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz
Day 5: Asynchronous Background Tasks & Webhook Event Queues
1. First-Principles Problem: Protecting the HTTP Critical Response Path
In high-performance AI services, only operations required to generate the user's tokens belong in the synchronous HTTP request path.
Consider an incident copilot endpoint that executes:
- Retrieval + LLM Generation: $1,200\text{ms}$
- Audit DB Logging: $45\text{ms}$
- Slack Webhook Notification: $350\text{ms}$
- Asynchronous LLM-as-a-Judge Eval: $2,100\text{ms}$
- Vector Cache Refresh: $400\text{ms}$
Latency Comparison:
Furthermore, if the external Slack API experiences a network timeout, a synchronous endpoint crashes with HTTP 500, failing the entire user query even though the AI response was computed successfully!
2. Decoupled Event Architecture & Dead Letter Queues (DLQ)
Architecture Flow Diagram
Client HTTP Request->API[FastAPI Endpoint
|1. Immediate Stream / Return (1200ms)| Client
|2. Enqueue Background Task| Q[In-Memory / Redis Queue
W[Asynchronous Worker
DB[Audit DB Write
WH[Webhook Dispatcher (HMAC Signed)
|Permanent Failure| DLQ[Dead Letter Queue (DLQ)
- Fire-and-Forget / Non-Blocking Queue: The main event loop delegates post-processing tasks immediately after response transmission.
- Cryptographic Webhook Signatures: Outgoing webhook payloads are signed via
HMAC-SHA256in theX-Copilot-Signatureheader, allowing subscribers to verify payload integrity and authenticity. - Dead Letter Queue (DLQ): Failed webhook deliveries are retried up to times with exponential backoff. If all retries fail, payloads are archived in a DLQ for manual inspection without crashing the application.
In Simple English: Don't Make the User Wait for Slow Email / Analytics Work
Suppose after answering a chat question (1,200 ms), your server also writes analytics to Snowflake (350 ms), sends a Slack webhook (2,100 ms), and emails an audit receipt (400 ms):
- Doing Everything Inline: User waits
4,095 ms(over 4 seconds!) just so Slack can finish! - Background Queue (
Decoupled): Return the AI answer immediately in1,200 msand drop the Slack/email jobs onto a fast background queue (with a Dead Letter QueueDLQcatching any failed webhooks for safe retry later)!