Week 17 • Day 1Week 17

Day 1:FastAPI Architecture — ASGI vs. WSGI, Lifespan State & Dependency Injection

Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz

Day 1: FastAPI Architecture — ASGI vs. WSGI, Lifespan State & Dependency Injection

1. First-Principles Problem: Why WSGI Collapses Under AI Workloads

In traditional Web development (e.g., Flask, Django under Gunicorn with sync workers), servers implement the WSGI (Web Server Gateway Interface) specification:

  • Each HTTP request occupies an operating system thread for its entire lifecycle: Trequest=Tnetwork_in+Tdb+Tllm+Tnetwork_outT_{\text{request}} = T_{\text{network\_in}} + T_{\text{db}} + T_{\text{llm}} + T_{\text{network\_out}}
  • In classic CRUD apps, Trequest≈15msT_{\text{request}} \approx 15\text{ms}.
  • In AI applications, an LLM call or multi-step agent reasoning loop takes $1,500\text{ms to } 5,000\text{ms}$!

The Thread Exhaustion Math

If a server has W=16W = 16 worker threads and average LLM turnaround time is Wtime=2.0sW_{\text{time}} = 2.0\text{s}: Max ThroughputWSGI=WWtime=162.0=8 requests / second\text{Max Throughput}_{\text{WSGI}} = \frac{W}{W_{\text{time}}} = \frac{16}{2.0} = 8 \text{ requests / second} When the 9th concurrent user arrives, their request queues up. Under 100 concurrent users, latency skyrockets and connections timeout.


2. ASGI (Asynchronous Server Gateway Interface) & The Event Loop

In ASGI (FastAPI + Uvicorn), network connections are managed by a single-threaded asynchronous event loop (asyncio / uvloop):

  • When an endpoint calls await client.post(".../v1/chat/completions"), the event loop registers the OS socket wait with epoll / kqueue / IOCP and immediately yields execution.
  • While waiting for the LLM token response, the same thread processes hundreds of other incoming HTTP requests, headers, and validations. Max ConcurrencyASGI≈Available File Descriptors≈10,000+\text{Max Concurrency}_{\text{ASGI}} \approx \text{Available File Descriptors} \approx 10,000+

3. Production Lifespan Protocol & Dependency Injection

  1. Lifespan Context (@asynccontextmanager): Instead of deprecated @app.on_event("startup"), modern FastAPI uses async context managers to initialize shared resources (database connection pools, semantic cache clients, vector search engines) once at application boot, and safely close them on SIGTERM.
  2. Dependency Injection (Depends): Provides per-request state (e.g., authenticated tenant identity, session logger) cleanly to route handlers without global variable coupling.

In Simple English: Why Async ASGI Handles 1,000 AI Users Where Sync WSGI Freezes at 10

Imagine a coffee shop where brewing one espresso takes 2 seconds:

  • Sync WSGI (1 barista freezes at the espresso machine): While waiting 2 seconds for User #1's espresso to drip (waiting on OpenAI/Gemini network I/O), the barista stands idle staring at the machine and makes Users #2–#50 wait in line!
  • Async ASGI (Smart barista starts steam, takes User #2's order, pours User #3's cup): Because 99% of an LLM API call is waiting for tokens over the internet, a single async/await event loop handles 1,000+ simultaneous chats on 1 CPU thread cleanly!