Week 18: LLMOps — Observability (OpenTelemetry, Langfuse), Semantic Caching & Docker

1. Executive Overview

In enterprise AI deployments, an application is only as viable as its operational observability and resilience. Week 18 equips you with the complete production LLMOps (Large Language Model Operations) stack:

1. OpenTelemetry GenAI Semantic Conventions: Industry-standard hierarchical tracing tracking gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, and token latency.
2. LLM Observability (Langfuse / Arize Phoenix): Trace schemas, multi-step generation recording, user session attribution, and cost tracking.
3. Semantic Prompt Caching ($\text{Cosine Similarity} \ge 0.95$): Returning answers to paraphrased or identical incident questions in < 15ms at $0 cost, cutting API bills by 30–60%.
4. Resilient AI Gateway & Model Fallbacks (LiteLLM Pattern): Cascade routing with automatic failover (Primary $\to$ Secondary $\to$ Fallback) and provider load balancing.
5. Docker Multi-Stage Containerization: Minimalist, secure non-root containers using uv packaging for sub-second container cold starts.
6. Kubernetes Health Probes & Prometheus Metrics: Standard /healthz, /readyz, and /metrics exposition for cloud-native orchestration.

---

2. Weekly Curriculum Roadmap