Week 18 • Day 1Week 18

Day 1:OpenTelemetry GenAI Semantic Conventions

Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz

Day 1: OpenTelemetry GenAI Semantic Conventions

1. First-Principles Problem: The Proprietary Vendor Lock-in Trap

When engineering teams begin adding observability to AI apps, they often hardcode proprietary SDKs (Datadog, New Relic, LangSmith). When the enterprise switches logging vendors, every file in the codebase must be rewritten.

The Cloud Native Computing Foundation (CNCF) created the OpenTelemetry (OTel) GenAI Semantic Conventions to standardize trace attributes across the entire industry:

Attribute NameStandard TypeExample ValueDescription
gen_ai.systemstring"openai", "gemini", "vllm"The model inference provider
gen_ai.request.modelstring"gpt-4o-mini"The requested model name
gen_ai.usage.input_tokensint480Number of tokens in prompt
gen_ai.usage.output_tokensint125Number of tokens in response
gen_ai.response.finish_reasonsarray[string]["stop"]Reason generation finished
gen_ai.latency.ttft_msfloat240.5Time-to-First-Token in ms

2. W3C Trace Context Standard (traceparent)

To link distributed microservices across network hops:

text
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01 │ └────────────────┬────────────────┘ └───────┬──────┘ └─┬─┘ │ │ │ └── Flags (01 = Sampled) │ │ └───────────── Parent Span ID (16 hex) │ └──────────────────────────────────────── Trace ID (32 hex) └──────────────────────────────────────────────────────────── Version (00)

When an API Gateway receives an HTTP request with traceparent, the downstream GenAI spans attach to that exact trace_id and parent_span_id, forming a unified distributed trace graph across all microservices.


In Simple English: Universal Telemetry Names (OpenTelemetry GenAI)

If Company A logs token counts as prompt_tks and Company B logs them as input_token_count, switching observability vendors forces rewriting 500 dashboards! OpenTelemetry (OTel) GenAI Conventions give every AI engineer one universal dictionary:

  • gen_ai.system (openai, anthropic, gemini)
  • gen_ai.usage.input_tokens & gen_ai.usage.output_tokens
  • traceparent header passed across microservices so front-end -> gateway -> RAG DB share 1 unified timeline!