Week 15 • Day 5Week 15
Day 5:Agentic Token Budgets, Cost Caps & Oscillation Circuit Breakers
Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz
Day 5: Agentic Token Budgets, Cost Caps & Oscillation Circuit Breakers
1. First-Principles Production Risk: The $500 Runaway Agent Loop
Unlike a single-turn RAG call whose cost is bounded by 1 LLM invocation, an autonomous multi-agent loop with tool retries can get stuck oscillating between two states (Agent A calls Agent B, Agent B calls Agent A) or repeatedly calling a failing tool, burning $50–$500 on a single user request!
Three Mandatory Production Circuit Breakers
- Per-Request Dollar & Token Budget (
max_cost_usd,max_total_tokens): - Hop-Count Cap (
max_hops): Hard ceiling on total agent transitions per user request. - Action-Signature Oscillation Detector: Hashes
(agent_name, tool_name, canonical_args). If the exact same signature appears times (e.g.,2times) within a sliding window, trip the circuit breaker immediately.
In Simple English: Putting a Credit Limit on Your Agents (Cost Caps & Circuit Breakers)
Never deploy an autonomous loop without hard spending brakes! Look at :
- Every turn adds both Prompt Input Tokens (which grow larger on every step as chat history piles up!) AND Output Tokens .
- If an agent gets stuck retrying broken searches for 20 steps, cumulative cost grows quadratically!
- Production Safety Brakes:
max_stepscap (e.g., 10 turns hard stop).max_budget_usdaccumulator checked before every LLM call.- Oscillation Detector: If the agent flips
Call A -> Call B -> Call A -> Call B, halt immediately and summarize partial findings!