Week 12 • Day 1Week 12

Day 1:Week 12, Day 1: Corrective RAG (`CRAG`) — Pre-Generation Relevance Grading & Query Rewrite Loops

Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz

Week 12, Day 1: Corrective RAG (CRAG) — Pre-Generation Relevance Grading & Query Rewrite Loops


1. The Fatal Flaw of "Naive Linear RAG"

In a linear Retrieve -> Generate pipeline, the vector database always returns top_k = 5 chunks even when the user's question has zero relevant documents in the database! Feeding 5 irrelevant distractor chunks to an LLM either:

  1. Causes the LLM to hallucinate a mashup of those 5 irrelevant chunks, or
  2. Wastes 3,000 input tokens on garbage context.

2. Corrective RAG (CRAG — Yan et al., 2024)

Corrective RAG (CRAG) places a lightweight Retrieval Evaluator Gate (using the Cross-Encoder Reranker confidence score smax⁡∈[0,1]s_{\max} \in [0, 1]) between Retrieval and Generation:

code
User Query ──► Hybrid Retrieval + Reranker (s_max) │ ┌─────────────┼─────────────────┐ ▼ ▼ ▼ [s_max >= 0.65] [0.35 <= s_max < 0.65] [s_max < 0.35] CORRECT! AMBIGUOUS! INCORRECT! Filter chunks Rewrite Query & Abstain safely (or fallback with s >= 0.45 re-retrieve once! to external search tool) ──► Generate ──► Re-grade ──► "Insufficient context"

By stripping out low-confidence chunks (si<τkeeps_i < \tau_{\text{keep}}) before calling the LLM and triggering a single bounded Query Rewrite retry when confidence is ambiguous, CRAG cuts hallucination rates in half!


In Simple English: Corrective RAG (CRAG) = Double-Checking Sources Before Answering

Normal RAG blindly trusts whatever chunks the search DB returns — even if ALL 5 chunks are off-topic! Corrective RAG (CRAG) adds a smart Pre-Generation Gatekeeper:

  • Score CORRECT: At least one chunk cleanly answers the query →\rightarrow generate grounded answer!
  • Score AMBIGUOUS / INCORRECT: Zero relevant chunks found →\rightarrow rewrite the search query and retry 1 more time (or route to web search / politely say "Not found in knowledge base" instead of hallucinating!).