Week 9 • Day 2Week 9
Day 2:Week 9, Day 2: Chunking Strategies — Fixed Overlap, Recursive & Parent-Child
Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz
Week 9, Day 2: Chunking Strategies — Fixed Overlap, Recursive & Parent-Child (Small-to-Big)
1. The Fundamental Tension in RAG Chunk Sizing
Every RAG engineer faces a fundamental tradeoff when choosing chunk_size:
- Small Chunks (
64–128tokens / single sentences):- Pros: Sharp, focused embedding vector! The vector isn't diluted by 5 other topics in the same paragraph, so Cosine Similarity with a user's question is very high.
- Cons: Missing surrounding context! The LLM receives a single sentence without the setup, caveats, or full CLI command block around it.
- Large Chunks (
512–1,024tokens / full sections):- Pros: Rich LLM context! Contains the complete procedure, prerequisites, and rollback steps.
- Cons: Blurry mean-pooled embedding vector! Averaging
800tokens across multiple sub-steps dilutes the vector, lowering retrieval recall on specific questions.
2. The Production Solution: Parent-Child (Small-to-Big) Chunking
Why choose between small and large when you can decouple what you embed from what you feed the LLM?
code
┌─────────────────────────────────────────────────────────────────────────┐
│ PARENT CHUNK (parent_id = "P-01", ~500 tokens — Full Runbook Section) │
│ "Payments API 502 Triage: First check Stripe status. Next inspect pods │
│ via kubectl get pods -n payments. If OOMKilled, increase memory limit │
│ to 4Gi and run kubectl rollout undo deployment/payments-api." │
└───────────┬─────────────────────────────┬───────────────────────────────┘
│ │
▼ ▼
┌───────────────────────────────┐ ┌───────────────────────────────────────┐
│ CHILD CHUNK C-01 (~25 tokens) │ │ CHILD CHUNK C-02 (~30 tokens) │
│ "Inspect pods via kubectl get │ │ "If OOMKilled, increase memory limit │
│ pods -n payments." │ │ to 4Gi and run kubectl rollout undo" │
│ [Embedded in Vector DB!] │ │ [Embedded in Vector DB!] │
└───────────────────────────────┘ └───────────────────────────────────────┘
- Indexing Time: Split documents into large Parent Chunks (
parent_id), and split each Parent into fine-grained Child Chunks (child_id, storingparent_idas foreign key). Embed the Child Chunks into the Vector Index. - Query Time: Search the Vector Index against the Child Chunks (high retrieval precision!), then deduplicate and fetch their Parent Chunks to pass full, coherent sections to the LLM!
In Simple English: Fixed Overlap vs Parent-Child Chunking
- Fixed Overlap (Roof Shingles): Cut every 300 words with 50 words repeating at each seam — so a sentence sitting on the boundary is never sliced in half.
- Parent-Child Chunking (Small Key, Big Folder): Store tiny, razor-sharp 150-word Child chunks in the Vector DB so vector matching finds the exact paragraph — then fetch its large 800-word Parent section to hand to the LLM so the AI reads full surrounding context!