Week 10 Master Guide
Week 10: Vector Databases Internals — `IVF`, `HNSW`, Vector Quantization (`SQ8`/`Binary`) & Filtered ANN
Week 10 Daily Breakdown (7 Days)
Day 1
Week 10, Day 1: Exact Flat `kNN`, RAM/Bandwidth Math & The Curse of Dimensionality
1 code file(s) • 3 Q&ADay 2
Week 10, Day 2: Partition-Based ANN — Inverted File Index (`IVF`), Voronoi Cells & `nlist`/`nprobe`
1 code file(s) • 3 Q&ADay 3
Week 10, Day 3: Graph-Based ANN — Hierarchical Navigable Small World
1 code file(s) • 3 Q&ADay 4
Week 10, Day 4: Vector Compression — Scalar (`SQ8`) & Binary (`1-Bit`) Quantization with Oversampled Rescoring
1 code file(s) • 3 Q&ADay 5
Week 10, Day 5: Metadata & RBAC Security — Why Post-Filtering Fails (`Zero-Results Cliff`) vs Pre-Filtering
1 code file(s) • 3 Q&ADay 6
Week 10, Day 6: Index Lifecycle — Tombstones, Versioned Upserts & Blue-Green Alias Swapping
1 code file(s) • 3 Q&ADay 7
Week 10, Day 7 (Mini-Project): Quantized RBAC-Filtered Vector Database Engine
1 code file(s) • 3 Q&AWeek 10: Vector Databases Internals — IVF, HNSW, Vector Quantization (SQ8/Binary) & Filtered ANN
Month 3 — Enterprise Hybrid RAG, Vector Databases & Information Retrieval Goal: Understand how production vector databases (
Qdrant,Milvus,pgvector,FAISS,Pinecone) search100,000,000+vectors in< 5 msusingIVF,HNSW,INT8/BinaryQuantization, and RBAC Pre-Filtering.
Weekly Architecture & Daily Breakdown
- Day 1: Exact Flat
kNN, RAM/Bandwidth Math & The Curse of Dimensionality (day01_exact_knn_curse_of_dimensionality_and_latency_math/) - Day 2: Partition-Based ANN — Inverted File Index (
IVF), Voronoi Cells &nlist/nprobe(day02_ivf_inverted_file_index_and_voronoi_partitions/) - Day 3: Graph-Based ANN — Hierarchical Navigable Small World (
HNSW) (M,efConstruction,efSearch) (day03_hnsw_hierarchical_navigable_small_world_graphs/) - Day 4: Vector Compression — Scalar (
SQ8) & Binary (1-Bit) Quantization with Oversampled Rescoring (day04_vector_quantization_pq_sq8_and_binary_quantization/) - Day 5: Metadata & RBAC Security — Why Post-Filtering Fails (
Zero-Results Cliff) vs Pre-Filtering (day05_metadata_filtering_pre_vs_post_filtering_and_acls/) - Day 6: Index Lifecycle — Tombstones, Document Upserts & Blue-Green Alias Swapping (
day06_index_lifecycle_upserts_tombstones_and_zero_downtime_reindexing/) - Day 7: Week 10 Mini-Project — Quantized RBAC-Filtered Vector Database Engine (
day07_week10_mini_project_filtered_quantized_vector_store/)