DataArena
Vector Database Architecture: HNSW vs IVF & pgvector
RAG, Vector Search & Embeddings • Data Infrastructure10 min read

Vector Database Architecture: HNSW vs IVF & pgvector

Hierarchical Navigable Small World graphs, Inverted File indexing, pgvector in PostgreSQL, and memory footprints.

Practice in Prompt LabTest in AI Interview

Core Conceptual Brief

Exact vector search (k-NN) has O(N x D) complexity, making it too slow at scale. Approximate Nearest Neighbor (ANN) indexing using HNSW graphs or IVF solves this in milliseconds.

Theoretical & Empirical Architecture

Searching through 10 million 1536-dimensional vectors using exhaustive flat search requires billions of floating-point operations per query, resulting in multi-second latencies.

Approximate Nearest Neighbor (ANN) algorithms trade a minuscule amount of recall (e.g. 98% recall instead of 100%) for 1000x faster search latencies (sub-5ms). The dominant production algorithm today is HNSW (Hierarchical Navigable Small World).

Under the Hood: Key Engineering Mechanics

1HNSW Graph Mechanics (Skip-List for Vectors)

HNSW constructs a multi-layered geometric graph. Upper layers have sparse long-range connections between distant vectors (allowing rapid geometric jumping across the high-dimensional space). Lower layers have dense short-range connections for fine-grained local neighborhood search. Search time scales logarithmically (O(log⁡N)O(\log N)).

2IVF (Inverted File Index)

IVF clusters vectors into KK centroids using k-means. At query time, only vectors belonging to the nearest NprobeN_{probe} centroids are inspected. IVF uses less RAM than HNSW but typically achieves lower recall on high-dimensional vectors.

3pgvector vs Dedicated Vector DBs (Pinecone, Qdrant)

For datasets under 1-5 million vectors, adding pgvector to PostgreSQL is almost always superior to running a standalone vector database. It enables ACID transactions, relational joins, and strict tenant row-level security (RLS) without cross-system sync pipelines.

Production pgvector Schema with HNSW Index in PostgreSQL

-- Enable the pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Create table with 1536-dimensional embeddings (e.g. text-embedding-3-small)
CREATE TABLE enterprise_documents (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id VARCHAR(64) NOT NULL,
    document_title TEXT NOT NULL,
    chunk_content TEXT NOT NULL,
    embedding vector(1536) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Build HNSW index with cosine distance operator (<=>)
-- m = 16 (bidirectional links per node), ef_construction = 64 (build exploration budget)
CREATE INDEX idx_docs_hnsw_embedding 
ON enterprise_documents 
USING hnsw (embedding vector_cosine_ops) 
WITH (m = 16, ef_construction = 64);

-- Sub-millisecond ANN search with tenant isolation:
SELECT id, document_title, chunk_content, 
       1 - (embedding <=> $query_vector) AS cosine_similarity
FROM enterprise_documents
WHERE tenant_id = 'acme_corp'
ORDER BY embedding <=> $query_vector
LIMIT 5;

Production Guardrails & Takeaways

  • HNSW is the gold standard for high-accuracy, low-latency ANN search in memory.
  • Use pgvector if you already use PostgreSQL and have fewer than a few million vectors.
  • Always combine vector similarity filtering with relational metadata (tenant_id, user permissions).

Real Staff AI Systems Interview Question

Q: What is the trade-off between `m` and `ef_construction` parameters when tuning an HNSW index?

Staff Answer:

m governs the maximum number of bidirectional connections per node. Higher m increases recall and improves search in clustered spaces, but drastically increases RAM usage. ef_construction controls the exploration depth during index creation: higher values produce higher quality graphs (better recall at search time) at the expense of slower indexing build times.

Vector Database Architecture: HNSW vs IVF & pgvector | AI Systems Engineering Guide · DataArena