Week 5 • Day 5Week 5

Day 5:Unsupervised Machine Learning — `K-Means` Clustering on Text Embeddings

Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz

Week 5 — Day 5: Unsupervised Machine Learning — K-Means Clustering on Text Embeddings

Mentor's Note (10+ YOE Staff AI Engineer POV): What do you do when your CTO hands you 50,000 unlabeled customer chatbot questions or server incident logs and asks: "What are the top 5 recurring problem topics our users are asking about?" You don't have labels (yy), so you can't use Supervised Learning! Instead, you embed the texts into vectors xi∈Rd\mathbf{x}_i \in \mathbb{R}^d and run Unsupervised Clustering (K-Means) to automatically discover the natural topic groups!


Part 1: How K-Means Clustering Works (Lloyd's Algorithm & Objective Function)

K-Means partitions NN vectors into KK clusters C1,C2,…,CKC_1, C_2, \dots, C_K by minimizing the Within-Cluster Sum of Squares (WCSS / Inertia): J=∑k=1K∑xi∈Ck∥xi−μk∥22\mathcal{J} = \sum_{k=1}^{K} \sum_{\mathbf{x}_i \in C_k} \|\mathbf{x}_i - \boldsymbol{\mu}_k\|_2^2 where μk\boldsymbol{\mu}_k is the Centroid (center of gravity / mean vector) of cluster kk.

The 2-Step Alternating Loop (EM-Style):

  1. Assignment Step: Assign every point xi\mathbf{x}_i to the closest centroid μk\boldsymbol{\mu}_k: c(i)=arg⁡min⁡k∈{1..K}∥xi−μk∥22c^{(i)} = \arg\min_{k \in \{1..K\}} \|\mathbf{x}_i - \boldsymbol{\mu}_k\|_2^2
  2. Update Step: Recompute each centroid μk\boldsymbol{\mu}_k as the exact mean of all points assigned to cluster kk: μk=1∣Ck∣∑xi∈Ckxi\boldsymbol{\mu}_k = \frac{1}{|C_k|} \sum_{\mathbf{x}_i \in C_k} \mathbf{x}_i Repeat Steps 1 & 2 until centroids stop moving!

Part 2: Hands-On Lab — K-Means Coded From Scratch (kmeans_topic_clusterer.py)

Check out src/kmeans_topic_clusterer.py (in Tab 2: Codes — click Run in Browser to execute) and run the script and tests:

powershell
python week05_math_data_science_and_classical_ml/day05_unsupervised_ml_kmeans_clustering_embeddings/src/kmeans_topic_clusterer.py python -m unittest discover -s week05_math_data_science_and_classical_ml/day05_unsupervised_ml_kmeans_clustering_embeddings/tests -v

In Simple English: Grouping Books on Shelves Without Labels (K-Means)

Suppose someone drops 5,000 unlabeled customer support tickets on your desk and says "Sort these into 4 piles by topic!" K-Means works in a simple 3-step dance:

  1. Drop 4 empty boxes (centroids μ1..μ4\mu_1..\mu_4) randomly onto the floor.
  2. Assign: Walk through every ticket and put it into whichever box is closest to its embedding point.
  3. Recenter: Move each box to the exact middle (average) of all tickets sitting inside it. Repeat steps 2–3 until the boxes stop moving!

AI Use Case: Before building expensive RAG taxonomies, engineers run K-Means on support queries to discover the top 10 recurring user problems automatically! Check Tab 2 (Codes) to run our K-Means clusterer right in your browser.