DataArena
Week 5 Master Guide

Week 5: Linear Algebra, Data Science & Classical Machine Learning

Week 5 Daily Breakdown (7 Days)

Day 1

Day 1: Linear Algebra for AI Engineers — Vectors, Matrices, Norms, Attention & LoRA

1 code file(s) • 2 Q&A
Day 2

Week 5 — Day 2: Data Wrangling (`Pandas`), Stratified Splits & Preventing Data Leakage

1 code file(s) • 2 Q&A
Day 3

Week 5 — Day 3: Supervised ML — Logistic Regression, Sigmoid & Gradient Descent Coded From Scratch

1 code file(s) • 2 Q&A
Day 4

Week 5 — Day 4: Text Vectorization (`TF-IDF`) & Sub-Millisecond Classical Text Classifiers

1 code file(s) • 1 Q&A
Day 5

Week 5 — Day 5: Unsupervised Machine Learning — `K-Means` Clustering on Text Embeddings

1 code file(s) • 2 Q&A
Day 6

Week 5 — Day 6: ML Evaluation Metrics Deep Dive — Confusion Matrix, Precision, Recall, F1 & ROC-AUC

1 code file(s) • 1 Q&A
Day 7

Week 5 — Day 7: Mini-Project (Hybrid Classical ML -> LLM Confidence Cascade Router) + Week 5 Interview

1 code file(s) • 0 Q&A

Week 5: Linear Algebra, Data Science & Classical Machine Learning (week5.md)

Mentor's Note (10+ YOE Staff AI Engineer POV): Why does a Senior AI Engineer still master Linear Algebra (Vectors, Matrices, Cosine Similarity), Data Wrangling, Classical ML (TF-IDF, Logistic Regression, Decision Trees, Gradient Boosting), Clustering (K-Means), and Precision / Recall / F1 / ROC-AUC in the era of LLMs?

  1. Embeddings & Vector Databases ARE Linear Algebra (Day 1): Every OpenAI/Gemini embedding is just a 768- or 1536-dimensional vector v∈Rd\mathbf{v} \in \mathbb{R}^d. Ranking RAG chunks is literally a Matrix-Vector Dot Product (Mq\mathbf{M}\mathbf{q})!
  2. When NOT to Use an LLM (Days 3 & 4): Calling GPT-4o or Gemini takes 300ms–1,500ms and costs money on every call. If you need to route 10,000 incoming server logs per second into CRITICAL vs INFO, a TF-IDF + Logistic Regression / XGBoost classifier runs in 0.05 milliseconds (10,000x faster) at $0.00 cost!
  3. Evaluating AI Systems (Day 6): Whether you are evaluating a fraud model, a RAG reranker, or an LLM-as-a-Judge guardrail, interviewers will grill you on Precision vs Recall trade-offs, F1-Score, ROC-AUC vs PR-AUC on imbalanced datasets, and Data Leakage!

Week 5 Architecture & Daily Folder Map

text
week05_math_data_science_and_classical_ml/ ├── week5.md # Master Week 5 Syllabus & Architecture ├── day01_linear_algebra_vectors_matrices_cosine/ # Day 1: Vectors, Dot Products, L2 Norm & Cosine Math │ ├── README.md │ ├── src/linear_algebra_engine.py │ ├── tests/test_linear_algebra_engine.py │ └── interview_questions.md ├── day02_data_wrangling_cleaning_and_leakage_splits/ # Day 2: Data Cleaning, Stratified Splits & Leakage │ ├── README.md │ ├── src/dataset_pipeline.py │ ├── tests/test_dataset_pipeline.py │ └── interview_questions.md ├── day03_supervised_ml_logistic_regression_gradient_descent/# Day 3: Logistic Regression & Gradient Descent Math │ ├── README.md │ ├── src/logistic_regression_scratch.py │ ├── tests/test_logistic_regression_scratch.py │ └── interview_questions.md ├── day04_tfidf_vectorization_and_fast_text_classifiers/ # Day 4: TF-IDF Math & Sub-Millisecond Alert Classifier │ ├── README.md │ ├── src/tfidf_classifier.py │ ├── tests/test_tfidf_classifier.py │ └── interview_questions.md ├── day05_unsupervised_ml_kmeans_clustering_embeddings/ # Day 5: K-Means Clustering on Text Embeddings │ ├── README.md │ ├── src/kmeans_topic_clusterer.py │ ├── tests/test_kmeans_topic_clusterer.py │ └── interview_questions.md ├── day06_ml_evaluation_metrics_precision_recall_f1_auc/ # Day 6: Confusion Matrix, Precision, Recall, F1 & AUC │ ├── README.md │ ├── src/metrics_auditor.py │ ├── tests/test_metrics_auditor.py │ └── interview_questions.md └── day07_week5_mini_project_hybrid_ml_llm_router/ # Day 7: Hybrid ML + LLM Confidence Cascade Router ├── README.md ├── src/hybrid_ml_llm_router.py ├── tests/test_hybrid_ml_llm_router.py └── interview_questions.md

Daily Schedule & Core Mathematical Foundations

DayTopicCore Equations & TheoryProduction Deliverable
Day 1Linear Algebra: Vectors, Matrices & Cosine SimilarityDot product a⋅b=∑aibi\mathbf{a}\cdot\mathbf{b} = \sum a_i b_i; L2L_2 Norm ∥a∥2=∑ai2\|\mathbf{a}\|_2 = \sqrt{\sum a_i^2}; Cosine Similarity cos⁡(θ)=a⋅b∥a∥2∥b∥2\cos(\theta) = \frac{\mathbf{a}\cdot\mathbf{b}}{\|\mathbf{a}\|_2 \|\mathbf{b}\|_2}; Matrix multiplication Cij=∑kAikBkj\mathbf{C}_{ij} = \sum_k \mathbf{A}_{ik}\mathbf{B}_{kj}linear_algebra_engine.py — Pure-Python + NumPy compatible vector/matrix engine & top-K cosine similarity search
Day 2Data Wrangling, Deduplication & Preventing Data LeakageMissing value imputation (ZZ-score normalization z=x−μσz = \frac{x - \mu}{\sigma}), Stratified Train/Val/Test splitting, why fitting a scaler on the full dataset before splitting causes Data Leakagedataset_pipeline.py — Leak-free StandardScaler (fit on train only!) + stratified splitter
Day 3Supervised ML: Logistic Regression & Gradient DescentSigmoid σ(z)=11+e−z\sigma(z) = \frac{1}{1 + e^{-z}}; Binary Cross-Entropy (Log Loss) L=−1N∑[yln⁡(y^)+(1−y)ln⁡(1−y^)]\mathcal{L} = -\frac{1}{N}\sum [y\ln(\hat{y}) + (1-y)\ln(1-\hat{y})]; Gradient descent update w←w−η∇wL\mathbf{w} \leftarrow \mathbf{w} - \eta \nabla_{\mathbf{w}}\mathcal{L}logistic_regression_scratch.py — Trains a binary classifier via Gradient Descent from first principles
Day 4Text Classification: TF-IDF VectorizationTF(t,d)=ft,d∑t′ft′,d\text{TF}(t, d) = \frac{f_{t,d}}{\sum_{t'} f_{t',d}}; Smooth Inverse Document Frequency IDF(t)=ln⁡(1+N1+df(t))+1\text{IDF}(t) = \ln\left(\frac{1 + N}{1 + \text{df}(t)}\right) + 1tfidf_classifier.py — Sub-millisecond SRE incident classifier costing $0.00
Day 5Unsupervised ML: K-Means ClusteringLloyd's algorithm: Step 1 (Assign each point to nearest centroid arg⁡min⁡k∥xi−μk∥2\arg\min_k \|\mathbf{x}_i - \boldsymbol{\mu}_k\|^2), Step 2 (Recompute centroids $\boldsymbol{\mu}_k = \frac{1}{C_k
Day 6ML Evaluation Metrics: Precision, Recall, F1 & ROC-AUCPrecision=TPTP+FP\text{Precision} = \frac{TP}{TP+FP}, Recall=TPTP+FN\text{Recall} = \frac{TP}{TP+FN}, F1=2PRP+RF_1 = \frac{2PR}{P+R}, Accuracy Paradox on 99% imbalanced data, ROC-AUC vs PR-AUCmetrics_auditor.py — Complete classification metrics & threshold optimizer
Day 7Week 5 Mini-Project: Hybrid ML →\rightarrow LLM Cascade RouterRouting high-confidence queries (p≥0.85p \ge 0.85) through fast $0.00 Classical ML (0.1ms) and escalating only ambiguous queries (p<0.85p < 0.85) to the expensive LLM (800ms)!hybrid_ml_llm_router.py — Cuts LLM API costs by 70%+ while maintaining 99% accuracy!
Week 5: Week 5: Linear Algebra, Data Science & Classical Machine Learning | 24-Week AI Engineer Academy · DataArena