Week 5 — Day 2: Data Wrangling (`Pandas`), Stratified Splits & Preventing Data Leakage

Mentor's Note (10+ YOE Staff AI Engineer POV): Ask any Staff Machine Learning Engineer what the #1 silent bug is that ruins ML models and LLM Fine-Tuning evaluations: Data Leakage (Train-Test Contamination)!

Look at the two ways Junior Engineers accidentally cause Data Leakage:

Feature Scaling / Vocabulary Leakage: Calling scaler.fit_transform(X_entire_dataset) (or fitting TfidfVectorizer on the entire dataset) BEFORE splitting into X_train and X_test! That leaks the mean $\mu_{\text{test}}$ , standard deviation $\sigma_{\text{test}}$ , and vocabulary of the future test set into the training phase!

Duplicate / Near-Duplicate Text Leakage: In AI datasets, if the exact same support ticket or runbook question appears 3 times in your raw dataset and you do a random split, 2 copies land in train and 1 copy lands in test. Your model gets 99% accuracy by memorizing the duplicate, then fails miserably in production!

Part 1: The Golden Rules of Leak-Free Data Engineering

text

[Raw Noisy Dataset (CSV / JSONL / SQL)]
                 │
                 ▼  1. Deduplicate exact & normalized text FIRST!
      [Deduplicated Clean Data]
                 │
                 ▼  2. Stratified Split (Preserves exact class ratio, e.g. 10% CRITICAL in both Train & Test!)
       ┌─────────┴─────────┐
       ▼                   ▼
  [Train Set (80%)]   [Test Set (20%)] (Locked in a vault!)
       │                   │
       ▼                   │
  3. `scaler.fit(Train)`   │ (Computes mean μ_train and std σ_train ONLY from Train!)
       │                   │
       ▼                   ▼
  4. `transform(Train)`  `transform(Test)` (Uses μ_train and σ_train to scale Test!)

Z-Score Standardization Formula

$z = \frac{x - \mu_{\text{train}}}{\sigma_{\text{train}}}$

Part 2: Hands-On Lab — Leak-Free Data Pipeline & Stratified Splitter (`dataset_pipeline.py`)

Check out src/dataset_pipeline.py (in Tab 2: Codes — click Run in Browser to execute) and run the script and tests:

powershell

python week05_math_data_science_and_classical_ml/day02_data_wrangling_cleaning_and_leakage_splits/src/dataset_pipeline.py
python -m unittest discover -s week05_math_data_science_and_classical_ml/day02_data_wrangling_cleaning_and_leakage_splits/tests -v

In Simple English: What Is Z-Score Scaling & Why Does Data Leakage Ruin Models?

Look at that formula $z = \frac{x - \mu_{\text{train}}}{\sigma_{\text{train}}}$ :

Analogy: Suppose Alice scored 85 on an exam where the class average ( $\mu$ ) was 75 and the normal spread ( $\sigma$ ) was 5. Her Z-score is $\frac{85 - 75}{5} = +2.0$ ("two steps above average!"). That puts age (20–70) and salary ($30k–$200k) on the exact same fair scale!
Why Data Leakage Is Like Cheating on a Practice Test: If you compute the class average using BOTH your training set AND final test set before splitting, your model peeks at answers from the test set. In production on brand-new users, accuracy suddenly collapses! Always fit scalers strictly on training rows.

Day 2:Data Wrangling (`Pandas`), Stratified Splits & Preventing Data Leakage

Week 5 — Day 2: Data Wrangling (Pandas), Stratified Splits & Preventing Data Leakage

Part 1: The Golden Rules of Leak-Free Data Engineering

Z-Score Standardization Formula

Part 2: Hands-On Lab — Leak-Free Data Pipeline & Stratified Splitter (dataset_pipeline.py)

In Simple English: What Is Z-Score Scaling & Why Does Data Leakage Ruin Models?

Week 5 — Day 2: Data Wrangling (`Pandas`), Stratified Splits & Preventing Data Leakage

Part 2: Hands-On Lab — Leak-Free Data Pipeline & Stratified Splitter (`dataset_pipeline.py`)