Codebase 2: Classic ML Pipeline & Train-Serving Skew Audit

Audit a real payment fraud ML training pipeline and live serving API. Offline metrics claim 0.99+ accuracy, yet live production fails silently because of target leakage, time-series shuffle leakage, preprocessing fit over train+test combined, stale serving constants, and unguarded NaN crashes.

System Design Perspective

Feature Parity, Leakage & Lakehouse ML

Before You Start (Databricks Mosaic)

Audit `train_pipeline.py` and `serving_api.py` in Codebase 2 to practice eliminating training-serving skew and temporal data leakage.

Download & Run LocallyDownload .zip

Runtime: Python 3.9+ (Pure stdlib — zero pip install required)

unzip classic-ml-pipeline.zip
cd classic-ml-pipeline
python test_ml_pipeline.py
python serving_api.py
# Open http://localhost:8150 to compare offline training vs live checkout scoring!

Meta ML / Databricks / Uber / Quant ML Debugging Style

# BUG 1 (Target Leakage): post-event column unknown at checkout!
LEAKY_FEATURES = ["amount", "velocity_1h", "chargeback_settled_30d"]

# BUG 2 (Temporal Split Leakage): shuffles time-series fraud events!
random.Random(42).shuffle(shuffled)

# BUG 3 (Preprocessing Leakage): fits mean/std on train + test TOGETHER!
all_amounts = [r["amount"] for r in shuffled]
mean_amt = sum(all_amounts) / len(all_amounts)