Meta AI (FAIR)Classic ML Leakage + AI-Assisted Review

E6 Staff Open-Weights Serving & Post-Training Engineer • $460K – $920K+ TC (E6–E7)

Classic ML & Open-Weights ServingWith-AI Coding Allowed

Codebase 2: Classic ML Pipeline & Train-Serving Skew Audit

Audit a real payment fraud ML training pipeline and live serving API. Offline metrics claim 0.99+ accuracy, yet live production fails silently because of target leakage, time-series shuffle leakage, preprocessing fit over train+test combined, stale serving constants, and unguarded NaN crashes.

System Design Perspective

Classic ML Leakage + AI-Assisted Review

Before You Start (Meta AI (FAIR))

Run `python test_ml_pipeline.py` and `python serving_api.py` in Codebase 2 to catch target leakage, time-series split leakage, train+test scaling, and serving NaN crashes.

Download & Run LocallyDownload .zip

Runtime: Python 3.9+ (Pure stdlib — zero pip install required)

unzip classic-ml-pipeline.zip
cd classic-ml-pipeline
python test_ml_pipeline.py
python serving_api.py
# Open http://localhost:8150 to compare offline training vs live checkout scoring!

Meta ML / Databricks / Uber / Quant ML Debugging Style

# BUG 1 (Target Leakage): post-event column unknown at checkout!
LEAKY_FEATURES = ["amount", "velocity_1h", "chargeback_settled_30d"]

# BUG 2 (Temporal Split Leakage): shuffles time-series fraud events!
random.Random(42).shuffle(shuffled)

# BUG 3 (Preprocessing Leakage): fits mean/std on train + test TOGETHER!
all_amounts = [r["amount"] for r in shuffled]
mean_amt = sum(all_amounts) / len(all_amounts)