Month 2 Capstone Project: Automated LLM Evaluation & Regression Test Suite

Blueprint Alignment: Project 2 of the 6-Month Staff AI Engineering Curriculum (ai-engineering-6-projects-blueprint.md) Key Capabilities:

1. Stratified Golden Dataset (evals/golden_sre_dataset.json) covering happy_path, multi_hop, out_of_scope, and adversarial slices.
2. Multi-Layer Evaluation Pipeline combining Deterministic Assertions (Fact Recall, Citation Grounding, Zero-Phantom Citations, Command Safety Gate) with Atomic-Claim LLM-as-a-Judge Rubrics (Faithfulness & Completeness).
3. 95% Statistical Confidence Intervals ($\hat{p} \pm 1.96\,\text{SE}$) and Automated CI/CD Regression Gating against evals/baseline_metrics.json.
4. Automated Markdown Regression Report Generator suitable for posting directly as a GitHub Pull Request comment.