Master Troubleshooting, Bug & Mentor-Mentee Q&A Log (troubleshooting_log.md)

Why This File Exists: Most courses only show you polished code that works on the first try. In the real world, 80% of Senior AI Engineering is debugging, architectural trade-offs, and fixing weird environment/data issues.

Every time we hit a terminal error, a subtle Python/SQL bug, an LLM hallucination, or an architectural design question during our 24-week journey, we log it here (and in the respective weekXX/troubleshooting/ folder). When this curriculum is published on your website, this becomes a goldmine for students around the world!

---

Index of Logged Issues & Architecture Discussions

---

Detailed Case Studies

Case #001: Docker CLI Config Permission Warning on Windows

• Location: setup.md (in Troubleshooting Guide)
• Command Run: docker --version
• Error / Warning Output:
  text

  Copy

  WARNING: Error loading config file: open C:\Users\SUDHANSHU\.docker\config.json: Access is denied. Docker version 28.4.0, build d8eb465
• Root Cause: Either the terminal runs inside a restricted sandbox that blocks access to $HOME\.docker, or Docker Desktop previously created config.json with elevated Administrator permissions.
• Fix / Takeaway:
  1. For version checks (docker --version), the CLI still succeeds (exit code 0).
  2. In a standard Windows PowerShell session, if .docker/config.json has locked permissions, reset ACL ownership using: icacls "$env:USERPROFILE\.docker" /grant "${env:USERNAME}:(OI)(CI)F" /T

---

Case #002: Designing a Curriculum for Both Experienced Builders & Non-Tech Website Users

• Question Asked by Builder: "Before we start, do we not need to learn Python, SQL, and other basics? I know these things, but I want to make .md files for basics and theories too with code, steps, commands, structure, and all the things that matter so a non-tech person visiting my website around the world can do the same."
• Mentor's Architectural Decision:
  • Never assume prior knowledge when building an educational platform!
  • Even for an experienced developer, reviewing Python data structures (dict hash tables O(1) vs list O(n)), generators (yield), and SQL internals (B-Tree indexes, Window Functions, ACID) from an AI Engineering & Senior Interview perspective turns shallow knowledge into deep expertise.
  • Action Taken: Upgraded Month 1 (Weeks 1–4) in plan.md (in Troubleshooting Guide) so:
    • Week 1: Covers How Computers/OS/Terminal Work, Git, and Core Python from variables to OOP.
    • Week 2: Covers Modern Pro Python (uv, Type Hints, Pydantic v2, asyncio, pytest).
    • Week 3: Covers Relational Databases & SQL from SELECT * to JOINs, CTEs, Window Functions, B-Tree Indexes, and Python Async DB drivers.
    • Week 4: Covers HTTP/REST APIs, LLM Tokenization/Cost Math, Prompt Engineering, and Month 1 Capstone (SRE Runbook Copilot CLI).

---

Case #003: Do We Need Multiple LLM API Keys or Just OpenAI?

• Question Asked by Builder: "In .env you mentioned many APIs—do we need all or one API only? Can OpenAI alone be okay?"
• Mentor's Answer:
  • One single API key (OPENAI_API_KEY) is 100% sufficient for the entire 6-month curriculum!
  • Why? OpenAI provides both chat/reasoning models (gpt-4o-mini, gpt-4o) and state-of-the-art embedding models (text-embedding-3-small) under one single API key. Using gpt-4o-mini ($0.15 per 1M input tokens), a developer can complete all 6 months of exercises and capstones for less than $1–$2 total.
  • Why mention Ollama / Gemini / Groq at all? Because the official openai Python SDK (AsyncOpenAI(api_key=..., base_url=...)) has become the universal industry standard protocol. In Week 3/4, we show a 5-line trick where changing base_url lets a student without a credit card run the exact same OpenAI code against local Ollama—Zero code rewrite required!

---

Case #004: Identifying API Key Prefixes (sk-proj-... vs AIzaSy... vs AQ.Ab8...)

• Question Asked by Builder: "AQ.Ab8RN6K-YEbPuJAdQnCzSl......... something like this I copied from AI Studio—is this the one or another key?"
• Mentor's Diagnosis:
  1. Google AI Studio (aistudio.google.com) vs OpenAI (platform.openai.com):
     • OpenAI keys come from platform.openai.com/api-keys and always start with sk-proj-... (or sk-...). They require a paid credit balance ($5 minimum).
     • Google AI Studio keys come from aistudio.google.com/app/apikey and power Google Gemini models (which have a generous 100% FREE tier!).
  2. AIzaSy... vs AQ.Ab8... inside Google:
     • A standard Google AI Studio Gemini API key starts with AIzaSy... (39 characters).
     • A token starting with AQ.Ab8... is a Google Cloud / Vertex AI Express / OAuth token.
  3. Can we use Google AI Studio (Free) with the OpenAI Python SDK?
     • YES! Google provides an official OpenAI-compatible endpoint (https://generativelanguage.googleapis.com/v1beta/openai/). That means we can use AsyncOpenAI in Python with a free Google AI Studio key just by setting base_url—or use an OpenAI sk-proj-... key directly!

---

Case #005: Loading the Master Root AI_Engineer/.env Across Nested Subfolders

• Location: issue_01_root_dotenv_loading.md (in Troubleshooting Guide)
• Symptom: sys_inspector.py initially printed LLM_API_KEY (Safe Mask) : <NOT_SET> because Python's os.environ only reads OS shell variables unless a .env loader explicitly reads D:\Spark\AI_Engineer\.env.
• Fix: Added find_root_dotenv() and load_root_dotenv() using Path(__file__).resolve().parents to automatically walk up from any nested weekXX/dayYY/src/ script to the master D:\Spark\AI_Engineer\.env file. Verified output now reads AQ.A***986g (gemini-2.5-flash) from D:\Spark\AI_Engineer\.env!

---

Case #006: Windows asyncio.run() Hanging in Restricted Socket Sandboxes (socketpair() / IOCP)

• Location: issue_01_windows_asyncio_sandbox_loop.md (in Troubleshooting Guide)
• Symptom: Running asyncio.run() inside the Windows restricted terminal sandbox hung indefinitely while initializing the event loop's internal self-pipe (socket.socketpair() on loopback 127.0.0.1).
• Root Cause & Fix: On Windows, Python's BaseEventLoop creates a local TCP loopback socketpair to wake up the event loop across threads. When a sandbox blocks local socket creation, asyncio.run() blocks at loop startup. We documented this in Week 2's troubleshooting folder and added a zero-socket coroutine stepper fallback so tests run instantaneously in any sandbox!

---

Case #007: PyTorch RuntimeError: view size is not compatible with input tensor's size and stride on Sliced Targets (Week 7, Day 7)

• Symptom: When computing F.cross_entropy(logits.view(-1, vocab_size), targets.view(-1)) where targets = data[:, 1:], PyTorch raised RuntimeError: view size is not compatible with input tensor's size and stride... Use .reshape(...) instead.
• Root Cause: Slicing columns (data[:, 1:]) on a 2D (batch, seq_len) tensor creates a non-contiguous view whose row stride is still seq_len while its row width is seq_len - 1. Calling .view(-1) requires contiguous memory.
• Resolution: Use .reshape(-1) (or .contiguous().view(-1)) when flattening sliced tensors in causal language modeling loss functions.

---

Case #008: Pydantic v2 @field_validator Field Order Pitfall in Cross-Field Validation (Week 8, Day 1)

• Symptom: In GoldenEvalCase, @field_validator("expected_facts") checked info.data.get("must_abstain", False), which always evaluated to False even when must_abstain=True was passed!
• Root Cause: In Pydantic v2, @field_validator runs in the order fields are declared on the class. Because expected_facts was declared above must_abstain, must_abstain was not yet in info.data when expected_facts was validated.
• Resolution: Always use @model_validator(mode="after") for cross-field invariants that depend on multiple fields on a Pydantic v2 model.

---

Case #013: LoRA Gradient Initialization Math ($B=\mathbf{0} \implies \nabla_A \mathcal{L} = \mathbf{0}$) at Step 0 (Week 21, Day 3)

• Symptom: In test_custom_lora_layer.py, assert lora_A.grad.abs().sum().item() > 0.0 failed with AssertionError: 0.0 not greater than 0.0.
• Root Cause: In standard LoRA initialization, $A \sim \mathcal{N}(0, \sigma^2)$ and $B = \mathbf{0}$ so that $\Delta W = \frac{\alpha}{r} B A = \mathbf{0}$ at step 0. Under the chain rule, $\nabla_A \mathcal{L} = \frac{\alpha}{r} B^T (\nabla_{\Delta W} \mathcal{L})$. Because $B = \mathbf{0}$, $\nabla_A \mathcal{L}$ is mathematically zero until $B$ receives a non-zero gradient update.
• Resolution: Tested gradient flow after performing a gradient step or initializing $B \ne \mathbf{0}$ to simulate active training, and added a dedicated test confirming $A$'s gradient is mathematically zero at step 0.

---

Case #014: Loss Diagnostic Engine Early Exit Before Threshold Evaluation (Week 21, Day 5)

• Symptom: In test_gradient_explosion_diagnostics, loss explosion with NaN returned HEALTHY instead of GRADIENT_EXPLOSION.
• Root Cause: The method checked if len(loss_history) < 3: return DiagnosticReport(status=HEALTHY) before checking for math.isnan(loss_history[-1]).
• Resolution: Re-ordered checks to evaluate NaN, Inf, and large single-step spikes before checking the history length threshold.

---

Case #015: LoRA Adapter Classifier Convergence on Small SRE Dataset (Week 21, Day 7)

• Symptom: Incident classifier predicted SEV-2 instead of SEV-1 for database connection exhaustion in test assertion.
• Root Cause: 35 epochs was insufficient for the AdamW optimizer to overcome the random projection barrier for nuanced severity signals.
• Resolution: Increased training epochs to 60 with learning rate 0.03, ensuring full convergence on all 9 canonical incident patterns.

---

Case #016: Continuous Batching Step Timing in PagedAttention Simulator (Week 22, Day 2)

• Symptom: test_paged_attention_simulator.py expected 1 request admitted in step 2, but 0 were admitted until step 3.
• Root Cause: Sequence completion and block release happened after the admission phase in step 2, so KV cache slots were not yet freed until step 3.
• Resolution: Updated test assertions to match the continuous batching step lifecycle: request 1 completes at step 2, freeing blocks for request 3 admission at step 3.

---

Case #017: BM25 Single-Token False Positive Retrieval Pruning (Week 23, Day 3)

• Symptom: test_query_out_of_domain with query "chocolate cake recipe" returned 1 chunk because the common word "recipe" appeared incidentally in Week 1 Day 1.
• Root Cause: Without a minimum score threshold and stop-word filtering, rare word overlap in non-technical queries produced spurious matches.
• Resolution: Added token length filtering ($>2$ chars), English stop-words pruning, and enforced min_score = 0.45 to guarantee zero false positives on out-of-domain queries.

---

Case #018: Markdown Heading Regex Flexibility in Troubleshooting Scraper (Week 23, Day 5)

• Symptom: test_troubleshooting_showcase_renderer.py parsed 0 cases from troubleshooting_log.md.
• Root Cause: The parser regex looked for ## Case #... (h2), but troubleshooting_log.md formatted cases as ### Case #... (h3).
• Resolution: Updated regex to #{2,3}\s+Case\s+#\d+ and made key matching flexible to handle both Symptom and Problem.

---

Case #019: FastAPI Query(None) Parameter Handling in Direct Unit Test Calls (Week 23, Day 7)

• Symptom: Direct Python function calls to FastAPI route handlers raised AttributeError: 'Query' object has no attribute 'lower'.
• Root Cause: FastAPI injects Query(None) as the default argument value when an endpoint function is called directly as a regular Python function rather than through TestClient.
• Resolution: Checked isinstance(param, str) before string operations, falling back to default values when passed a Query object.

---

Case #020: Python hash() Non-Determinism Across Multi-Process Test Suites (Capstone 6)

• Symptom: Running pytest capstones failed on test_serving_chat_completion_contract with NETWORK predicted instead of DATABASE, while running the test file standalone passed.
• Root Cause: Python's built-in hash() uses randomized seeds per process (PYTHONHASHSEED), causing feature hash indices to shift depending on process execution order.
• Resolution: Replaced hash(word) with zlib.crc32(word.encode("utf-8")), guaranteeing 100% deterministic feature vectors across all Python environments, operating systems, and test runners.