Day 5:Working with Files
Battle-tested daily lesson note • 100% runnable code, interview Q&A, test suite & interactive quiz
Week 1 — Day 5: Working with Files (pathlib), JSON, Markdown & Error Handling (try/except)
Mentor's Note (10+ YOE Staff AI Engineer POV): Why is Day 5 a huge milestone for your AI Engineering career?
- Every RAG Pipeline Starts With Reading Files: Whether you are indexing
.mdrunbooks,.jsonAPI logs, or.csvdatasets, you need to traverse folders safely across Windows (\) and Linux (/) usingpathlib.Path.- LLMs Speak JSON: When an LLM returns a structured output, it arrives over the network as a JSON string (
'{"reply": "...", "covered_by_doc": true}'). You must parse it withjson.loads()and catch malformed JSON withtry / except json.JSONDecodeErrorso your application never crashes!
Part 1: Modern File Paths with pathlib.Path (Goodbye os.path!)
Never hardcode string slashes like "runbooks\\payments.md" (breaks on Linux Docker containers!) or "runbooks/payments.md". Always use Python's pathlib.Path, which uses the / operator to join paths cleanly on any OS:
from pathlib import Path
# Get the directory where the current .py script lives:
BASE_DIR = Path(__file__).resolve().parent
# Join paths safely using `/` (works on Windows, macOS, and Linux!):
runbook_file = BASE_DIR / "runbooks" / "payments_service.md"
# Find all .md files inside a folder:
for md_path in (BASE_DIR / "runbooks").glob("*.md"):
print(md_path.name)
Part 2: Reading & Writing Files Safely with Context Managers (with open)
Why Do We Always Use with open(...)?
If you write f = open("data.txt") and an error happens on the next line before f.close() runs, the Operating System file descriptor stays locked in RAM (causing a Too many open files crash on production servers!).
The with statement (Context Manager) guarantees that Python automatically closes the file the microsecond the block exits—even if an exception crashes inside the block!
# Always specify encoding="utf-8" (Especially important on Windows where the default is cp1252!)
with open(runbook_file, mode="r", encoding="utf-8") as f:
content = f.read()
# Or even shorter using pathlib:
content = runbook_file.read_text(encoding="utf-8")
[!WARNING] Windows Encoding Trap (
UnicodeDecodeError: 'charmap' codec can't decode byte...): On Linux/macOS, Python defaults toutf-8. On Windows,open()withoutencoding="utf-8"defaults to legacy Windows-1252 (cp1252), which crashes the moment a Markdown file contains an emoji, an arrow (→), or a smart quote! Always passencoding="utf-8"!
Part 3: Parsing & Saving JSON (json.loads vs json.dumps)
| Function | What It Does | Memory Mnemonic |
|---|---|---|
json.loads(json_str) | Converts a JSON String Python dict / list | load s = load from string (used on LLM API responses!) |
json.dumps(py_dict, indent=2) | Converts a Python dict JSON String | dump s = dump to string |
json.load(file_obj) | Reads JSON directly from an open file dict | From disk file |
json.dump(py_dict, file_obj) | Writes a Python dict directly to an open file | To disk file |
Part 4: Defensive Error Handling (try / except / else / finally)
In AI Engineering, external things fail constantly: an LLM returns invalid JSON (```json { ... } ``` wrapped in Markdown backticks!), a file is missing, or a network call times out.
import json
raw_llm_output = '```json\n{"reply": "Restart pod", "covered": true}\n```'
try:
# 1. Strip accidental Markdown code fences if the LLM added them:
cleaned = (
raw_llm_output.strip().removeprefix("```json").removesuffix("```").strip()
)
parsed_data = json.loads(cleaned)
except json.JSONDecodeError as err:
# 2. Runs ONLY if json.loads() failed:
parsed_data = {"reply": "Error parsing LLM JSON", "error": str(err)}
else:
# 3. Runs ONLY if `try` succeeded with zero exceptions:
print("Successfully parsed JSON!")
finally:
# 4. ALWAYS runs (cleanup/metrics logging), whether an error happened or not!
print("Request audit complete.")
[!IMPORTANT] Senior Engineering Rule: Never write a bare
except:orexcept Exception: passthat silently swallows errors! Always catch the specific exception (except json.JSONDecodeError:orexcept FileNotFoundError:) and log the context.
Part 5: Hands-On Lab — Markdown Runbook Section Parser & Safe JSON Extractor (runbook_parser.py)
Check out src/runbook_parser.py (in Tab 2: Codes — click Run in Browser to execute). This module builds the exact Header-Aware Markdown Parser and Resilient LLM JSON Parser that we will use in Month 1 Capstone and Month 3 RAG!
Run Day 5 Code & Tests:
python week01_computer_terminal_git_python_basics/day05_files_json_errors_modules/src/runbook_parser.py
python -m unittest discover -s week01_computer_terminal_git_python_basics/day05_files_json_errors_modules/tests -v