Loading Academy Hub...
Understand CPython references, reference counts, mutable defaults in functions, and shallow vs deep copies.
In Python, everything is an object, and variables are merely labels (references) pointing to boxed PyObject structs in memory.
id(x): Memory address of the object in CPython.is vs ==: == checks value equality; is checks memory identity (id(a) == id(b)).# ❌ DANGEROUS BUG IN ETL SCRIPTS:
def add_metadata(record, tags=[]):
tags.append("processed")
return tags
[] is evaluated once at function definition time, NOT at invocation! Every call without tags appends to the exact same list instance!tags=None and initialize inside function: if tags is None: tags = [].sys.getrefcount). Assigning b = a on a dict/list aliases the exact same mutable object—always use .copy() or copy.deepcopy() when mutating pipeline state configs!def append_record(row, buf=[]): creates one shared list once at module import time across all future Airflow/ETL runs. Always default to buf=None and initialize if buf is None: buf = [].dict record of 5 fields costs ~232+ bytes; __slots__ dataclasses slash per-record RAM by 50%+ when transforming millions of domain events before PyArrow/Parquet serialization.Review the production code, company interview Q&As, incident traces, and comprehension quiz inside this lesson.
import copy
# Mutable default demonstration
def safe_ingest_record(record_id: str, tags: list = None) -> dict:
if tags is None:
tags = []
tags.append("ingested_v1")
return {"id": record_id, "tags": tags}
rec1 = safe_ingest_record("REC_01")
rec2 = safe_ingest_record("REC_02")
print(rec1) # {'id': 'REC_01', 'tags': ['ingested_v1']}
print(rec2) # {'id': 'REC_02', 'tags': ['ingested_v1']} -> Clean separation!
# Shallow vs Deep Copy
original = {"customer": "Acme", "orders": [101, 102]}
shallow = original.copy()
deep = copy.deepcopy(original)
original["orders"].append(103)
print("Shallow:", shallow["orders"]) # [101, 102, 103] -> Mutated!
print("Deep:", deep["orders"]) # [101, 102] -> Preserved!
Bug: Record 45 contains audit tags from Record 1, 2, and 3