Netflix: Cascading Microservice Meltdown & The Missing Circuit Breaker
Netflix Forensic IncidentStaff Reliability & Resilience Engineer$370,000 - $450,000 / yr
Netflix: Cascading Microservice Meltdown & The Missing Circuit Breaker
“Welcome to Netflix Chaos Engineering. A third-tier service slowed down from 20ms to 4,000ms. Within 90 seconds, the entire Netflix homepage was returning 500 errors. You will demonstrate how thread pool isolation, timeout budgets, and the Circuit Breaker pattern prevent cascading system failure.”
A minor latency spike in the Netflix Personalization Recommendation service caused the API Gateway worker thread pool to saturate. Inbound requests backed up, cascading across 40 downstream microservices and taking down playback authorization for 45 million viewers.