Scaling to 100k RPS: Strangler Figs, CDC, and Kafka скачать в хорошем качестве

Scaling to 100k RPS: Strangler Figs, CDC, and Kafka

Hey everyone. A few months ago, I posted a system design challenge involving a massive 50TB Oracle monolith taking 100,000 writes per second. This video is a deep dive into how you’d actually approach this migration without stopping the world or losing a single bit of data. We move from first principles—viewing the database not as a static table, but as a transaction log—to building out a full Change Data Capture (CDC) pipeline using Kafka and Debezium. This is "Advanced" system design, so we’ll be going into the weeds on data consistency, idempotency, and the Strangler Fig pattern. The Log: What every software engineer should know about real-time data's unifying abstraction (Jay Kreps, 2013) This is the definitive paper on why logs are the heartbeat of distributed systems. Link:   / log-what-every-software-engineer-should-kn...   The Strangler Fig Application (Martin Fowler) The core pattern for breaking down monoliths incrementally. Link: https://martinfowler.com/bliki/Strang... Debezium Documentation If you want to see how CDC is implemented in the real world. Link: https://debezium.io/documentation/ #systemdesign #systemdesigninterview #ai #codingchallenge