How to Build 1000 Microservices with Kafka and Thrive - Natan Silnitsky скачать в хорошем качестве

How to Build 1000 Microservices with Kafka and Thrive - Natan Silnitsky

In today's environment of globally distributed large scale web applications, reactive, event-driven design is mandatory. In this talk, Natan showcases how we implemented a reliable, scalable, event-driven, distributed Mega Microservices architecture on top of Kafka but without Kafka streams. We, at Wix Engineering, have built powerful (soon-to-be-os) libraries for data streaming and data storage using Kafka features like compacted logs and exactly once semantics. These libraries offer low latency, high throughput event processing and also high volume data-storage with very simple and easy to understand APIs that work seamlessly together with other technologies such as gRPC. Natan presents the challenges faced while creating and fine-tuning such a system. An architecture that is comprised of a 1,000 micro-services that need to handle incoming traffic from multiple data centers while preserving eventual consistency and high throughput.