Multi-Cluster Apache Kafka with Cluster Linking ft. Nikhil Bhatia скачать в хорошем качестве

Multi-Cluster Apache Kafka with Cluster Linking ft. Nikhil Bhatia

https://cnfl.io/podcast-episode-174 | Note: This episode was recorded when Cluster Linking was in preview mode. It’s now generally available as part of the Confluent Q3 ‘21 release on August 17, 2021. Infrastructure needs to react in real time to support globally distributed events, such as cloud migration, IoT, edge data collection, and disaster recovery. To provide a seamless yet cloud-native, cross-cluster topic replication experience, Nikhil Bhatia (Principal Engineer I, Product Infrastructure, Confluent) and the team engineered a solution called Cluster Linking. Available on Confluent Cloud, Cluster Linking is an API that enables Apache Kafka® to work across multi-datacenters, making it possible to design globally available distributed systems. As industries adopt multi-cloud usage and depart from on-premises and single cluster operations, we need to rethink how clusters operate across regions in the cloud. Cluster Linking as an inter-cluster replication layer into Confluent Server, allowing you to connect clusters together and replicate topics asynchronously without the need for Connect. Cluster Linking requires zero external components when moving messages from one cluster to another. It replicates data into its destination by partition and byte for byte, preserving offsets from the source cluster. Different from Confluent Replicator and MirrorMaker2, Cluster Linking simplifies failover in high availability and disaster recovery scenarios, improving overall efficiency by avoiding recompression. As a great cost-effective alternative to Multi-Region Cluster, Cluster Linking reduces traffic between data centers and enables inter-cluster replication without the need to deploy and manage a separate Connect cluster. With low recovery point objective (RPO) and recovery time objective (RTO), Cluster Linking enables scenarios such as: ► Migration to cloud: Remove the complexity layer of self-run datacenters with fully managed cloud services. ► Global reads: Enable users to connect to Kafka from around the globe and consume data locally. Empowering better performance and improving cost effectiveness. ► Disaster recovery: Prepare your system for fault tolerance, from datacenter, regional, or cloud-level disasters, ensuring zero data loss and high availability. Find out more about Cluster Linking architecture and set your data in motion with global Kafka. EPISODE LINKS ► Announcing the Confluent Q3 ‘21 Release: https://cnfl.io/confluent-q3-21-relea... ► Introducing Cluster Linking in Confluent Platform 6.0: https://cnfl.io/cluster-linking-in-co... ► What is Cluster Linking?: https://cnfl.io/what-is-cluster-linki... ► Resurrecting In-Sync Replicas with Automatic Observer Promotion ft. Anna McDonald: https://cnfl.io/resurrecting-in-synnc... ► Join the Confluent Community: https://cnfl.io/join-confluent-commun... ► Learn more with Kafka tutorials, resources, and guides at Confluent Developer: https://cnfl.io/visit-confluent-devel... ► Live demo: Intro to Event-Driven Microservices with Confluent: https://cnfl.io/event-driven-microser... ► Use PODCAST100 to get an additional $100 of free Confluent Cloud usage: https://cnfl.io/podcast100-for-conflu... ► Promo code details: https://www.confluent.io/confluent-cl... ABOUT CONFLUENT Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io. #confluentcloud #apachekafka #kafka #confluent