Commit Log :Apache Kafka скачать в хорошем качестве

Commit Log :Apache Kafka

Apache kafka:Commit Log This lesson delves into the concept of a commit log and how it relates to Kafka. Kafka is based on the concept of a commit log. Understanding the design and concept of the commit log will help us understand Kafka better. A commit log is a sequence of records where each record has its own unique identifier. There are some characteristics of a commit log that differentiate it from other kinds of logs and storage: Records can only be appended at the end of a commit log. Records are immutable (can’t be modified). A commit log is always read from left to right. The record offsets can be used to specify the start and end of a read. Commit logs have been used in software for a long time and serve as a reliable record of what has happened in a system and in what order. Databases and caches often used commit logs to reconstruct the state of the system from a crash or for optimizing performance. Commit logs are also known as write ahead logs. In databases, any change is first written to the write-ahead log before the database itself. The write to a commit log is relatively faster, even on disk, than a corresponding write to a complex data-structure such as an index or a B-tree on disk. This allows the database to defer making the changes to the disk and only reflect them in memory. The database can rewind and go through the series of events in the commit log and make the changes to the database asynchronously. Even if the changes haven’t been committed and a crash occurs, the database can still recover using the changes recorded by the commit log. The commit log can be thought of as a story of events that are ordered in time and can be used to recreate or replicate changes. For instance, a replica of a database can read the changes from a commit log to bring itself up to speed with the state of the active database replica. Commit logs are simple, fast, and can handle large volumes of data better than a traditional relational database. Another benefit of commit logs is that a complex system can continue to work when certain subcomponents face failure. These components can consult the commit log when coming back online and replay the events to get to the current state of the system. Kafka’s design and data model is inspired by the commit log. You’ll see several parallels between the two.