Berlin Buzzwords 2014: Andreas Neumann - Transactions and Abstractions for Apache HBase скачать в хорошем качестве

Berlin Buzzwords 2014: Andreas Neumann - Transactions and Abstractions for Apache HBase

When Apache Hadoop was first introduced to the Open Source, it was focused on implementing Google's Map/Reduce, a framework for batch processing of very large files in a distributed system. Built for running on large cluster of commodity hardware, Hadoop also included a cluster resource manager to divide the capacity of the cluster between the various Map/Reduce jobs that can run at a given time. A Hadoop cluster, however, is not always fully utilized, and idle resources would best be used for other compute-intensive tasks like real-time stream processing, message passing, or graph algorithms. Unfortunately, the cluster resource manager was specialized in Map/Reduce execution and did not allow other types of workloads. This situation changes with Apache Hadoop 2.0 and its resource manager, YARN, which is decoupled from the Map/Reduce execution engine. It allows running arbitrary workloads on a cluster, as long as they are built against YARN's application manager interface: YARN manages the cluster's resources as a set of "containers". Each application can obtain containers from YARN and is then free to use them for any type of computation. Hence different types of distributed applications can share a single cluster. This allows for more innovation, agility, and better hardware utilization. Read more: https://2014.berlinbuzzwords.de/sessi... About Andreas Neumann: https://2014.berlinbuzzwords.de/user/... Website: https://berlinbuzzwords.de/ Twitter:   / berlinbuzzwords   LinkedIn:   / berlin-buzzwords   Reddit:   / berlinbuzzwords