Anomaly Detection with Apache Spark - Sean Owen скачать в хорошем качестве

Anomaly Detection with Apache Spark - Sean Owen

Data Science London Meetup - Anomaly Detection There has been an explosion of interest in Apache Spark as a new, alternative computing paradigm for Hadoop. It offers something to interest data scientists of all stripes, from interactive REPL to distributed functional programming to implementations of standard machine learning techniques. In fact, it promises big scalability improvements over MapReduce for iterative algorithms, like k-means clustering, which can be used to detect anomalous data in a huge data set, for example. This session will walk through a complete example of anomaly detection using Apache Spark and it's MLlib subproject, as applied to the well-known network intrusion detection data set from KDD Cup '99. It will impart a taste of Scala (Spark's native language), Spark's core concepts like RDDs, and usage of MLlib for k-means clustering, in real-time on a Hadoop cluster. It will also introduce the concept of k-means clustering and how a data scientist would iteratively improve clustering in a session with Spark. Sean is Director of Data Science at Cloudera, based in London. Before Cloudera, he founded Myrrix Ltd, a company commercializing large-scale real-time recommender systems on Apache Hadoop. He has been a primary committer and VP for Apache Mahout, and co-author of Mahout in Action. Previously, Sean was a senior engineer at Google. He holds an MBA from the London Business School and a BA in Computer Science from Harvard. Sorry about the audio quality. http://datasciencelondon.org