Matthew Rocklin Dask A Pythonic Distributed Data Science Framework PyCon 2017 скачать в хорошем качестве

Matthew Rocklin Dask A Pythonic Distributed Data Science Framework PyCon 2017

"Speaker: Matthew Rocklin Dask is a general purpose parallel computing system capable of Celery-like task scheduling, Spark-like big data computing, and Numpy/Pandas/Scikit-learn level complex algorithms, written in Pure Python. Dask has been adopted by the PyData community as a Big Data solution. This talk focuses on the distributed task scheduler that powers Dask when running on a cluster. We'll focus on how we built a Big Data computing system using the Python networking stack (Tornado/AsyncIO) in service of its data science stack (NumPy/Pandas/Scikit Learn). Additionally we'll talk about the challenges of effective task scheduling in a data science context (data locality, resilience, load balancing) and how we manage this dynamically with aggressive measurement and dynamic scheduling heuristics. Slides can be found at: https://speakerdeck.com/pycon2017 and https://github.com/PyCon/2017-slides"