Training Pytorch Models Faster with Dask | Scott Sievert | Dask Summit 2021 скачать в хорошем качестве

Training Pytorch Models Faster with Dask | Scott Sievert | Dask Summit 2021

Recent empirical and theoretical results provide strong motivation for increasing the batch size. This results in fewer model updates to train a model. At first, that seems like an internal detail. However, if each Dask worker processes a constant number of gradients, each model update can be made agnostic to the batch size. That means that the wall-clock time required is proportional to the number of model updates, not the number of floating point operations. In this talk, I'll present software that combines those two facts (and also show the details). The main benefit: the wall-clock time required to train a model be reduced from 120 minutes to 45 minutes. This isn't a free lunch but luckily will not require any more floating point operations. === The Dask Distributed Summit is where users, contributors, and newcomers can share experiences to learn from one another and grow together. The Dask Distributed Summit provides content, information, and learning opportunities for attendees of all levels of Dask familiarity and expertise. Check out  for more information.  https://summit.dask.org/