Geometric Intuition for Training Neural Networks скачать в хорошем качестве

Geometric Intuition for Training Neural Networks

Leo Dirac (@leopd) gives a geometric intuition for what happens when you train a deep learning neural network. Starting with a physics analogy for how SGD works, and describing the shape of neural network loss surfaces. This talk was recorded live on 12 Nov 2019 as part of the Seattle Applied Deep Learning (sea-adl.org) series. References from the talk: Loss Surfaces of Multilayer networks https://arxiv.org/pdf/1412.0233.pdf Sharp minima papers: -Modern take https://arxiv.org/abs/1609.04836 -Hochreiter, Schmidhuber 1997 http://www.bioinf.jku.at/publications... SGD converges to limit cycles: https://arxiv.org/pdf/1710.11029.pdf Entropy-SGD: https://arxiv.org/abs/1611.01838 Parle: https://arxiv.org/abs/1707.00424 FGE: https://arxiv.org/abs/1802.10026 SWA: https://arxiv.org/pdf/1803.05407.pdf SWA implementation in pytorch: https://pytorch.org/blog/stochastic-w...