From Gaussian Process to Neural Tangent Kernel - A Guide to Infinitely Wide Neural Networks скачать в хорошем качестве

From Gaussian Process to Neural Tangent Kernel - A Guide to Infinitely Wide Neural Networks

Introduction to infinitely wide neural networks algorithms #NNGP and #NTK. 0:00 From Gaussian Process to Neural Tangent Kernel 0:19 Outline 0:58 Introduction to Gaussian Process 3:24 Gaussian Process Regression(#GPR) 12:11 Example of GPR 16:07 GPR with Kernel 17:07 Deep Neural Network as Gaussian Process and NNGP Algorithm 17:55 Single Layer Neural Network 22:35 Arc-Cos Kernel 30:36 Multilayer Neural Network 32:59 Bayesian Inference with Gaussian Process and NNGP Numerical Algorithm 40:47 NNGP Experiment 45:40 Neural Tangent Kernel(NTK) 46:41 Dual Space and Bilinear Form 49:13 NTK at Initialization 1:03:59 NTK during Training 1:14:50 NTK Extension 1:17:18 Neural Tangent: NNGP and NTK Library reference: https://github.com/google/neural-tang... https://github.com/brain-research/nngp A. Jacot et al. 2018, Neural Tangent Kernel: Convergence and Generalization in Neural Networks S. Arora et al. 2019, On Exact Computation with an Infinitely Wide Neural Net R. Navok et al. 2019, Neural Tangents: Fast and Easy Infinite Neural Networks in Python J. Lee et al. 2018, Deep Neural Networks as Gaussian Process Z. Li et al. 2019, Enhanced Convolutional Neural Tangent Kernels Liu et al. 2019, When Gaussian Process Meets Big Data: A Review of Scalable GPs A. Rahimi et al. 2007, Random Features for Large-Scale Kernel Machines Y. Cho et al. 2009, Kernel Methods for Deep Learning J. Lee et al. 2019, Wide Neural Networks of Any Depth Evolves as Linear Models Under Gradient Descent J. Hron et al. 2020, Infinite Attention: NNGP an d NTK for Deep Attention Networks A. G Matthews et al. 2018, Gaussian Process Behavior in Wide Deep Neural Networks T. Beckers 2020, An introduction to Gaussian Process Models U.H. Gerlach 2016, The Dual of a Vector Space: From the Concrete to the Abstract to the Concrete https://www.maths.tcd.ie/~pete/ma1212... https://www.cs.princeton.edu/courses/... https://rajatvd.github.io/NTK/