ML Basics and Kernel Methods (Tutorial) by Mikhail Belkin скачать в хорошем качестве

ML Basics and Kernel Methods (Tutorial) by Mikhail Belkin

Statistical Physics Methods in Machine Learning DATE:26 December 2017 to 30 December 2017 VENUE:Ramanujan Lecture Hall, ICTS, Bengaluru The theme of this Discussion Meeting is the analysis of distributed/networked algorithms in machine learning and theoretical computer science in the "thermodynamic" limit of large number of variables. Methods from statistical physics (eg various mean-field approaches) simplify the performance analysis of these algorithms in the limit of many variables. In particular, phase-transition like phenomena appear where the performance can undergo a discontinuous change as an underlying parameter is continuously varied. A provocative question to be explored at the meeting is whether these methods can shed theoretical light into the workings of deep networks for machine learning. The Discussion Meeting will aim to facilitate interaction between theoretical computer scientists, statistical physicists, machine learning researchers and mathematicians interested in these questions. Examples of specific topics to be covered include (but are not limited to) problems such as phase transitions in optimization and learning algorithms, matrix approximation, mixing in large networks, sub-linear time algorithms, learning theory and non convex optimization. The meeting will allow structured and and unstructured interactions among the participants around the main theme. CONTACT US:spmml2017 ictsresin PROGRAM LINK:https://www.icts.res.in/discussion-me... Table of Contents (powered by https://videoken.com) 0:00:00 Start 0:00:04 Statistical Physics Methods in 0:00:09 Kernels and modern machine learning. 0:02:07 Supervised ML 0:05:16 Reproducing Kernel Hilbert Space 0:08:22 Some examples of PSD kernels 0:09:06 RKHS defined by a kernel 0:10:37 Linear kernel 0:11:49 The reproducing property 0:17:04 Practical implications 0:21:17 Theoretical "analysis" 0:26:13 Machine Learning/AI is becoming a backbone of commerce and society. 0:28:51 Goal: a model for modern ML competitive on modern data analytically tractable 0:29:30 This talk 0:31:57 Modern ML 0:34:43 "Shallow" /kernel architectures 0:34:49 Kernel learning for modern ML 0:35:59 Kernel methods for big data 0:38:03 Simple 1-D example: Heaviside function 0:38:57 Real Data: gradient descent 0:39:37 The limits of kernels 0:40:39 Eigenvalue decay 0:40:49 Computational reach of Kernel GD 0:44:17 Eigenpro Kernel 0:45:09 Eigenpro kernel learning 0:45:44 Computational reach after iterations 0:46:46 Eigenpro: practical implementation 0:47:37 EigenPro: practical implementation II 0:48:10 Eigenpro acceleration 0:48:36 Comparison with state-of-the-art 0:50:45 This talk 0:51:49 Modern ML 0:58:10 Stochastic Gradient Descent 1:00:18 Understanding SGD 1:02:27 Batch size for parallel computation 1:06:17 Empirical results: MNIST-10k. 1:09:19 Overfitting with kernels 1:18:55 Kernel overfitting/interpolation 1:27:03 Accelerated/momentum/Nesterov methods 1:28:00 Accelerated methods for kernels 1:29:14 Parting Thoughts 1:32:38 Q&A