05-c LFD: Growth function: effective model size for infinite models. скачать в хорошем качестве

05-c LFD: Growth function: effective model size for infinite models.

Machine Learning From Data, Rensselaer Fall 2020. Professor Malik Magdon-Ismail talks about generalization for infinite learning models. In this lecture we begin the theoretical discussion of the link between training and testing for infinite hypothesis sets. The main goal is to introduce an effective effective number of hypotheses which will replace the role of the size of the hypothesis set in the Hoeffding bound. This effective number of hypotheses is the growth function. We introduce the growth function and discuss some examples. We argue that an exponential growth function is not of much use in the error bar, so our next goal is to polynomialy bound the growth function. This is the fifth lecture in a "theory" course focusing on the foundations of learning, as well as some of the more advanced techniques like support vector machines and neural (deep) networks that are used in practice. Level of the course: Advanced undergraduate, beginning graduate. Knowledge of probability, linear algebra, and calculus is helpful. Material is from Chapter 2 of "Learning From Data", amlbook.com, 2012.