More Than Image Generators: A Science of Problem-Solving using Probability | Diffusion Models скачать в хорошем качестве

More Than Image Generators: A Science of Problem-Solving using Probability | Diffusion Models

This is my entry to #SoME4, 3Blue1Brown's Summer of Math Exposition Competition! Diffusion models are typically portrayed as models that learn to denoise a corrupted image. This way, they can generate new images by gradually removing noise from a sample of pure noise. Theoretically, they are usually described as a particular type of VAE with a fixed encoder, treating the intermediate stages of the denoising process as latent variables. Through a series of painful derivations, the denoising training objective can be cast as a variational lower bound on the data likelihood, connecting it to VAEs directly, and to other maximum likelihood methods more loosely. For the amount of effort required to understand them this way, this approach never seemed to advance my understanding of diffusion models, or why we should care about them over other generative models in any meaningful way, other than "I guess we just found out they're better". This video approaches diffusion models through a different lens, one based on score matching and Langevin dynamics, that will hopefully be more intuitive and practical; in particular, I show how this perspective results in some interesting realizations, that most other resources seem to miss: 1. Image generation is the same thing as rolling a dice: they can be reduced to the same basic computational recipe; 2. How diffusion models are connected to perhaps the most influential idea in modern artificial intelligence -- gradient descent in high-dimensional spaces -- and how they elegantly extend its success from train time to test time; 3. How diffusion models separate the "creative" and "logical" capabilities of image generation into two different actors/players; 4. And other insights that are obscured by the standard denoising VAE story we've all grown used to. Main paper this video is based on: Song, Y. and Ermon, S. (2020) Generative Modeling by Estimating Gradients of the Data Distribution: https://arxiv.org/abs/1907.05600 Other references: Ho, J., Jain, A. and Abbeel, P. (2020) Denoising Diffusion Probabilistic Models: https://arxiv.org/abs/2006.11239 Although this is part of my Diffusion Model series (part 2), you don't need to watch part 1 to understand part 2; rather they each complement the other and you can watch them in any order. Timestamps: 0:00 - Diffusion models are not (only) denoisers/VAEs 3:35 - Probability primer 5:51 - Images are just samples from a probability distribution 6:49 - Assigning probability values to images 7:28 - Challenges in sampling from probability distributions 9:17 - The probability distribution that helps you sample from (almost) any other 10:47 - Examples on a toy distribution 11:18 - Components of a universal sampler (the score/"F" function) 12:05 - An algorithm that generates samples from any probability distribution (Langevin sampling) 14:55 - Intuition for each component of Langevin sampling 15:59 - The score function = gradient of the (log) probability density function 18:09 - Exercise: write a dice roll sampler from scratch using Langevin sampling 22:26 - A Langevin approach to image generation 23:48 - Visualizing score functions in increasingly high dimensions 27:15 - Diffusion models estimate unknown score functions from existing samples 29:04 - Recap of diffusion models and image space 30:07 - Diffusion models secretly predict the score function (the gradients of the distribution) 32:05 - Tying Langevin sampling into diffusion models 33:53 - Why add more noise in the denoising process 36:30 - Bumpiness of the image distribution; how this leads to problems for the "greedy" score function 38:56 - Noise as the "raw material" (high-variance detail) of an image; diffusion model turns it into low-variance patterns that are actually meaningful 40:02 - Intuition: diffusion model as a logical artist, noise as a creative artist 41:53 - Separation of creative and logical capabilities leads to better image generation 44:23 - Langevin sampling tells us that knowing the gradients of a distribution is sufficient to generate samples 46:15 - Eerie parallels with stochastic gradient descent 49:56 - Langevin sampling/diffusion models just extend gradient descent to test time