Скачать с ютуб DreamDiffusion - Thought to Image Generation | Paper Summary в хорошем качестве

DreamDiffusion - Thought to Image Generation | Paper Summary

In this video we will explain DreamDiffusion, a new method to generate images directly from brain signals, which was presented in a recent research paper titled "DreamDiffusion: Generating High-Quality Images from Brain EEG Signals". We first explore the motivation for thought-to-image generation comparing to text-to-image generation. Then we review benefits of using EEG signals comparing to previous methods that generated images from brain signals that have used fMRI. Following that, we explain the idea of DreamDiffusion, which is to leverage the power of Stable Diffusion for generating high quality images, by replacing the input text embeddings from CLIP, with EEG embeddings. This idea comes with challenges, since generating semantic representations for EEG data is not trivial, and also Stable Diffusion was not trained with these EEG embeddings, so we then move on to describe how the researchers overcame these challenges. One method is to use self-supervised learning to train the EEG encoder, by learning to predict masked tokens in the input EEG signals. Second, they also fine-tune Stable Diffusion with the EEG embeddings, and also try to make the EEG embeddings similar to the CLIP embeddings which Stable Diffusion was trained on, by bringing them to the same embedding space. Paper page on arxiv - https://arxiv.org/abs//2306.16934 👍 Please like & subscribe if you enjoy this content ---------------------------------------------------------------------------------- Support us - https://paypal.me/aipapersacademy ---------------------------------------------------------------------------------- Chapters: 0:00 Introducing DreamDiffusion 1:09 Why EEG Signals? 2:07 DreamDiffusion High-Level Idea 3:53 Masked Tokens Pre-training 5:06 Adapting Stable Diffusion