Multimodal AI from First Principles - Neural Nets that can see, hear, AND write. скачать в хорошем качестве

Multimodal AI from First Principles - Neural Nets that can see, hear, AND write.

Generative Large Language Models like OpenAI's GPT-4, Google's PaLM 2, and Discriminative models like ImageBind are models released in 2023 that combine visual and textual input to perform multi-modal tasks. Multimodal modeling combines multiple modalities to train neural networks - images, text, audio, etc empowering ML models to perform amazing multimodal tasks like text-image retrieval, multimodal vector arithmetic, visual question answering, and language modelling. To support the channel and access the Word documents/slides used in this video, consider JOINING the channel on Youtube or Patreon. Members get access to scripts, slides, animations, and illustrations for most of the videos on my channel! Patreon -   / neuralbreakdownwithavb   Follow on Twitter: @neural_avb In this video, I covered the essential published techniques for Multimodal Modelling and so many amazing results of the past few years that have left my jaws on the floor. Hope you enjoy it! Watch how Multimodal models generate images:    • If LLMs are text models, how do they gener...   #deeplearning #languagemodel #gpt #computervision Papers references in this video: Unifying Visual-Semantic Embeddings: https://arxiv.org/pdf/1411.2539.pdf CLIP: https://arxiv.org/abs/2102.02779 ImageBInd: https://arxiv.org/abs/2305.05665 BLIP: https://arxiv.org/abs/2201.12086 HERO: https://arxiv.org/pdf/2005.00200.pdf VL-T5: https://arxiv.org/pdf/2102.02779.pdf OFA: https://arxiv.org/abs/2202.03052 SimVLM: https://arxiv.org/abs/2108.10904 Frozen: https://arxiv.org/abs/2106.13884 Flamingo: https://arxiv.org/abs/2204.14198 MiniGPT4: https://arxiv.org/abs/2304.10592 Kosmos-1: https://arxiv.org/abs/2302.14045 PaLM-E: https://arxiv.org/abs/2303.03378 Timestamps: 0:00 - Intro 02:55 - Basics 05:05 - Contrastive Learning 07:54 - Masked Visual Language Models 10:20 - Unified Models 13:41 - Generative LLMs