Unifying Vision and Language Models with Mohit Bansal - 636 скачать в хорошем качестве

Unifying Vision and Language Models with Mohit Bansal - 636

Today we're joined by Mohit Bansal, Parker Professor, and Director of the MURGe-Lab at UNC, Chapel Hill. In our conversation with Mohit, we explore the concept of unification in AI models, highlighting the advantages of shared knowledge and efficiency. He addresses the challenges of evaluation in generative AI, including biases and spurious correlations. Mohit introduces groundbreaking models such as UDOP and VL-T5, which achieved state-of-the-art results in various vision and language tasks while using fewer parameters. Finally, we discuss the importance of data efficiency, evaluating bias in models, and the future of multimodal models and explainability. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confi... 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/ 📖 CHAPTERS =============================== 00:00 - Background 03:48 - Unification of Models for Vision and Language Tasks 09:30 - Improving Coreference in 3D Captions through Grounding 13:35 - Language Model Learns from Videos for Text Tasks 18:04 - Efficient Task Unification with Language Generation 21:46 - Advancements in Multimodal Models with VL-T5 28:10 - Learning to Generate and Edit Documents for Understanding 30:25 - Flexible Analogy: Learning through Generation and Editing 32:38 - Efficiency with Unification and Adapters in Models 35:54 - Enhancing Video Efficiency with Keyframes & Audio 41:27 - Assessing Diverse Aspects of Text-to-Image Model Evaluation 47:35 - Connecting Object Detection to Image Evaluation Metrics 48:47 - Incorporating Nonverbal Cues and Challenges in Interactive Learning 52:16 - Exploring Program-Based Explainability and Tools-Based Validation of Generated Content 🔗 LINKS & RESOURCES =============================== Paper: Unifying Vision, Text, and Layout for Universal Document Processing - https://arxiv.org/abs/2212.02623 Paper: Unifying Vision-and-Language Tasks via Text Generation - https://arxiv.org/abs/2102.02779 Paper: DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models - https://arxiv.org/abs/2202.04053 For this episode's complete show notes, head over to https://twimlai.com/go/636. 📸 Camera: https://amzn.to/3TQ3zsg 🎙️Microphone: https://amzn.to/3t5zXeV 🚦Lights: https://amzn.to/3TQlX49 🎛️ Audio Interface: https://amzn.to/3TVFAIq 🎚️ Stream Deck: https://amzn.to/3zzm7F5