Building more efficient AI with vLLM ft. Nick Hill | Technically Speaking with Chris Wright скачать в хорошем качестве

Building more efficient AI with vLLM ft. Nick Hill | Technically Speaking with Chris Wright

Massive language models are here, but getting them to run efficiently is a major challenge. In this episode, Red Hat CTO Chris Wright sits down with Nick Hill, Senior Principal Software Engineer at Red Hat, to explore vLLM, an open-source project revolutionizing AI inference. They discuss how innovations born from systems-level thinking are making AI more practical and accessible. 00:00 - The challenge of running massive language models 00:59 - Nick Hill's journey from IBM Watson to generative AI 03:03 - What is vLLM and why is it different? 05:41 - Optimizing the KV Cache and GPU utilization 07:35 - PagedAttention: Virtual memory for your GPU 09:51 - Speculative decoding and its CPU parallels 11:50 - The future of distributed and heterogeneous hardware in AI 16:38 - How open source and community are accelerating AI innovation Learn More: vLLM Project: https://vllm.ai/ Sky Computing Lab at UC Berkeley: https://sky.cs.berkeley.edu/ Follow us: Chris Wright:   / chris-wright-b733851   Chris Wright:   / kernelcdub   What is Technically Speaking? Technically Speaking taps into emerging technology trends with insights from leading experts across the globe and Red Hat CTO Chris Wright. The series blends deep-dive discussions, tech updates, and creative short-form content, solidifying Red Hat’s role as a pioneer in technology innovation and open source thought leadership. Want to participate? Leave us a comment if there's a topic or a guest you'd like to see featured. Watch More Technically Speaking: YouTube Playlist:    • Technically Speaking with Chris Wright   Show Page: https://www.redhat.com/en/technically... Subscribe to Red Hat's YouTube channel: https://www.youtube.com/redhat/?sub_c... #RedHat #vLLM #AIInference #TechnicallySpeaking #OpenSource