Meta, Google & NVIDIA Engineer Manish Gupta on the Future of GPUs & AI Infrastructure | Ep 6 | TIS скачать в хорошем качестве

Meta, Google & NVIDIA Engineer Manish Gupta on the Future of GPUs & AI Infrastructure | Ep 6 | TIS

In this episode of The Inference Show, we are joined by Dr. Manish Gupta, a leading expert in AI training, GPU performance, and compiler optimization. Manish brings a wealth of experience from his work at Magic, Meta, Google, NVIDIA, AMD, and Qualcomm, where he has been at the forefront of scaling custom compute systems, optimizing large language models, and pioneering GPU innovations. Manish takes us on a journey through his career and dives deep into the cutting edge of AI infrastructure, discussing: His early experiences with low-level assembly programming*and how it shaped his approach to GPU optimization. Insights from working on NVIDIA’s Cutlass project, which powers nearly every major AI training pipeline today. The bottlenecks in scaling massive models like LLaMA, including precision trade-offs and checkpointing strategies. How test-time compute and reinforcement learning are redefining the future of inference and model performance. Why programmability and software-hardware co-design are key for emerging AI accelerators. The evolution of GPU architecture from Volta to Blackwell and what it means for developers. His vision for the future of AI-driven code generation and automated kernel development. — Manish’s work has directly influenced AI training and inference at scale, with his contributions now used by every major company developing foundational models. From building core libraries to optimizing for cutting-edge hardware, he offers a rare perspective on where AI infrastructure is heading and the deep technical challenges ahead. — In this video 0:00 – Coming Up… 2:05 – Guest Intro: Manish Gupta’s AI & GPU Journey 5:16 – Nvidia, Matrix Multiplication & AI Breakthroughs 10:09 – Qualcomm, AMD & Early Career Lessons 14:16 – Building GPU Abstractions at Nvidia 17:33 – Why AI Became the Central Focus 22:03 – What Success Looks Like for a GPU Engineer 32:21 – Architectural Breakthroughs in Nvidia GPUs 38:44 – Why Nvidia’s Success is Software + Hardware 43:10 – Cutlass, Triton, and Many Paths to GPU Programming 45:01 – Is AI Coming for GPU Engineers’ Jobs? 47:09 – How to Learn Parallel Programming Today 52:16 – Scaling Llama: Bottlenecks in Training Large Models 56:16 – The Future of Inference: Test-Time Compute & RL 1:03:11 – GPUs vs. Custom AI Hardware 1:05:18 – Looking Ahead: Manish’s Next Big AI Challenge 1:07:00 – Closing Reflections & Farewell — About Dr. Manish Arora Dr. Arora is the co-founder of LearnDesk and Insaito, where he leads marketing and sales. He has grown LearnDesk into a global platform supporting over 25,000 businesses and is now focused on Automatan, an AI platform for automating business workflows. With 80+ patents and decades of industry experience, Dr. Arora brings deep technical and strategic insights to every conversation. — *The Inference Show* Stay connected with us and explore more about our guests, topics, and future episodes: 🔗 Manish Gupta:   / mguptaiitr   🔗 Automatan:   / automatan-ai   🔗 Insaito:   / insaito-inc   🔗 Dr. Manish Arora:   / manish7901   🔗 Vivek Puri:   / vivpuri   🔗 LearnDesk: https://www.learndesk.us/ 🔗 Insaito: https://www.insaito.com/ — Be our next guest by emailing us at [email protected] We’d love to hear your insights and have you join the conversation!