LLM Inference Deep Dive: TensortRT-LLM, KV Cache, Prefill vs Decode, TTFT, TPOT | NVIDIA NCP-GENL скачать в хорошем качестве

LLM Inference Deep Dive: TensortRT-LLM, KV Cache, Prefill vs Decode, TTFT, TPOT | NVIDIA NCP-GENL

Why are your expensive GPUs sitting idle while your text generation maxes out? In this complete guide to LLM inference, we strip away the hype to show you the actual hardware mechanics of how Large Language Models generate text. Whether you're an AI architect trying to cut cloud costs, an ML engineer optimizing an API, or studying for the NVIDIA Certified Professional Generative AI (NCP-GENL) exam, you need to understand these underlying physics. 🚀 Ace your AI Certification! Practice Tests for NCP-GENL: https://preporato.com/certificates/ge... Practice Tests for other AI/IT certifications: https://preporato.com/exams What we cover in this deep dive: 0:00 - The GPU Utilization Problem 1:09 - GPU Architecture 2:30 - Prefill Phase (Compute-Bound) 3:33 - The KV Cache Explained 4:28 - The Decode Phase (Memory-Bound) 5:37 - TTFT, TPOT, and Throughput 6:55 - The Batching Trade-Off 8:03 - Inference Engines & AOT Compilation 9:05 - Continuous Batching (In-Flight Batching) 9:09 - PagedAttention & VRAM Fragmentation 10:42 - The Triton Inference Server 11:21 - NVIDIA GPU Operator 12:06 - Triton Ensemble Models 14:10 - NVIDIA NIMs