HPC Café: Inference in the Age of Reasoning Models скачать в хорошем качестве

HPC Café: Inference in the Age of Reasoning Models

Speaker: Dr. Séverine Habert, NVIDIA Date: November 11, 2025 Slides: https://hpc.fau.de/files/2025/11/infe... Abstract: This presentation explores how distributed and disaggregated inference techniques enable scalable execution of large language models (LLMs), particularly in the context of reasoning and agentic AI. It highlights architectural optimizations such as KV caching, prefix reuse, KV-cache aware routing and KV-cache offloading which improve performance, reduce latency, and support efficient deployment at the cluster level of inference workloads. Material from past events is available at: https://hpc.fau.de/teaching/hpc-cafe/