Optimizing Large-Scale RL with SGLang | Chenyang Zhao | AER Labs скачать в хорошем качестве

Optimizing Large-Scale RL with SGLang | Chenyang Zhao | AER Labs

This talk addresses the Training-Inference Mismatch problem commonly encountered in large-scale reinforcement learning (RL) environments, and presents practical strategies for system-level optimization. It introduces the concept of partial rollout in SGLang and the technique of CUDA Graph Aware Refit to diagnose performance bottlenecks and significantly reduce model weight synchronization time. These methods offer effective solutions for optimizing complex RL pipelines. Speaker: Chenyang Zhao X: @GenAI_is_real Github: @zhaochenyang20 LinkedIn: @chayennezhao TimeStamp: 00:03 Introduction 03:15 Overview 04:51 Overview of RL Systems & 5-Stage Lifecycle 11:23 Partial Rollouts in Multi-Turn RL 24:31 Advanced Topic 1: Train-Inference Mismatch & Importance Sampling 33:38 Advanced Topic 2: CUDA Graph-Aware Model Refit 41:48 Advanced Topic 3: Refit Optimization in Distributed Environments Hosted by AER Labs with ModuLabs South Korea #ai #SGLang #inference #opensource #llm #nvidiaomniverse #quantization #deeplearning #infrastructure