Hybrid RL + Imitation Learning for Robotics with Ray at RAI Institute скачать в хорошем качестве

Hybrid RL + Imitation Learning for Robotics with Ray at RAI Institute

The nature of machine learning in robotics demands complex abstractions of hardware and training/simulation layers to use a combination of RL and IL (Imitation Learning). In this respect, policy learning for robotics rarely fits on one kind of machine. For instance, massive simulation parallelization with GPU physics and rendering in Isaac Lab demand RTX‑class GPUs, while policy training benefits from large VRAM and FLOPs. Over the past year we have built our infra on Ray that hides this hardware/software diversity and lets researchers focus on science, not sys‑admin. Our platform offers: Unified orchestration – a single Ray workflow allows to train full state RL models that are used to train multi-task IL policy, and evaluation in simulation. Heterogeneous GPU scheduling – placement groups assign Isaac Lab simulators to RTX workers and gradient computation to A100/H100 trainers without manual mapping. Isolated deployment targets – the same job definition that trains a policy can package it into a lightweight Ray Serve micro‑service that runs next to the robot or on a nearby edge server, shielding control code from research churn. During the live demo we will: Launch a hybrid RL‑IL run that automatically provisions both Nvidia-RTX GPUs and A100/H100 nodes. Watch Ray adapt the cluster as workloads shift from simulation to learning to evaluation. Deploy the resulting policy to an isolated runtime on the robot—ready for immediate testing. Attendees will leave with practical design patterns for juggling simulator‑heavy and large‑scale network training inside one reproducible Ray ecosystem, plus insights on meeting real‑time robotics constraints while remaining GPU‑efficient.