Romit Maulik speaks on Stability Analysis in the Latent Space of Actions for Reinforcement Learning скачать в хорошем качестве

Romit Maulik speaks on Stability Analysis in the Latent Space of Actions for Reinforcement Learning

Consider subscribing to this channel to be notified of future seminars! It was our pleasure to hear from Romit Maulik (Penn State University) on an interpretability approach for pretrained RL agents in Deep Reinforcement Learning! Romit introduced RL for science applications, and highlighted several works related to simulation control, architecture search, multi-fidelity applications, and stability analysis. We thank him for the great insights and discussion. More details can be found in the associated ArXiV: https://arxiv.org/abs/2502.15512 (Note: There was a delay in the start of the recording, so it kicks off from the introduction to RL portion of the talk) Check out and enjoy this great talk! Abstract: Modern deep reinforcement learning (DRL) methods have made significant advances in handling continuous action spaces. However, real-world control systems--especially those requiring precise and reliable performance--often demand interpretability in the sense of a-priori assessments of agent behavior to identify safe or failure-prone interactions with environments. To address this limitation, we propose SALSA-RL (Stability Analysis in the Latent Space of Actions), a novel RL framework that models control actions as dynamic, time-dependent variables evolving within a latent space. By employing a pre-trained encoder-decoder and a state-dependent linear system, our approach enables interpretability through local stability analysis, where instantaneous growth in action-norms can be predicted before their execution. We demonstrate that SALSA-RL can be deployed in a non-invasive manner for assessing the local stability of actions from pretrained RL agents without compromising on performance across diverse benchmark environments. By enabling a more interpretable analysis of action generation, SALSA-RL provides a powerful tool for advancing the design, analysis, and theoretical understanding of RL systems. Bio: Romit Maulik is an Assistant Professor in the College of Information Sciences and Technology at Pennsylvania State University (Penn State). He is also a co-hire in the Institute for Computational and Data Sciences at Penn State and a Joint Appointment Faculty at Argonne National Laboratory. He obtained his PhD in Mechanical and Aerospace Engineering at Oklahoma State University (in 2019) and was the Margaret Butler Postdoctoral Fellow (from 2019-2021) before becoming an Assistant Computational Scientist at Argonne National Laboratory (from 2021-2023). His group studies high-performance multifidelity scientific machine learning algorithm development with applications to various multiphysical nonlinear dynamical systems such as those that arise in fluid dynamics, geophysical modeling, nuclear fusion, and beyond. He is an Early Career Awardee of the Army Research Office. 00:00 Start 04:37 RL for Science Applications 14:38 Characterizing Pre-trained Agent Behaviour: SALSA-RL 25:58 Stability Analysis 28:21 Examples 44:10 Transient Growth Analysis and Floquet Analysis 50:02 Performance with latent dimension 51:38 Conclusions, Q&A