Nishanth Anand - The permanent and transient framework for continual reinforcement learning скачать в хорошем качестве

Nishanth Anand - The permanent and transient framework for continual reinforcement learning

The ability to learn continually from new experiences is a hallmark of natural intelligence. In the mammalian brain, this ability is often attributed to Complementary Learning Systems (CLS), where a slow-learning neocortex acquires structured, generalizable knowledge, and a fast-learning hippocampus rapidly encodes new experiences to aid adaptation. These two systems are believed to provide a robust balance between stability and plasticity---a trade-off that is central to continual learning. Inspired by these principles, this talk presents the Permanent and Transient (PT) framework, a conceptual approach for estimating predictive knowledge in Continual Reinforcement Learning. Specifically, we decompose the agent's predictive knowledge, such as value functions and successor features---into permanent and transient components. The permanent component captures stable, long-term structure in the predictions, while the transient component rapidly adapts estimates to the current situation. We develop novel representations and update rules to efficiently learn this decomposition. We provide theoretical guarantees for these algorithms and demonstrate that our framework outperforms competitive baselines across a variety of continual learning benchmarks. Finally, we close the loop between RL and neuroscience by showing that the PT framework offers a normative explanation for the dopamine ramping phenomenon. Nishanth Anand is a Ph.D. candidate in Computer Science at McGill University and Mila, supervised by Prof. Doina Precup. His research focuses on Continual Reinforcement Learning, where he develops novel algorithms inspired by cognitive science and neuroscience to help AI agents adapt to non-stationary environments. Grounded in both theoretical rigour and practical utility, Nishanth has authored first-author papers at NeurIPS and ICML. Beyond his research, he has co-instructed graduate-level RL courses at McGill and Polytechnique Montreal, served as a lead organizer for the Mila RL Workshop and the "RL Sofa" meeting series, and currently mentors several M.Sc. and Ph.D. students in his lab. This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Rahul Narava Gusti Winata, Leads of our Reinforcement Learning group for their dedication in organizing this event. If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker. Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommuni....