Tutorial 2B Hands On Reinforcement Learning for recommender systems скачать в хорошем качестве

Tutorial 2B Hands On Reinforcement Learning for recommender systems

RecSys 2021 RecSys 2022 Hands-on Reinforcement learning for recommender systems – From Bandits to SlateQ to Offline RL with Ray RLlib by Kourosh Hakhamaneshi (Anyscale, USA) and Christy Bergman (Anyscale, USA) Traditional supervised ML techniques are efficient at capturing detailed patterns for recommender systems (RecSys) , but the models are static and do not easily adapt to users with changing preferences and behaviors. It is natural to model recommender systems as repeated decision-making processes. Each user action (for example clicking on a search or recommendation result) has an impact on immediate next actions and the user’s long-term satisfaction or LTV. Each action in the sequence may yield an immediate (short-term) engagement, but the more interesting (longer-term) reward is not known until the user completes their interaction cycle. Reinforcement learning (RL) is gaining traction as a complementary approach to supervised learning for RecSys due to RL’s sequential decision-making process and its ability to learn from delayed rewards. Recent advances in offline reinforcement learning, off-policy evaluation, and more scalable, performant system design with the ability to run code in parallel, have made RL more tractable for the RecSys real time use cases. In this hands-on Tutorial, you will learn about RLlib, which is the most comprehensive open-source reinforcement learning framework, built for production workloads. RLlib is built on top of Ray, an easy-to-use, open-source, distributed computing framework for Python that can handle complex, heterogeneous applications. Ray and RLlib run on compute clusters on any cloud without vendor lock. Since Ray is open-source, it is possible to bring new innovations faster to your members. Using Colab notebooks, we will combine theoretical concepts of RL with practical exercises. You will leave with a complete, working example of parallelized Python RL code using Ray RLlib for RecSys on a github repo. LEARNING OBJECTIVES. IN THIS TUTORIAL YOU WILL: • Customize a RecSys RL environment (Recsim with Long Term Satisfaction) using OpenAI Gym APIs. • Train and hyperparameter tune a RLlib algorithm (SlateQ, MARWIL) using Ray Tune • Checkpoint, Save, and Load the RL model using RLlib • Use RL techniques in offline learning to initialize a RL policy and keep training and evaluating it • Use Python decorators to Deploy and Serve the trained recommender model using Ray Serve • Visualize results PREPARE FOR YOUR HANDS-ON TUTORIAL TRAINING This tutorial is aimed for an audience with introductory to intermediate experience in Python, deep learning, and reinforcement learning who are interested in using RL methods in their recommender systems. • Bring your own laptop. All software will be pre-installed for you in Colab notebooks. • Have a Google account, such as the one you use for gmail. To get the most from your hands-on learning experience, following is a recommended reading list, if you need a quick refresher, prior to taking the tutorial class: • Basic knowledge of Python (Intro to Python) • Deep learning using either PyTorch or TensorFlow • Reinforcement learning (Intro to RL)