PGMO days 2020: Regret Bounds for Kernel-Based Reinforcement Learning скачать в хорошем качестве

PGMO days 2020: Regret Bounds for Kernel-Based Reinforcement Learning

Title : Regret Bounds for Kernel-Based Reinforcement Learning Authors: Omar Darwiche Domingues, Pierre Menard, Matteo Pirotta, Emilie Kaufmann and Michal Valko Abstract: We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning problems whose state-action space is endowed with a metric. We introduce Kernel-UCBVI, a model-based optimistic algorithm that leverages the smoothness of the MDP and a non-parametric kernel estimator of the rewards and transitions to efficiently balance exploration and exploitation. Unlike existing approaches with regret guarantees, it does not use any kind of partitioning of the state-action space. For problems with $K$ episodes and horizon $H$, we provide a regret bound of $\widetilde{O}\left( H^3 K^{\frac{2d}{2d+1}}\right)$, where $d$ is the covering dimension of the joint state-action space. We empirically validate Kernel-UCBVI in continuous MDPs with sparse rewards. Links : Domingues, O. D., Ménard, P., Pirotta, M., Kaufmann, E., & Valko, M. (2020). A kernel-based approach to non-stationary reinforcement learning in metric spaces.