What Does an RL Parameter Space Look Like? скачать в хорошем качестве

What Does an RL Parameter Space Look Like?

In this video, we use random guessing to explore the linear parameter space of the CartPole problem. This gives us an idea of what the parameter space of machine learning problems look like, and helps us build an intuition for how moving through a parameter space will affect the performance of that model. With this simple problem, we can use random guessing to find a very good solution, but in more complex problems, we'll want to use more complex models that have larger parameter spaces that can't be explored effectively by random guessing. So instead we'll use methods that estimate the gradient of the parameter space with respect to the expected reward, and move our parameters in the direction which increases that expected reward. We'll cover these techniques in future videos. Note: In this video, we only used a linear layer with weights for the environment variables without using a bias variable. It turned out that adding a random bias variable consistently made the model perform worse. Can you guess why? If you think about what the bias would do, it makes sense. It would make the model favor moving the cart one way over the other. And since we want the cart to stay upright and in the middle of the screen, it makes sense that adding a bias would lower performance. So in this video, I didn't include exploring using a bias variable. Also, the fewer parameters there were to plot, the easier it was to visualize them. OpenAI CartPole environment: 💻 https://github.com/openai/gym/wiki/Ca... Join our Discord community: 💬   / discord   Connect with me: 🐦 Twitter -   / elliotwaite   📷 Instagram -   / elliotwaite   👱 Facebook -   / elliotwaite   💼 LinkedIn -   / elliotwaite   🎵 ksolis - Nobody Else (   • ksolis - Nobody Else  )