Скачать с ютуб Part 1 of 3 — Proximal Policy Optimization Implementation: 11 Core Implementation Details в хорошем качестве

Part 1 of 3 — Proximal Policy Optimization Implementation: 11 Core Implementation Details

Proximal Policy Optimization (PPO) is one of the most popular reinforcement learning algorithms, and works with a variety of domains from robotics control to Atari games to chip design In this video, we dive deep into 11 core implementation details of PPO and build the algorithm from scratch in PyTorch, step-by-step. --- Source code: https://github.com/vwxyzjn/ppo-implem... Related blog post: https://iclr-blog-track.github.io/202... Background music: Flutes Will Chill — https://artlist.io/song/48722/flutes-... --- 0:00 Introduction 2:01 Dev environment 2:19 Common variables 3:18 Tensorboard 4:02 Weights and Biases 6:05 1. Vector environment 9:53 Agent setup 10:13 2. Layer initialization 11:48 3. Adam's epsilon 12:15 Training loop 15:36 4. Learning rate annealing 17:15 5. General Advantage Estimation 18:49 6. Minibatch update 20:22 7. Advantage normalization 20:45 8. Clipped objective 21:07 9. Value loss clipping 21:32 10. Entropy loss 22:12 11. Global gradient clipping 22:30 Debug variables 23:10 Bonus. Early stopping 24:17 Visualize training on W&B