Notes on RL: Policy Gradient & Log Derivative Trick

Recently I followed this tutorial to implement a basic Q-Learning reinforcement learning algorithm for Flappy Bird, which surprisingly worked quite well, reaching over 10,000 points. Q-Learning essentially estimates a value $Q(s, a)$ for each state-action pair $(s, a)$, and then selects actions based on these $Q$ values. Deep Q-Network (DQN) builds on basic Q-Learning by adding a neural network to…