KL regularization against a reference policy can stabilize and improve RL training of LLMs, but computing the KL term requires an additional forward pass through the reference model. We show that asynchronous RL settings that avoid this extra cost by defining the reference policy to be the inference policy (the trainer policy from ∆ steps [ ]
Andrej Karpathy wrote a great post last year on how to train a neural network to play the Atari game Pong by using the Policy Gradients reinforcement learning (RL) algorithm. Given the game s state as input, the neural network outputs a probability with which we should move the Pong paddle up or down. I converted [ ]
My research group has been discussing Artificial Neuron-Glia Networks lately. These algorithms add artificial astrocytes to the traditional Artificial Neural Network scheme, and they may also feature a Genetic Algorithm in lieu of back-propagation. See http://www.ncbi.nlm.nih.gov/pubmed/21526157 for an example. To better understand the implementation of a neural net, I constructed one that is…