Topic
All digests tagged Policy Gradient
Suggested route
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.
Score Centering stabilizes Off-policy RL for LLMs by correcting the policy gradient loss when the training and inference policies mismatch. The method uses a Top-K approximation for efficient computation.
Read article
The lecture details advanced RL techniques for LLMs, covering Policy Gradient derivations and the stabilization mechanism of Proximal Policy Optimization (PPO). It shows how to model Chain-of-Thought prompting as an MDP problem.
Read article
This session covers Reinforcement Learning, detailing the Markov Decision Process (MDP) framework and the Policy Gradient algorithm. Key concepts include defining states, actions, transition dynamics, and using the Bellman equation for optimal value estimation.
Read article