Hugging Face Journal Club: Direct On-Policy Distillation
This paper proposes Direct On-Policy Distillation, a cheap way to transfer RL benefits from small models to large ones by using policy shifts as dense reward signals.
Read articleTopic
Knowledge Distillation
All digests tagged Knowledge Distillation
Suggested route
Topic journey
Move from earlier context through the available material to the newest signals. This route is a reading aid, not a difficulty ranking.