5 useful things you'll learn in my new post-training textbook (shipping now!)

Newsletter · Watch/Read

• Nathan Lambert’s new textbook, “Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs,” is now shipping, offering a comprehensive resource for those with a CS bachelor’s degree to master post-training LLMs.
• The book provides intuitions for RL algorithms (e.g., PPO’s surrogate objective reducing to six regions), explains crucial system design factors like off-policy data and loss aggregation (DAPO, Dr. GRPO), and demystifies “distillation” by tracing its evolution to multi-teacher on-policy distillation (MOPD).
• It aims to develop “research taste” in readers, crucial for understanding which research matters given the rapid 3-9 month cycle for research to land in frontier models, and is freely available online with a 12-hour course and code.