Stanford Online

Optimizing AI Agents: Prompts vs. Weights (GEPA, GRPO) | Stanford CS329Z Lecture 9 with Michael Ryan

Published 2026-11-17 · Duration 1:20:34

Summary

This lecture provides a comprehensive guide to optimizing AI agents, contrasting prompt-based (context) optimization with weight-based fine-tuning. The speaker, Michael Ryan, details various techniques, including FShot optimization, Reflective Optimization (OPRO), and genetic algorithms (GEPA), alongside weight optimization methods like Supervised Fine-Tuning (SFT) and Generalized Policy Optimization (GRPO). The discussion emphasizes that optimization is crucial for 'last mile adaptation' when general models are insufficient for specific domain tasks, and that the choice between prompt and weight optimization depends on resource constraints and system requirements.

Download summary

Key takeaways

  1. Context Optimization vs. Weight Optimization 18:25

    Context optimization (prompting) is useful for API-based models where fine-tuning is unavailable, and is often faster for rapid iteration. Weight optimization (fine-tuning) is necessary when context length is a constraint or when a cheaper, smaller model is required.

  2. Bootstrap Reasoning and FShot Optimization 5:38

    Methods like DSPI (Bootstrap Random Search) and Fantastically Ordered Prompts use rejection sampling to generate high-quality few-shot examples and reasoning traces from unlabeled data, which are then used in the prompt context.

  3. Reflective Optimization (OPRO) 26:40

    OPRO uses a meta-prompt to allow the LLM to reflect on past attempts (including their scores) and generate new, improved instructions, effectively using the LLM as an optimizer.

  4. Weight Optimization Methods (SFT & RL) 39:10

    SFT uses a 'teacher' model's successful trajectories to train a 'student' model. RL methods like GRPO (Generalized Policy Optimization) reward the student model for task completion, allowing the model to learn from diverse and complex environments.

  5. Agents as Optimizers 1:05:00

    Advanced techniques like the Darwin Gödel Machine and Auto-Research demonstrate that agents can self-modify their own code or harness based on performance metrics, leading to open-ended self-improvement.

More digests on AI Agents

Technical details

  • FShot Optimization 750s

    This involves providing examples in the prompt. Fantastically Ordered Prompts specifically addresses the model's sensitivity to the ordering of these examples, using metrics like global and local entropy to find optimal sequences.

  • Bootstrap Random Search (DSPI) 1050s

    This method generates high-quality few-shot examples by running the program multiple times (rejection sampling) and only keeping traces that score well on a metric, which are then combined and tested.

  • Reflective Optimization (OPRO) 1750s

    OPRO uses a meta-prompt to feed the LLM a history of past instructions and their scores, asking the LLM to generate a new, improved instruction.

  • Genetic Prompt Evolution (GEPA) 2800s

    GEPA starts with a candidate pool of prompts. It uses a Pareto frontier approach to filter out non-optimal candidates and then generates new candidates via 'reflective prompt mutation' or 'system aware merge.'

  • Reinforcement Learning (GRPO) 3450s

    GRPO is an efficient RL method that uses a group advantage calculation (based on mean reward and standard deviation) instead of a separate value model, reducing the number of models required compared to PO.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.