Stanford Online

RAG Explained: From BM25 to Agentic RAG | Stanford CS329Z Lecture 3 with Diyi Yang

Published 2026-11-11 · Duration 1:16:31

Summary

This lecture provides a comprehensive, deep dive into Retrieval-Augmented Generation (RAG), detailing its evolution from classic document retrieval to advanced, agentic systems. The discussion covers the entire RAG pipeline, including indexing strategies (e.g., chunking, contextual retrieval), retrieval methods (BM25, DPR, ColBERT), and advanced search architectures (HNSW, GraphRAG). A key focus is placed on moving beyond static RAG by incorporating agentic components like Self-RAG, CRAG, and Search-R1, which allow the system to dynamically decide whether to retrieve, what query to issue, and when to stop, thereby mitigating compound errors.

Download summary

Key takeaways

  1. RAG's Core Principle 18:43

    RAG's design principle is to prevent LLMs from relying solely on memorized knowledge by providing relevant, verifiable information 'just in time' via a retrieval mechanism. This makes the process dynamic and interpretable, allowing for citations.

  2. Hybrid and Advanced Retrieval 1:00:27

    Optimal retrieval often requires combining methods. Hybrid search (e.g., combining BM25's exact term matching with DPR's semantic understanding) and contextual retrieval (prepending surrounding context to a chunk) significantly improve performance and robustness.

  3. Vector Search Efficiency 1:05:25

    Due to the high complexity of exact search (O(ND)), Approximate Nearest Neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small World) are essential for scalable, fast retrieval.

  4. Agentic RAG for Decision Making

    Modern RAG moves toward agentic systems. Instead of a fixed pipeline, agents (e.g., Self-RAG, CRAG, Search-R1) incorporate decision-making capabilities, allowing the model to decide whether to retrieve, critique the evidence, or refine the query, which is crucial for complex, multi-step tasks.

More digests on Retrieval-Augmented Generation (RAG)

Technical details

  • Document Indexing & Chunking 3955s

    Chunking strategies range from simple fixed-size segmentation to advanced methods like semantic segmentation or AI-driven dynamic chunking, which use LLMs to detect natural breakpoints. Contextual retrieval enhances units by prepending surrounding context to the segment, improving both embedding and BM25 performance.

  • Retrieval Components 4222s

    Retrievers include: 1) **BM25**: A classic, non-neural, term-frequency approach (no training required). 2) **DPR (Dense Passage Retrieval)**: A neural approach using a query encoder and a passage encoder, trained with contrastive loss to maximize the inner product similarity between relevant pairs. 3) **ColBERT**: A method that improves scoring by keeping a separate vector for each token, calculating a max similarity score across all query tokens and document tokens.

  • Advanced Indexing Structures 4300s

    For complex knowledge, indexing can be hierarchical (PeaKtor, building a tree of summaries from segments) or graph-based (GraphRAG, extracting entities and relationships to build a knowledge graph, allowing retrieval across multiple documents and modalities).

  • Encoder Architectures 4460s

    Retrieval can use **Bi-encoders** (query and passage encoded separately, e.g., DPR), **Cross-encoders** (query and passage processed together for full attention), or **Late Interaction** (encoding independently but calculating interaction/score at the final stage, e.g., ColBERT).

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.