# Stanford CS193T Thinking with AI | Autumn 2026 | Managing Sources

## Executive summary

This lecture provides an advanced overview of managing the context window in Large Language Models (LLMs), moving beyond simple prompting. Key topics include the function of system prompts, automatic memory collection, and the implementation of Retrieval-Augmented Generation (RAG) to handle context overflow. A significant portion of the discussion focuses on the architectural vulnerabilities of LLMs, specifically prompt injection, and the importance of 'strategic withholding' to mitigate bias and security risks. The speaker emphasizes that the context window is a single, contiguous stream of tokens, and that external data sources (like web searches) introduce dangerous security risks.

## Key takeaways

- Context Window Composition: The context window is a single, contiguous stream of tokens that includes the system prompt, user prompts, model responses, uploaded documents, and web search results. Structurally, there are no clear, separate boundaries between these components, which is critical for understanding vulnerabilities. [~18:00]
- Retrieval-Augmented Generation (RAG): RAG is a solution for grounding LLMs when source material exceeds the context window capacity. It involves performing a search over a curated, external data store (the RAG data store) and selectively injecting only the most relevant document content into the context window. [~28:00]
- Prompt Injection Vulnerability: Because the LLM cannot reliably distinguish between its own system instructions and external text (e.g., a web document), a malicious document can contain text that instructs the model to ignore previous safety guidelines and execute harmful actions (e.g., sending spam emails). This is a major security concern. [~32:00]
- Strategic Withholding: To avoid bias or sycophancy, users should intentionally withhold information from the context window. For example, framing a request as evaluating a candidate's resume (rather than evaluating one's own) can elicit more candid, objective advice. [~45:00]

## Technical details

- Context Window Management: The context window is not limited to the immediate prompt/response pair; it accumulates all chat history, system preferences, uploaded files, and web search results. Tools like ChatGPT allow users to manage this via 'Personalization' settings, which essentially modify the system prompt. [~01:30]
- System Prompts and Customization: System prompts dictate the model's behavior (e.g., tone, formatting, level of challenge). Users can override or augment the default system prompt through platform settings (e.g., ChatGPT's personalization box or Claude's instructions). [~02:10]
- Memory and Data Leakage: Major AI tools automatically glean and remember information (e.g., location, job) from past chats, injecting this 'memory' into the context window of new sessions. Users must be aware of and control these memory collection settings. [~04:00]
- Skills and Gems: These features allow users to save complex, repeatable instructions (detailed scope of work prompts) within the tool itself, acting as reusable assets that function similarly to a persistent system prompt. Gemini calls these 'Gems,' while Claude uses 'Skills.' [~55:00]

## Practical implications

- When designing AI-powered workflows, always assume the context window is a single, unsegmented data stream, making security boundaries difficult to enforce.
- Implement RAG architectures using curated, controlled data stores rather than relying solely on the LLM's general training knowledge to ensure factual grounding.
- When developing agents or automated prompts, incorporate 'guardrails' that explicitly instruct the model to resist external or conflicting instructions, mitigating prompt injection risks.
- For sensitive tasks, utilize temporary or incognito chat modes to prevent chat history from polluting the context window and compromising privacy.

## Topics

Context Window Management, Retrieval-Augmented Generation (RAG), LLM Security and Vulnerabilities, Prompt Engineering, AI Ethics and Bias Mitigation, ChatGPT, Claude, Gemini, Gemini Notebook (formerly Notebook LM)

Source: https://www.youtube.com/watch?v=8Ok6ZYILp2Q
