AI Costs: Tokens, Energy, Model Choice & API Keys | Stanford CS193T Lecture 7 with Cynthia Bailey
Summary
This lecture provides a deep dive into the operational costs and resource consumption of Large Language Models (LLMs). It analyzes token usage (input vs. output), the efficiency gains from caching, and the critical role of model selection (e.g., Claude Opus 5.5 vs. Claude Haiku) in multi-step agentic workflows. The discussion also covers the technical mechanics of billing via API keys and the security risks associated with key leakage in web applications.
Key takeaways
-
Caching significantly reduces input token costs.
17:13
Because the attention mechanism calculation is expensive, precomputing and reusing results via cache reads can provide a discount of up to 90% compared to raw input tokens.
-
Model choice must be balanced against latency.
23:07
Bigger models are not always better; latency (the time to process results) compounds significantly in multi-agent workflows, making thoughtful model assignment crucial.
-
Unused skills still incur token overhead.
14:09
Every enabled skill places its text into the context window on each prompt, meaning unused skills still contribute to input token costs, necessitating agent decluttering.
-
API keys are essential for distributed cost sharing.
29:51
API keys allow a computer program to act as a user, enabling systems to charge individual users for their usage, which is critical for sustainable tool deployment.
Technical details
-
Token Usage Metrics
380s
Input tokens include the system prompt, settings, and enabled tool connectors. Output tokens include the LLM's response and any internal reasoning/chain-of-thought generation.
-
Duration Metrics
465s
API duration measures the time spent waiting for the model (including network transit and data center load), while wall-clock time is the total elapsed time since the session began.
-
Agent Architecture and Guardrails
1150s
Complex systems, like the Wayfinder AI tutor, require multiple agents (Input Guardrail, Response Generation, Output Guardrail) to prevent hallucination and ensure compliance, which increases latency.
-
API Key Security Pitfalls
1650s
A common security vulnerability is encoding an API key into public-facing website code (e.g., HTML), making it visible to anyone who views the page source.
Mentioned resources
- Claude Code
- Claude Opus 5.5
- Claude Haiku
- Hopscotch
- Open Router
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.