# AI Costs: Tokens, Energy, Model Choice & API Keys | Stanford CS193T Lecture 7 with Cynthia Bailey

## Executive summary

This lecture provides a deep dive into the operational costs and resource consumption of Large Language Models (LLMs). It analyzes token usage (input vs. output), the efficiency gains from caching, and the critical role of model selection (e.g., Claude Opus 5.5 vs. Claude Haiku) in multi-step agentic workflows. The discussion also covers the technical mechanics of billing via API keys and the security risks associated with key leakage in web applications.

## Key takeaways

- Caching significantly reduces input token costs.: Because the attention mechanism calculation is expensive, precomputing and reusing results via cache reads can provide a discount of up to 90% compared to raw input tokens.
- Model choice must be balanced against latency.: Bigger models are not always better; latency (the time to process results) compounds significantly in multi-agent workflows, making thoughtful model assignment crucial.
- Unused skills still incur token overhead.: Every enabled skill places its text into the context window on each prompt, meaning unused skills still contribute to input token costs, necessitating agent decluttering.
- API keys are essential for distributed cost sharing.: API keys allow a computer program to act as a user, enabling systems to charge individual users for their usage, which is critical for sustainable tool deployment.

## Technical details

- Token Usage Metrics: Input tokens include the system prompt, settings, and enabled tool connectors. Output tokens include the LLM's response and any internal reasoning/chain-of-thought generation.
- Duration Metrics: API duration measures the time spent waiting for the model (including network transit and data center load), while wall-clock time is the total elapsed time since the session began.
- Agent Architecture and Guardrails: Complex systems, like the Wayfinder AI tutor, require multiple agents (Input Guardrail, Response Generation, Output Guardrail) to prevent hallucination and ensure compliance, which increases latency.
- API Key Security Pitfalls: A common security vulnerability is encoding an API key into public-facing website code (e.g., HTML), making it visible to anyone who views the page source.

## Practical implications

- Implement robust cost monitoring by analyzing usage reports (e.g., using the `slash usage` command) to track total cost, API duration, and token consumption.
- Design multi-agent systems with specific, appropriately sized models for each task (e.g., using a smaller, cheaper model for a simple guardrail check).
- Ensure API keys are stored securely outside of public view when integrating LLMs into web applications to prevent unauthorized usage and billing fraud.
- Mitigate latency risks in complex workflows by optimizing agent handoffs and minimizing unnecessary iterations.

## Topics

AI Economics, LLM Engineering, Agentic Workflows, API Security, Claude Code, Claude Opus 5.5, Claude Haiku, Hopscotch, Open Router

Source: https://www.youtube.com/watch?v=l07K2cdLofM
