# Without Caching, Your AI Agent Costs 10x More - George Cameron, Artificial Analysis

## Executive summary

The cost of running AI agents is driven by more than just the model's intelligence; key cost drivers include input tokens, the cache hit rate, and the number of steps (turns) required for the task. George Cameron of Artificial Analysis emphasizes that neglecting caching can lead to costs increasing by orders of magnitude (e.g., from $21 to over $200 per task). Developers must make a strategic trade-off decision, classifying tasks as either 'unlimited intelligence' (where more intelligence always improves results) or 'limited intelligence' (where a cheaper model suffices).

## Key takeaways

- Cost Drivers in AI Agents: The cost of an agent task is determined by input tokens, the cache hit rate, and the number of steps (turns). The biggest cost driver for long-horizon agent tasks is the number of input tokens, followed by the proportion of those tokens that are cache hits, and finally, the number of turns per task. (12:59)
- Impact of Caching: The cache hit rate is critical. If caching fails (zero cache hits), the cost can dramatically increase, illustrating a potential jump from $21 per task to over $200 per task. (13:38)
- Model Selection Trade-offs: Developers should not automatically choose the most intelligent model. They must analyze the task's requirements against the cost-intelligence Pareto front. Tasks should be categorized as 'unlimited intelligence' (where more intelligence is always better) or 'limited intelligence' (where a cheaper model provides sufficient results). (15:30)
- Cost Reduction Over Time: The cost to access a given level of intelligence (e.g., a 30 on the Intelligence Index) is rapidly decreasing. Historically, the cost could be over $1 per task, but it can fall to the $0.10 - $1.00 range within a year. (10:18)

## Technical details

- AI Benchmarking Scope: Artificial Analysis benchmarks the full AI stack, including the agent layer, language models (for intelligence), image/video/speech models, and the hardware/inference layers, to understand inter-related tradeoffs. (0:00)
- Intelligence Index: The Intelligence Index is a synthesis metric of 10 different evaluations covering areas like knowledge work, scientific reasoning, and coding/terminal use, designed to help developers make decisions for next-generation use cases. (0:00)
- Agent Cost Calculation: The cost per task must account for 'reasoning tokens' and 'input tokens,' not just output tokens. The cost calculation must also factor in the number of steps (turns) taken during agentic trajectories. (12:59)
- Model Examples: The talk compared models like Opus (Anthropic) and GLM 5.3 Flash (open weights), demonstrating that cheaper models can be highly capable, provided the task is not limited by intelligence requirements. (15:30)

## Practical implications

- When building agents, prioritize optimizing the cache hit rate and minimizing the number of turns per task to drastically reduce operational costs.
- Before deployment, create a custom benchmark (using tools like Optima) to map your specific task's intelligence requirements against various model costs, rather than relying solely on general intelligence rankings.
- Determine if your task is 'limited intelligence' (where cost savings are possible) or 'unlimited intelligence' (where budget must be allocated for maximum performance).

## Topics

AI Cost Optimization, LLM Benchmarking, Agent Architecture, Token Economics, Build Engineering, Artificial Analysis, AA Analyst Agent

Source: https://www.youtube.com/watch?v=prGP0huECPo