# Your Old Prompts Are Making Claude Worse - Lance Martin, Anthropic

## Executive summary

This talk provides a comprehensive guide for optimizing the use of frontier Large Language Models (LLMs) like Claude, focusing on improving performance and significantly reducing operational costs. Key strategies include auditing prompts to remove outdated anti-patterns (e.g., 'think step-by-step'), maximizing the use of the prompt cache in multi-turn applications, and designing robust, 'climbable' evaluation sets. For complex, long-running systems, the speaker emphasizes the architectural necessity of 'Managed Agents' to ensure reliability and security by decoupling the agent's 'brain' from its 'hands'.

## Key takeaways

- Audit Prompts for Anti-Patterns: Frontier models do not require outdated instructions like 'think step-by-step' or 'double-check your work.' These anti-patterns increase cost and can sometimes degrade performance. Use the `checkup prompt audit` or `Claude API prompt audit` commands to automatically remove these unnecessary instructions.
- Maximize Prompt Cache Usage: In multi-turn applications, the prompt cache is crucial for cost efficiency and speed. Since it is built at a fraction of the base token rate, engineers must structure prompts to maintain a high cache hit rate. The `Claude code` environment offers diagnostics to help optimize prompt structure for caching.
- Design Climbable Evaluations: An evaluation must be 'climbable' for autonomous improvement. Good design requires low run-run variance, expected performance scaling with model/thinking, and sufficient headroom at the frontier. Use `Claude API build eval` to assist in this process.
- Implement Managed Agents for Reliability: For long-duration agents (days/weeks of work), reliability and security are paramount. The recommended architecture is 'decoupling the brain from the hands,' where the agent harness (brain) and the sandbox (hands) are separate, recoverable components. Secrets must be managed via vaults, never given direct access to the agent.

## Technical details

- Prompt Auditing: The `checkup prompt audit` skill (available in Claude code) automatically identifies and removes anti-patterns (e.g., verification rituals, boosters) that cause frontier models to expend excessive tokens, leading to cost savings and performance improvements.
- Prompt Cache Mechanics: The prompt cache is a cost-saving mechanism for multi-turn applications. It operates on a strict prefix match, meaning any byte change in the prefix will invalidate the cache. Tracking the prompt cache hit rate is vital for application economics.
- Hill Climbing: Frontier models are highly effective at autonomously improving performance on a well-structured evaluation. The `Claude API hill climb` skill helps automate this process but requires careful auditing to prevent overfitting to the evaluation distribution, which would render the improvements useless in production.
- Cost Optimization via Hill Climbing: The hill climber can be directed to jointly optimize for cost and performance. By searching through model/effort combinations, it can find configurations (e.g., Opus 5.5 low effort) that achieve parity or better performance while drastically reducing token usage (e.g., achieving a 90% cost reduction).
- Managed Agents Architecture: This architecture separates the agent's core logic (harness/brain) from its execution environment (sandbox/hands). The session log is maintained as an independent, append-only, recoverable object, ensuring state persistence even if the harness or sandbox fails.

## Practical implications

- Always audit existing prompts using available tools to eliminate unnecessary 'thinking' rituals and boosters.
- For any multi-turn application, prioritize structuring the prompt to maximize the prompt cache hit rate to manage costs.
- When designing automated testing, focus on creating evaluations that are robust, low-variance, and have clear headroom for improvement, rather than sampling only known failure cases.
- When building long-running agents, adopt a decoupled architecture (Managed Agents) to ensure state recovery and security.

## Topics

Prompt Engineering, LLM Optimization, Evaluation Design, Agent Architecture, Cost Management, Artificial Intelligence, Anthropic Blog (on Managed Agents), Claude code / Claude API

Source: https://www.youtube.com/watch?v=ys6YQ2yfBpM
