# Anthropic's New Study: You're Not Controlling AI as Much as You Think You Are

## Executive summary

The video addresses 'AI disempowerment,' the risk that users over-rely on AI coding agents, leading them to defer critical judgment even when they believe they are making independent decisions. Citing an Anthropic study, the speaker details how agents' recommendations can strongly shape expert judgment. To mitigate this, three practical strategies are proposed: focusing on priorities first, prototyping decisions, and forcing the agent to justify alternative options.

## Key takeaways

- AI Disempowerment is a Universal Trap: When using coding agents, users often assume they are making critical decisions, but they may be deferring judgment to the agent's recommendations, which is a significant problem in planning sessions.
- Anthropic Study Findings: Analysts followed the agent's lean 86% of the time across four analytical choices, suggesting a high degree of deference. Furthermore, 77% of analysts used their own agent's arguments when asked to explain their choices, even if they didn't credit the agent as the source.
- Solutions for Maintaining Agency: Three strategies to counteract over-reliance are: 1) Asking the agent about priorities first (focusing on the 'why' before the 'how'). 2) Using prototyping to build and test different options to gain certainty. 3) Playing devil's advocate by asking the agent, 'What would change your advice?'

## Technical details

- Anthropic Agent Study Methodology: The study tested whether experts truly make decisions or defer to AI agents. Agents were set up to favor one option (the 'lean') on each decision, and this lean was randomized. The agent was also prompted to argue its case but could change its mind if sufficient evidence was presented.
- Agentic Coding Process: The speaker describes a typical planning session where the agent uses an 'ask user question tool' to present multiple-choice questions, and the agent generally recommends the top answer, leading to passive acceptance of the recommendation.
- LLM Confidence and Bias: The speaker notes that LLMs are highly confident in their outputs, even when incorrect, contributing to the tendency for users to accept recommendations without critical review. The underlying model used for both building and reviewing code can lead to non-independent results.

## Practical implications

- When using coding agents, always build in a step that forces the agent to ask about the user's core priorities before generating solutions.
- To overcome uncertainty, build small prototypes for different options (UI, API endpoints) rather than relying solely on the agent's suggested plan.
- When accepting an agent's recommendation, challenge it by asking, 'What would have to change in the problem or codebase for option B to be optimal?'

## Topics

AI Ethics, Agentic Workflow, LLM Limitations, Build Engineering, Prompt Engineering, Anthropic's study: Agents' Judgment Calls, Plan With Priorities skill

Source: https://www.youtube.com/watch?v=BlbxhsItZPg