# Your Coding Agent Can't Undo Its Own Mistakes - Tanmay Sah & Dolly Sah

## Executive summary

Coding agents often struggle with recovery after making changes, as traditional version control systems like Git only track repository changes and cannot reverse external side effects (e.g., API calls, runtime config changes). The research presented introduces 'UndoBench' and 'EvoUndo' to address this gap. EvoUndo provides a structured approach for agents to capture the original state, apply a change, and then independently verify the recovery process, significantly improving the reliability of self-modifying AI agents.

## Key takeaways

- Limitations of Traditional Version Control: Git can track changes within a repository, but it fails to cover external side effects such as external API calls, runtime configurations, or notifications. Simple retries do not guarantee reversal of side effects.
- Agent Recovery Challenges (UndoBench): In UndoBench evaluations, while agents achieved an 84% success rate in completing tasks under normal conditions, their ability to recover from induced failures dropped significantly to only 47% success rate.
- The Need for State Capture: Recovery must account for the failure stage (before, during, or after a change). Simply reverting is insufficient; the system must capture the original state (e.g., knowing a timeout failure might mean the operation succeeded but the confirmation was lost).
- EvoUndo Mechanism: EvoUndo formalizes recovery by capturing a 'state witness' before applying a change. The process involves: 1) Capturing the original state, 2) Applying the modification, 3) Attempting recovery/undo, and 4) Independent verification of the restored state.

## Technical details

- UndoBench Findings: In 600 development tasks evaluated by EvoUndo, 478 improvements increased capability, but 197 of these improvements contained recovery flaws (approximately 42% failure rate).
- Recovery Strategies: Recovery responses must be context-aware, choosing between 'retry,' 'restore,' 'compensate,' or 'escalate,' depending on the nature of the failure and what can be safely reversed.
- EvoUndo Process Flow: The process requires the agent to first read the original value, save it as a state witness, apply the change (e.g., changing a timeout from 30 to 60), and then use the witness for restoration, which is finally verified by an independent checker.

## Practical implications

- For AI agents to become reliable, their development must treat recovery capability (undoing mistakes) as a core requirement alongside performance improvement.
- Build systems integrating AI agents must implement state-witnessing mechanisms to ensure that changes are reversible and verifiable, moving beyond simple Git-level rollbacks.
- The recovery mechanism must be sophisticated enough to handle external side effects and determine if a compensatory action is needed, rather than just attempting a simple reversal.

## Topics

AI Agents, Software Reliability, Version Control, State Management, Automated Testing, UndoBench, EvoUndo

Source: https://www.youtube.com/watch?v=BT1Mo-fD-g0
