# The Agents Were Fine. Everything Around Them Broke. - Joey Wang, Harvey

## Executive summary

Joey Wang discusses the architecture and scaling challenges of Spectre, an internal cloud agent platform at Harvey. The core thesis is that while LLM agents are powerful, the true product is the 'harness'—the surrounding infrastructure that manages, validates, and merges agent-written code. Key architectural improvements included implementing follow-up loops for automated fixes, making all platform actions callable via CLI (allowing agents to drive other agents), and introducing a fair launch queue to manage massive task bursts and prevent cluster saturation.

## Key takeaways

- The Product is the Harness, Not the Model: The value lies in the infrastructure (the 'harness') that manages the agent workflow, including checking out every ripple, opening the PR, and ensuring the merge. This layer is what enables agents to operate on the company's real stack and secrets, which is necessary for scaling beyond local machine capabilities.
- Automating the Full Development Lifecycle: The most impactful feature is the follow-up loop, which allows an agent to read a CI failure, push a fix, and continue the process automatically, moving beyond just writing initial code. This transforms an agent that merely writes code into one that 'finishes the job.'
- Enabling Agent-Driven Workflow via CLI: By making every action (e.g., publishing a page, starting a task, turning on automerge) callable from the Command Line Interface (CLI), the platform allows any agent—whether running on a laptop, in a sandbox, or via CI—to drive the entire workflow, significantly increasing the platform's utility.
- Scaling for Massive Task Bursts: To handle sudden, large-scale task increases (e.g., 900 tasks in four hours), a fair launch queue was implemented. This queue ensures that tasks are processed in a fair, orderly manner, preventing cluster overload and maintaining system stability.

## Technical details

- Agent Platform Architecture (Spectre): Spectre is the cloud agent platform built inside Harvey. It is designed to sit underneath various agent interfaces (Cursor, CodeEx, etc.), providing access to the company's real stack, secrets, and alerts.
- Automated Fix Loops: The follow-up loop reads CI failures and automatically pushes subsequent fixes, automating the debugging cycle and ensuring the PR moves closer to merge status without manual intervention.
- Workflow Automation and Callability: The platform was designed so that if a human could click a function (e.g., 'publish a page'), an agent could call it via the CLI. This expanded the scope from terminal-only tasks to web app and browser-based actions.
- Resource Management and Scaling: The implementation of a fair launch queue and dispatcher was critical for handling task bursts, improving task latency from nearly 2 seconds to 70 milliseconds during high load.
- Agent Guardrails: To prevent runaway automation, a guardrail was implemented: if an automation agent fails five times in a row, it automatically switches itself off and can be assigned a budget or expiry date.

## Practical implications

- When building agent-driven systems, focus on the 'harness'—the surrounding infrastructure that manages the workflow, rather than just the LLM model itself.
- Implement automated feedback loops (like CI failure reading) to allow agents to complete the entire job, not just write the initial code.
- Design the system to expose all core functionalities as callable APIs (via CLI) to ensure agents can drive any part of the platform.
- Build resource throttling and fair queuing mechanisms to manage unpredictable, high-volume task bursts from automated agents.

## Topics

Cloud Agents, CI/CD, Workflow Automation, LLM Agents, System Scaling, DevOps, Joey Wang, Harvey, Harvey

Source: https://www.youtube.com/watch?v=ktbAVYsAxpk
