# Engineer Your Coding Agents Like Any Other System — Zach Lloyd, Warp

## Executive summary

The shift from running coding agents locally to implementing a 'Software Factory' is presented as a necessary evolution for scalable, governed AI development. A Software Factory is defined as a code-defined, closed-loop system designed to continuously measure and improve cloud coding agents. This system moves beyond simple automation by integrating declarative configuration, comprehensive data recording, sandboxed compute, and automated self-improvement loops driven by metrics and benchmarks.

## Key takeaways

- The Limitations of Local Agents: Relying on local, interactive prompting limits throughput and creates significant challenges in cost management, security, and governance across platform teams (0:57, 1:42).
- Definition of a Software Factory: A Software Factory is a code-defined closed-loop system for continuously measuring and improving cloud coding agents. It is not merely a collection of AI teammates or personal agent swarms (3:57).
- The Closed Loop Principle: The system must be a continuous closed loop, where agents process work (from triage to verification) and the resulting data is used to drive efficiency improvements (4:42).
- New Engineering Primitives: The discipline of 'Factory Engineering' requires mastering tools like online evals, writing benchmarks, and conducting A/B tests on the factory itself, rather than just the product (7:42, 12:57).

## Technical details

- Factory Infrastructure Stack: The stack is layered: 1. Factories as Code (declarative definition, similar to Terraform); 2. Data/Context (recording all agent conversations, artifacts, and videos); 3. Compute (sandboxing for security and repeatability); 4. Inference (supporting swappable models like Claude Code, Codex, and custom harnesses); 5. Orchestration (moving work and integrating with tools like Jira and GitHub) (7:42).
- Continuous Improvement Mechanisms: Improvement is achieved through three mechanisms: 1. Measurement (DORA metrics, human interactions per PR, cost tracking); 2. Online Evals (having an agent score another agent's output, e.g., detecting redundant tests); 3. Self-Improvement (agents diagnosing failures and suggesting fixes as PRs against the factory definition) (12:57, 14:27).
- Benchmarking: To validate model or skill choices, benchmarks must be run against proprietary, real-world data, allowing teams to visualize trade-offs between cost and quality (15:57).

## Practical implications

- Teams must shift from viewing coding agents as isolated tools to viewing them as components within a measurable, engineered system.
- The focus of engineering shifts to system improvement (e.g., reducing cost, increasing throughput) rather than just feature delivery.
- Declarative configuration (Factories as Code) allows for version control, branching, and rollback of agent workflows.

## Topics

AI Engineering, Build Engineering, LLM Operations, CI/CD, Agentic Systems, DevOps, Warp, Warp Agentic Terminal

Source: https://www.youtube.com/watch?v=0LeT996Hz3E
