40x More Merged PRs With 2.7x the Engineers — Walden Yan, Cognition
Summary
Cognition achieved a 40x increase in merged PR throughput while growing its engineering headcount by 2.7x. This was driven by implementing a 'self-driving' software factory, where AI agents (Devin) handle processes from bug report to final merge. Key architectural changes include running all services locally via Docker Compose, rebuilding development VMs hourly, and shifting the engineering paradigm from task execution to goal-oriented 'jobs,' exemplified by maintaining an Android application in 44 days with minimal human input.
Key takeaways
-
Self-Driving Development by Default
2:02
The development process is designed to run end-to-end via AI agents, from initial bug reports to final monitoring, with human intervention reserved only for escalations.
-
Vertical Ownership and Small Pods
4:31
Large teams were broken into small pods (2-3 engineers) that hold vertical responsibility, managing everything from customer bug reports and QA testing to product management, all managed through a single linear board.
-
Infrastructure for Agent Testing
8:06
A major shift involved migrating all services to run locally using Docker Compose. Third-party services are mocked, allowing agents to test changes without touching live services. These VMs are rebuilt hourly.
-
Shift from Tasks to Jobs
12:30
The future workflow involves giving agents high-level goals (e.g., 'improve P99 latency') rather than specific tasks. Agents are expected to set up telemetry, create alerts, and optimize the system autonomously.
More digests on AI Engineering
Technical details
-
Local Service Execution and CI
486s
All services are configured to run locally using Docker Compose. This allows coding agents to test changes in a controlled environment, eliminating the need to interact with staging or live cloud deployments.
-
VM Management and Reliability
556s
Development VMs are rebuilt every hour to ensure agents always work on a fresh version of the codebase. Agents are also tasked with fixing VM snapshots if builds fail due to rapid code changes.
-
AI Optimization and Hardware Tuning
670s
Agents are used for recursive self-improvement, such as optimizing CI processes or tuning hardware. For example, agents were used to push the throughput of Vera Rubin GPUs beyond the performance of previous GB 300 GPUs.
-
Long-Duration AI Projects
An Android application was maintained in parity with an iOS blueprint over 44 days, with the agent recursively improving the application and performing live QA testing, resulting in minimal human interaction.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.