Your AI Agents Shouldn't Care Where They Run - Nikhil Harithas, Factory
Summary
The presentation details Factory's solution for deploying autonomous AI agents (like Droid) in highly regulated, complex enterprise environments (e.g., banks). The core challenge is ensuring that agents maintain operational sovereignty and cannot simply 'borrow' the user's identity or run on default cloud setups. The solution, Open Compute, is an open-source layer designed to abstract the entire compute lifecycle, allowing agents to operate within a customer's private VPC, regardless of the underlying infrastructure (EC2, Kubernetes, etc.). This architecture emphasizes control, observability, and the ability to run non-deterministic software on the customer's own terms.
Key takeaways
-
The Problem of Agent Sovereignty
4:04
In regulated industries, agents cannot rely on simply attaching to a user's identity. The fundamental questions are: Who is the agent acting on behalf of, and where does it run? (2:44)
-
Open Compute Abstraction Layer
9:02
Open Compute is an open-sourced project designed to abstract the compute lifecycle for any agent, allowing deployment on the customer's chosen compute layer (e.g., EKS, EC2) without requiring the customer to manage the complex compute lifecycle themselves. (5:42)
-
The Compute Plane Pattern
13:50
The system pattern involves a client talking to a back end, which uses a Compute Gateway to talk to Open Compute. This process utilizes a specific 'Driver' (e.g., EC2 driver, Kubernetes driver) to spin up a dedicated sandbox within the customer's VPC. (8:30)
-
Advanced Agent Capabilities
The architecture supports advanced agent patterns like snapshotting, forking, and resuming sessions, which are critical for multi-agent orchestration and handling long-running processes that exceed standard cloud session limits. (12:24)
Technical details
-
Open Compute Architecture
934s
Open Compute is designed to abstract the compute lifecycle for agents. It introduces a pattern involving a Compute Gateway and a Relay. The Gateway instructs Open Compute to spin up a sandbox in the customer's VPC. This sandbox then establishes an outbound connection (egress) to the Relay, minimizing concerns about ingress. (9:34)
-
Compute Drivers and Abstraction
The system uses 'Drivers'—a wrapper for a given compute type (e.g., EC2 driver, Kubernetes driver, or sandbox provider like Modal). This abstraction allows Open Compute to interact with various compute layers uniformly. (10:50)
-
State Management and Resilience
To handle complex compute lifecycles, the system supports snapshotting and forking. Snapshotting allows saving the exact state of a build or container, while forking enables a session to delegate a subtask to a new agent instance, crucial for multi-agent swarms. (12:24)
-
Data Security and Identity
The design emphasizes keeping all operations within the customer's VPC. It supports a Bring Your Own Key (BYOK) model for LLM calls, ensuring that even the AI inference remains within the customer's sovereign environment. (11:30)
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.