Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal
Summary
The talk outlines the advanced infrastructure requirements for modern AI agents, arguing that building such a platform requires specialized primitives rather than custom, 'vibe-coded' solutions. Key architectural needs include persistent, elastic workspaces, on-demand GPU/TPU resources, shared file systems for collaboration, and robust network controls to prevent data leakage. The evolution of agent infrastructure is moving from 'pets' (bespoke services) to 'microbes' (highly dynamic, self-evolving, and interconnected systems).
Key takeaways
-
Agent Workspaces and Persistence
7:03
Agents require a dedicated workspace (e.g., for a coding agent needing a GitHub repo, or an accounting agent needing work papers) that must persist state over potentially indefinite periods without requiring continuous compute, allowing the agent to resume work later.
-
Elastic and Specialized Compute
10:21
Infrastructure must support highly elastic resource consumption (CPU/Memory) and provide specialized compute like GPUs and TPUs on demand. This prevents over-provisioning by only billing for the duration of actual usage.
-
Security and Isolation Primitives
15:57
Due to the risk of data leakage, agents require granular network controls, such as restricting communication to specific IP addresses, domains, or even file paths (e.g., using sidecar proxies).
-
Observability for Agents
It is critical to expose operational metrics (logs, GPU utilization, budget) directly to the agents themselves, allowing them to reason about failures and optimize their own processes.
Technical details
-
Modal Sandbox Architecture
230s
A Modal sandbox is a secure environment for running untrusted code. Modern implementations can run multiple containers or micro VMs within a single sandbox, separating the agent loop (harness) from the tool execution environment for enhanced security and isolation.
-
Agent Collaboration and Storage
902s
Collaboration requires shared file systems (e.g., Modal volumes) that are fast, consistent, and allow multiple agents (or agents and humans) to work on the same data simultaneously.
-
Harness/Brain Separation
The architecture is evolving toward separating the 'brain' (the agent loop/harness) from the 'hands' (the runtime/sandbox). This allows the harness to dynamically spin up and manage environments based on immediate needs.
-
Scale and Reliability
Scaling to massive levels (e.g., millions of sandboxes, petabytes of data processed daily) requires aggregating capacity from multiple hyper-scalers and neo-clouds, as no single cloud provider can guarantee the necessary scale.
Mentioned resources
- Modal
- Ramp Inspect
- Basis
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.