# Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal

## Executive summary

The talk outlines the advanced infrastructure requirements for modern AI agents, arguing that building such a platform requires specialized primitives rather than custom, 'vibe-coded' solutions. Key architectural needs include persistent, elastic workspaces, on-demand GPU/TPU resources, shared file systems for collaboration, and robust network controls to prevent data leakage. The evolution of agent infrastructure is moving from 'pets' (bespoke services) to 'microbes' (highly dynamic, self-evolving, and interconnected systems).

## Key takeaways

- Agent Workspaces and Persistence: Agents require a dedicated workspace (e.g., for a coding agent needing a GitHub repo, or an accounting agent needing work papers) that must persist state over potentially indefinite periods without requiring continuous compute, allowing the agent to resume work later.
- Elastic and Specialized Compute: Infrastructure must support highly elastic resource consumption (CPU/Memory) and provide specialized compute like GPUs and TPUs on demand. This prevents over-provisioning by only billing for the duration of actual usage.
- Security and Isolation Primitives: Due to the risk of data leakage, agents require granular network controls, such as restricting communication to specific IP addresses, domains, or even file paths (e.g., using sidecar proxies).
- Observability for Agents: It is critical to expose operational metrics (logs, GPU utilization, budget) directly to the agents themselves, allowing them to reason about failures and optimize their own processes.

## Technical details

- Modal Sandbox Architecture: A Modal sandbox is a secure environment for running untrusted code. Modern implementations can run multiple containers or micro VMs within a single sandbox, separating the agent loop (harness) from the tool execution environment for enhanced security and isolation.
- Agent Collaboration and Storage: Collaboration requires shared file systems (e.g., Modal volumes) that are fast, consistent, and allow multiple agents (or agents and humans) to work on the same data simultaneously.
- Harness/Brain Separation: The architecture is evolving toward separating the 'brain' (the agent loop/harness) from the 'hands' (the runtime/sandbox). This allows the harness to dynamically spin up and manage environments based on immediate needs.
- Scale and Reliability: Scaling to massive levels (e.g., millions of sandboxes, petabytes of data processed daily) requires aggregating capacity from multiple hyper-scalers and neo-clouds, as no single cloud provider can guarantee the necessary scale.

## Practical implications

- Developers building agent systems must adopt specialized platforms that manage state persistence and resource elasticity, rather than relying on basic container orchestration.
- Security boundaries must be enforced at the network layer (IP/domain/path level) to prevent data exfiltration by autonomous agents.
- The shift from 'pets' (bespoke services) to 'microbes' (dynamic, self-evolving systems) requires infrastructure that treats environments as disposable, reusable, and highly interconnected artifacts.

## Topics

AI Agents, Distributed Computing, Cloud Infrastructure, Microservices, Data Security, Modal, Ramp Inspect, Basis

Source: https://www.youtube.com/watch?v=_0SQh8a1Osk
