AI Engineer

Don't Vibe Code Your Agent Infrastructure - Akshat Bubna, Modal

Published 2026-11-07 · Duration 17:35

Summary

The talk outlines the advanced infrastructure requirements for modern AI agents, arguing that building such a platform requires specialized primitives rather than custom, 'vibe-coded' solutions. Key architectural needs include persistent, elastic workspaces, on-demand GPU/TPU resources, shared file systems for collaboration, and robust network controls to prevent data leakage. The evolution of agent infrastructure is moving from 'pets' (bespoke services) to 'microbes' (highly dynamic, self-evolving, and interconnected systems).

Download summary

Key takeaways

  1. Agent Workspaces and Persistence 7:03

    Agents require a dedicated workspace (e.g., for a coding agent needing a GitHub repo, or an accounting agent needing work papers) that must persist state over potentially indefinite periods without requiring continuous compute, allowing the agent to resume work later.

  2. Elastic and Specialized Compute 10:21

    Infrastructure must support highly elastic resource consumption (CPU/Memory) and provide specialized compute like GPUs and TPUs on demand. This prevents over-provisioning by only billing for the duration of actual usage.

  3. Security and Isolation Primitives 15:57

    Due to the risk of data leakage, agents require granular network controls, such as restricting communication to specific IP addresses, domains, or even file paths (e.g., using sidecar proxies).

  4. Observability for Agents

    It is critical to expose operational metrics (logs, GPU utilization, budget) directly to the agents themselves, allowing them to reason about failures and optimize their own processes.

More digests on AI Agents

Technical details

  • Modal Sandbox Architecture 230s

    A Modal sandbox is a secure environment for running untrusted code. Modern implementations can run multiple containers or micro VMs within a single sandbox, separating the agent loop (harness) from the tool execution environment for enhanced security and isolation.

  • Agent Collaboration and Storage 902s

    Collaboration requires shared file systems (e.g., Modal volumes) that are fast, consistent, and allow multiple agents (or agents and humans) to work on the same data simultaneously.

  • Harness/Brain Separation

    The architecture is evolving toward separating the 'brain' (the agent loop/harness) from the 'hands' (the runtime/sandbox). This allows the harness to dynamically spin up and manage environments based on immediate needs.

  • Scale and Reliability

    Scaling to massive levels (e.g., millions of sandboxes, petabytes of data processed daily) requires aggregating capacity from multiple hyper-scalers and neo-clouds, as no single cloud provider can guarantee the necessary scale.

Mentioned resources

  • Modal (Platform/Primitive Set)
  • Ramp Inspect (Internal Coding Agent)
  • Basis (Accounting Agent)

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.