# Please Stop Telling Your AI to “Search Harder”

## Executive summary

The rise of autonomous AI agents is fundamentally changing infrastructure demands, moving beyond predictable machine loads. Agents, which can fire off massive, unpredictable bursts of API calls and compute-intensive requests, are straining traditional middleware, API gateways, and even Kubernetes-based microservices. The discussion highlights the need for specialized, high-density runtime environments, such as Agent Substrate, which uses actors and shared workers to manage agent lifecycles, enabling sub-second suspend-and-resume operations and context persistence for large-scale, unpredictable workloads.

## Key takeaways

- Agent traffic is unpredictable and high-density.: Unlike previous machine users (which were predictable, e.g., quarterly reports), agents are 'improvising actors' that can rapidly and unpredictably fire off numerous API calls, putting immense load on APIs and middleware (00:00, 03:47).
- Traditional API gateway pricing models are inadequate.: The current reliance on request-based pricing for API gateways is problematic because the unpredictable nature of agent traffic makes capacity planning (e.g., estimating calls over 3 years) nearly impossible (08:13).
- Agent workloads differ significantly from microservices.: While microservices involve multiple instances of the same service, agents often represent unique, context-dependent sessions (e.g., a user's unique coding session), leading to a magnitude of complexity beyond traditional microservice management (12:02, 16:12).
- Specialized platforms are needed for agent orchestration.: Tools like Agent Substrate are designed to manage agent workloads by mapping agents to smaller sets of ready workers, enabling heavy multiplexing and sub-second suspend-and-resume operations, building upon Kubernetes foundations (25:56).

## Technical details

- Agent Workloads vs. Microservices: Microservices typically involve multiple instances of a single service. Agent workloads, however, are highly individualized and context-dependent (e.g., a user's unique coding session), making them fundamentally different from standard service scaling models (12:02).
- Agent Substrate Architecture: Agent Substrate is an agent workload orchestration platform that uses Kubernetes underneath. It manages agents by mapping them to 'workers' (resources) and 'actors' (the application logic). This design allows for heavy multiplexing and sub-second suspend-and-resume operations, ensuring state persistence and quick context restoration (25:56).
- SaaS Multi-Tenancy Challenges: Traditional SaaS tenancy models (isolated vs. shared) face new challenges with agents. The need for fast context loading and state persistence across long periods of inactivity (hibernation cycles) is critical, requiring architectural solutions beyond standard API calls (16:12, 32:31).
- Compute and Scaling Limits: The sheer scale and unpredictability of agent activity challenge the assumption that Kubernetes is the 'be-all-to-end-all' solution for all types of rapidly spinning up and down, unique workloads (22:07).

## Practical implications

- Architects must move beyond traditional request-based pricing models for middleware and API gateways due to the unpredictable nature of agent traffic.
- When designing systems for agents, focus on state persistence, context management, and efficient resource multiplexing (actors/workers) rather than just raw compute power.
- Evaluate if the complexity added by specialized agent infrastructure (like Substrate) is justified by the use case, or if the functionality can be abstracted by hyperscalers (e.g., Google, Anthropic).
- For build engineers, understanding the difference between microservice scaling (multiple instances of the same service) and agent scaling (unique, stateful sessions) is crucial for selecting the right infrastructure.

## Topics

Agentic Workflows, Middleware Architecture, API Gateway Design, Cloud Computing, Microservices, State Management, High-Density Computing, Agent Substrate, Kubernetes, Google Cloud Run

Source: https://www.youtube.com/watch?v=O_qk3D3I3hI