# Multi-Agent Systems: When More Agents Help or Hurt | Stanford CS329Z Lecture 8 with Michael Ryan

## Executive summary

This lecture provides a comprehensive overview of Multi-Agent Systems (MAS), detailing when and how they should be used compared to single-agent architectures. MAS are defined as collections of multiple agents solving a shared task, offering benefits like parallel execution and specialization. However, they introduce significant complexity, including coordination overhead, information loss, and failure modes (e.g., conformity, incompatible goals). Successful implementation requires careful selection of the orchestration pattern (Independent, Centralized, Decentralized, or Hybrid) and robust coordination infrastructure (Task Ledgers, Shared Memory, or Work Tree Isolation).

## Key takeaways

- MAS vs. Single Agent Systems: Single-agent systems are preferred when tasks are highly dependent and sequential. Multi-agent systems are best when subtasks can be parallelized and information can be divided into focused contexts. The recommendation is to start with a strong single-agent baseline and measure the value of adding complexity.
- MAS Cost and Performance Trade-offs: MAS typically cost more than single-agent systems due to messaging and context reloading. While complex hybrid systems can achieve high performance, this represents a significant trade-off between cost and capability.
- Coordination Failure Modes: MAS can fail due to system design issues, inter-agent misalignment (e.g., expectation mismatch, commitment failure), or task verification errors. Behavioral issues include 'conformity' (agents behaving similarly) and 'incompatible goals' (agents having conflicting objectives).

## Technical details

- Orchestration Patterns: Four primary patterns govern MAS: **Independent** (agents work in parallel with no communication, requiring final aggregation); **Centralized** (an orchestrator divides tasks and maintains control flow); **Decentralized Swarm** (agents pull work from a shared state/task ledger and communicate via an all-to-all message board); and **Hybrid** (combines global oversight with local, direct message passing between workers).
- Coordination Infrastructure: Mechanisms for managing interdependent work include: **Task Ledger** (an explicit, to-do list for tracking tasks and progress, supporting recovery); **Shared Memory** (a global memory system that uses multiple knowledge graphs—Insight, Query, and Interaction—to maintain retrievable context); and **Work Tree Isolation** (using separate workspaces, similar to Git branches, to prevent interference and manage merge conflicts).
- Communication Protocols: Communication can occur via **Handoff** (transferring full conversation history and control to a new specialist agent) or **Agent as Tool** (where an orchestrator calls a colleague agent as a specialized tool). For large swarms, communication can be managed by **Message Boards** (allowing all agents to write to and subscribe from a shared message pool) or formalized via the **Agent-to-Agent Protocol** (a structured protocol defining agent capabilities via 'agent cards' for traceable, push-notification communication).

## Practical implications

- When designing a multi-agent system, explicitly define the communication channel (e.g., message board vs. handoff) and the coordination mechanism (e.g., task ledger) to manage complexity.
- Be aware of the trade-off: increased parallelism and specialization come at the cost of higher coordination overhead and increased computational expense.
- For robust systems, implement work tree isolation (like Git branches) to minimize interference and manage conflicts when multiple agents modify the same code base.

## Topics

Multi-Agent Systems, AI Architecture, Software Engineering, Distributed Computing, AI Safety, Agent-to-Agent Protocol, AutoGen, MetaGPT

Source: https://www.youtube.com/watch?v=TlX_ft_PQIM
