# Why AI Agents Need RAG + Observability

## Executive summary

AI agents require two distinct types of context to be effective in production environments: documented system knowledge and live operational context. Retrieval Augmented Generation (RAG) provides the former by grounding answers in runbooks and architectural diagrams. Observability provides the latter by supplying real-time signals (metrics, logs, traces, topology). Combining these two sources allows the AI agent to move beyond generic diagnoses, connecting documented procedures to current, verifiable evidence to guide human operators toward specific next steps.

## Key takeaways

- The Gap in AI Agents: AI agents, even with access to all documentation, struggle to answer the critical question: 'What is the system doing right now?'
- Two Contexts for AI Agents: An operation-focused AI agent needs both System Knowledge (provided by RAG) and Live Operational Context (provided by Observability).
- The Power of Combination: When RAG (documented 'how it should work') is combined with Observability (live 'what is happening'), the agent can connect known procedures to current evidence, providing a specific, evidence-based starting point for investigation.

## Technical details

- Retrieval Augmented Generation (RAG): RAG retrieves relevant information (documentation, runbooks, architectural knowledge) to ground the model's response, validating answers against internal sources rather than relying solely on general model training. This helps answer questions like 'How is the application supposed to work?'
- Observability Signals: Observability provides signals to answer 'What is happening now?' These signals include: Metrics (response time, throughput shifts), Logs (evidence of what occurred), Traces (request path and time spent across services), and Topology (service connections).
- Hybrid Transaction Analysis: In a complex, distributed application (e.g., mobile banking inquiry), the agent uses RAG to define the intended path and uses Observability to pinpoint where the transaction is slowing down (e.g., identifying increased elapsed time between middleware and mainframe, supported by growing queue metrics and logs).

## Practical implications

- Reduces manual searching by consolidating information from multiple tools and documents into a single, structured, evidence-based response.
- Allows the agent to separate known facts from possibilities, while clearly stating that evidence does not prove causation.
- Structures the response around three key areas for the operator: 1) Evidence (what signals changed), 2) Impact (which connected services depend on the area), and 3) Next Steps (validated actions based on runbooks and current evidence).

## Topics

AI Agents, Retrieval Augmented Generation (RAG), Observability, DevOps, System Monitoring, IBM Technology

Source: https://www.youtube.com/watch?v=hvnPWgfmvyI
