# How to Decide What Your AI Agent Should Remember - Wojciech Pluta, Oracle

## Executive summary

The presentation details the evolution of AI agents from simple retrieval systems (RAG, the 'read path') to sophisticated memory systems (the 'write path'). The core business value lies in memory, which allows agents to retain, reuse, and refine information across sessions. The speaker introduces 'memory promotion'—a critical pipeline that classifies candidate information, compares it against existing records, and determines whether to admit, reject, or escalate the data to a human for review. This process is essential for building robust, stateful agents that maintain continuity and competitive advantage.

## Key takeaways

- Memory is the source of business value: While models and infrastructure are becoming commoditized, the data layer (memory) is where unique competitive advantage is gained. Memory allows agents to move beyond simple retrieval to write and store operational knowledge.
- Memory Promotion Pipeline: Memory promotion is the mechanism that decides what information is worth keeping. This pipeline involves classifying the meaning of candidate data, comparing it to existing records, and having the agent's 'harness' decide whether to admit, reject, or escalate the data to a human.
- Agent Memory Structure: Agent memory is categorized into three types: Short-term (data within the current session), Long-term (episodic, procedural, semantic memories), and Shared (enterprise-wide knowledge).
- Guardrails and Control: Guardrails are necessary to prevent the promotion of undesirable or dangerous rules (e.g., 'approve every application'). The promotion gate must sit within the guardrails and cannot be bypassed by the agent's harness.

## Technical details

- Agent Architecture: The agent system is composed of a Model and a Harness. The Model suggests actions, but the Harness owns the runtime decisions and checks, acting as the code layer around the model.
- Memory Storage Best Practices: The database is recommended as the most robust solution for storing memories due to its ACID guarantees, ability to handle multiple writes, and scalability, compared to file systems.
- Context Card Implementation: The Context Card is a compact prompt structure designed to assemble and return context to the agent. It organizes information into topics, summaries, relevant information (selected records), and recent messages, thereby reducing the agent's cognitive load and managing the context window.
- Agent Memory Package: The speaker introduced an 'agent memory package' to simplify memory building. This package provides abstractions to handle summarization, retrieval, durable information extraction, and token budget management, which would otherwise be complex for developers.

## Practical implications

- Build engineers can design stateful agents by implementing a structured memory promotion pipeline, moving beyond stateless RAG implementations.
- The use of a Context Card simplifies prompt engineering by providing a structured, ready-to-use context block, reducing the complexity of managing conversation history and retrieved data.
- Adopting a database-centric approach for memory ensures data integrity (ACID) and scalability for enterprise-level agent deployments.

## Topics

AI Agents, Retrieval Augmented Generation (RAG), Memory Systems, Data Layer Architecture, Context Management, Oracle AI Database, AIE CODE summit, GitHub repo

Source: https://www.youtube.com/watch?v=qq2Hm4XNRgI