# Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread

## Executive summary

This talk addresses the critical bottleneck in building powerful AI agents: accessing and interpreting proprietary data. The speaker, Aamir Shakir of Mixedbread, argues that the standard 'Frankenstein stack' (OCR, chunking, embeddings, vector databases) is inherently lossy, failing to capture fine-grained, multimodal information from complex documents like SEC filings. Mixedbread proposes a solution using a 'late interaction model' that provides superior, contextualized retrieval, leading to significant improvements in answer accuracy (e.g., 36% to 98%) while maintaining high security standards (ISO compliant, BYOB).

## Key takeaways

- The Data Bottleneck: The primary limitation for AI agents is not intelligence, but the ability to access and process proprietary data. The traditional RAG pipeline is 'super lossy' because each step (OCR, chunking, embedding) introduces noise and loses context, especially from complex tables or diagrams.
- Limitations of the 'Frankenstein Stack': The combination of OCR, chunking, and pure embedding models is inefficient and unreliable. It forces agents to read every document, leading to high token costs and poor answer quality when scaling to large corpuses.
- Advanced Retrieval Solution: Mixedbread utilizes a 'late interaction model' that generates multiple embeddings for both documents and queries. This technique allows the model to contextualize fine-grained information (e.g., specific revenue figures from a specific year) and is multimodal, preserving visual layout information.
- Security and Scalability: The platform is designed for enterprise use, offering ISO compliance and a 'bring your own bucket' (BYOB) mode with zero data retention, making it suitable for banks and hedge funds with billions of documents.

## Technical details

- Traditional Retrieval Pipeline Failure: The standard process involves sequential steps (OCR $ ightarrow$ parsing $ ightarrow$ chunking $ ightarrow$ embedding $ ightarrow$ vector database). The speaker notes that each step is lossy, and the resulting single point embedding fails to represent the rich, complex information found in financial reports (e.g., distinguishing between Nvidia's 1996 and 2026 earnings).
- Embedding Model Limitations: Pure embedding models struggle with complex, structured data because the resulting vector points can look very similar across different time periods or contexts, preventing the system from accurately distinguishing fine-grained information.
- Late Interaction Model: This advanced technique overcomes embedding limitations by creating multiple embeddings for both documents and queries. This allows the model to contextualize information (e.g., identifying 'net income' within a specific segment and year) and is crucial for multimodal understanding.
- Agentic Search and Multihop Reasoning: For complex, qualitative queries (e.g., 'Is Intel's foundry profitably improving and what is driving it?'), the system employs a 'genetic search' model. This model performs multiple queries and multihop reasoning, decomposing the complex request and feeding the resulting data to an LLM for explanation.

## Practical implications

- Architects building AI agents must move beyond simple RAG implementations and consider multimodal, late-interaction retrieval methods to handle complex, structured data.
- The cost and efficiency of agents are directly tied to the quality of data access; poor retrieval leads to excessive token usage and high operational costs.
- For enterprise deployments, prioritizing security and compliance (ISO, BYOB) is critical, as demonstrated by the platform's ability to handle billions of documents with zero data retention.
- The use of multimodality (visual layout, OCR) is necessary to preserve information that is not purely textual, such as figures and tables.

## Topics

Retrieval-Augmented Generation (RAG), Multimodality, Vector Databases, AI Agents, Information Retrieval, Build Engineering, Mixedbread, Intel D270

Source: https://www.youtube.com/watch?v=3Ad73Onx3Aw