Better Search Took This AI Agent From 36% to 98% Accuracy - Aamir Shakir, Mixedbread
Summary
This talk addresses the critical bottleneck in building powerful AI agents: accessing and interpreting proprietary data. The speaker, Aamir Shakir of Mixedbread, argues that the standard 'Frankenstein stack' (OCR, chunking, embeddings, vector databases) is inherently lossy, failing to capture fine-grained, multimodal information from complex documents like SEC filings. Mixedbread proposes a solution using a 'late interaction model' that provides superior, contextualized retrieval, leading to significant improvements in answer accuracy (e.g., 36% to 98%) while maintaining high security standards (ISO compliant, BYOB).
Key takeaways
-
The Data Bottleneck
5:30
The primary limitation for AI agents is not intelligence, but the ability to access and process proprietary data. The traditional RAG pipeline is 'super lossy' because each step (OCR, chunking, embedding) introduces noise and loses context, especially from complex tables or diagrams.
-
Limitations of the 'Frankenstein Stack'
6:40
The combination of OCR, chunking, and pure embedding models is inefficient and unreliable. It forces agents to read every document, leading to high token costs and poor answer quality when scaling to large corpuses.
-
Advanced Retrieval Solution
Mixedbread utilizes a 'late interaction model' that generates multiple embeddings for both documents and queries. This technique allows the model to contextualize fine-grained information (e.g., specific revenue figures from a specific year) and is multimodal, preserving visual layout information.
-
Security and Scalability
The platform is designed for enterprise use, offering ISO compliance and a 'bring your own bucket' (BYOB) mode with zero data retention, making it suitable for banks and hedge funds with billions of documents.
Technical details
-
Traditional Retrieval Pipeline Failure
330s
The standard process involves sequential steps (OCR $ ightarrow$ parsing $ ightarrow$ chunking $ ightarrow$ embedding $ ightarrow$ vector database). The speaker notes that each step is lossy, and the resulting single point embedding fails to represent the rich, complex information found in financial reports (e.g., distinguishing between Nvidia's 1996 and 2026 earnings).
-
Embedding Model Limitations
Pure embedding models struggle with complex, structured data because the resulting vector points can look very similar across different time periods or contexts, preventing the system from accurately distinguishing fine-grained information.
-
Late Interaction Model
This advanced technique overcomes embedding limitations by creating multiple embeddings for both documents and queries. This allows the model to contextualize information (e.g., identifying 'net income' within a specific segment and year) and is crucial for multimodal understanding.
-
Agentic Search and Multihop Reasoning
For complex, qualitative queries (e.g., 'Is Intel's foundry profitably improving and what is driving it?'), the system employs a 'genetic search' model. This model performs multiple queries and multihop reasoning, decomposing the complex request and feeding the resulting data to an LLM for explanation.
Mentioned resources
- Mixedbread
- Intel D270
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.