AI Engineer

Your KV Cache Can Spot Attacks on Your AI Agents - Ari Azarafrooz, Intrinsec AI

Published 2026-11-09 · Duration 7:12

Summary

Ari Azarafrooz introduces a novel security approach for AI agents: treating the Key-Value (KV) cache, typically used for inference throughput optimization, as a foundational security signal. Instead of relying solely on guarding inputs and outputs at the edges (e.g., with judge models or classifiers), this method analyzes the computation record stored in the KV cache to detect anomalies, such as hidden backdoors or prompt injections, even in scenarios where no text is available for filtering (e.g., voice agents).

Download summary

Key takeaways

  1. KV Cache as a Security Signal

    The KV cache records every computation performed by the inference engine. By analyzing these internal pages, security can be implemented deeper than traditional edge guardrails, covering 'guardrails blind spots' and being useful for data privacy applications where inputs/outputs cannot be read.

  2. Three-Step Security Recipe 2:00

    The process involves three steps: 1) Learning a baseline for the AI workload's typical KV page structure; 2) Identifying KV pages that are rare against this baseline (anomalies); and 3) Classifying and explaining these detected anomalies.

  3. Integration Points in the Stack 2:30

    Integration can occur either directly inside the inference engine (e.g., via vLLM or LMCache) for real-time alerting, or by reading stored KV pages (e.g., from LMCache) to find deviations from the norm.

More digests on AI Security

Technical details

  • KV Cache Functionality 0s

    The KV cache stores keys and values computed per token, allowing the model to avoid redundant computations, thereby optimizing throughput in modern inference engines.

  • Use Case: Voice Agents 250s

    In a voice agent scenario (using Gemini 404B), the system successfully detected hidden instructions within audio clips by analyzing the KV pages, even when there was no text to filter.

  • Production Challenges 360s

    Two major hurdles in production are: 1) Model drift (models and agents constantly changing, requiring multiple baselines); and 2) Real-time processing of large objects (e.g., a 128-token chunk can be ~40MB of KV data), which requires efficient compression and streaming.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.