# Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp

## Executive summary

Alexander Shevchenko details why applied research remains critical for application companies, even with advanced frontier LLMs. He presents three production-ready projects: developing a specialized, faster model (Qwen 3.5) for financial QA that outperforms Opus 4.6 on latency; implementing KV cache cartridges to compress context for complex tasks like month-end close; and utilizing agentic swarms of 10,000 parallel agents for comprehensive security pen-testing. The overarching theme is achieving significant gains in time and cost efficiency through deep model specialization and advanced orchestration.

## Key takeaways

- Specialist Models Beat Frontier Models on Latency: By post-training a Qwen 3.5 model (35B parameters) for financial question answering, the team achieved accuracy above Opus 4.6 while significantly improving latency, making the model viable for high-volume production use cases. (5:16)
- KV Cache Cartridges for Context Compression: This technique compresses context up to 20 to 30 times more efficiently into the KV cache, allowing agents to be pre-loaded with necessary context (e.g., for month-end close) before the user starts work. (5:58)
- Agentic Swarms for Security and Efficiency: Using 10,000 parallel agents (via Ramp Inspect) for pen-testing found seven high-severity vulnerabilities. Furthermore, open-weight models can be used in swarms with detection rates comparable to frontier models, at a fraction of the cost. (7:50)

## Technical details

- Model Specialization and Post-Training: The team trained a retrieval/financial QA model by partnering with Prime Intellect to post-train Qwen 3.5. This approach allows the creation of highly specialized, faster models that complement, rather than replace, powerful frontier orchestrators. (3:06)
- Context Management (KV Cache Cartridges): Cartridges are a technique to compress large amounts of context into the KV cache offline, preparing the agent for complex, multi-step tasks like financial reconciliation. This is described as a lower-lift post-training method compared to full fine-tuning. (5:58)
- Distributed Agent Orchestration: The use of 10,000 parallel agents to perform 'agentic map reduce' for vulnerability scanning. The process involves agents finding vulnerabilities, deduplicating tickets using LLMs, and adversarially reproducing findings. (7:50)
- Optimization and Routing: Ramp Router and Token Spend Management tools help optimize LLM usage by providing deep visibility into token spend (by category/taxonomy) and improving agent latency/cost per token through strategies like Nvidia Switchyard and smart flex offloading. (10:36)

## Practical implications

- Implement specialized, smaller models (post-trained) for core business functions to reduce latency and cost compared to relying solely on general-purpose frontier models.
- Develop context pre-loading mechanisms (like KV cache cartridges) for multi-step, time-sensitive workflows (e.g., month-end close) to improve agent efficiency.
- Use agentic swarms and automated testing to scale security and quality assurance efforts, allowing for automated, comprehensive vulnerability discovery.
- Integrate token spend management and advanced routing strategies (e.g., Nvidia Switchyard) to maintain unit economic efficiency as LLM usage scales.

## Topics

LLM Agents, Applied AI, Model Optimization, Context Compression, Security Testing, Token Economics, Distributed Computing, Ramp Labs Blog, Ramp Labs Research (X/Twitter)

Source: https://www.youtube.com/watch?v=T-42NrYjwxM
