How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut
Summary
Revolut, a digital bank, achieved a dramatic 60-70% improvement in lead time and coding time for 2,000 engineers by self-hosting open-source LLMs (specifically GLM-53) on dedicated GPU infrastructure. This approach reduced the blended cost of tokens to one-tenth of what was spent with frontier labs (Anthropic/OpenAI), while maintaining high performance and ensuring compliance within an EU-regulated banking environment.
Key takeaways
-
Impact of AI on Engineering Velocity
3:20
The primary bottleneck for engineering is not coding itself, but rather the time spent navigating internal processes (e.g., Jira, Confluence, email). AI significantly improves lead time (time from feature pick-up to merge) more than pure coding time.
-
Self-Hosting Economics
9:40
By self-hosting open-source models, Revolut achieved a blended token cost of 30 cents per million, representing a tenfold cost reduction compared to using major API providers.
-
Performance Gains through Tuning
11:00
By optimizing parameters (e.g., quantization, tensor parallelism), the team achieved an 88% performance improvement for a 76% increase in demand, using the same GPU count (56 B200).
Technical details
-
Hardware Requirements
480s
To serve 2,000 engineers with GLM-53, approximately 56 B200 GPUs are required. The target latency metric is sub-3 second time to first token.
-
Model Optimization Parameters
600s
Key tuning parameters include: using 256 (instead of 1 million) for context length, prioritizing FP4 or VFP4 quantization (FP4 to FP8 offers 40% GPU footprint saving), and optimizing tensor parallelism (e.g., TP8) to ensure sufficient KV cache.
-
Compliance and Safety (EU Banking)
For compliance, the system must operate with a 'human in the loop' model. This is enforced by creating a sandbox environment that allows setting granular rules (e.g., allowing `grab` but blocking `Gradle`) to prevent autonomous agent use and protect against rogue use.
Mentioned resources
- David Abitbol
- Revolut
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.