# How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut

## Executive summary

Revolut, a digital bank, achieved a dramatic 60-70% improvement in lead time and coding time for 2,000 engineers by self-hosting open-source LLMs (specifically GLM-53) on dedicated GPU infrastructure. This approach reduced the blended cost of tokens to one-tenth of what was spent with frontier labs (Anthropic/OpenAI), while maintaining high performance and ensuring compliance within an EU-regulated banking environment.

## Key takeaways

- Impact of AI on Engineering Velocity: The primary bottleneck for engineering is not coding itself, but rather the time spent navigating internal processes (e.g., Jira, Confluence, email). AI significantly improves lead time (time from feature pick-up to merge) more than pure coding time.
- Self-Hosting Economics: By self-hosting open-source models, Revolut achieved a blended token cost of 30 cents per million, representing a tenfold cost reduction compared to using major API providers.
- Performance Gains through Tuning: By optimizing parameters (e.g., quantization, tensor parallelism), the team achieved an 88% performance improvement for a 76% increase in demand, using the same GPU count (56 B200).

## Technical details

- Hardware Requirements: To serve 2,000 engineers with GLM-53, approximately 56 B200 GPUs are required. The target latency metric is sub-3 second time to first token.
- Model Optimization Parameters: Key tuning parameters include: using 256 (instead of 1 million) for context length, prioritizing FP4 or VFP4 quantization (FP4 to FP8 offers 40% GPU footprint saving), and optimizing tensor parallelism (e.g., TP8) to ensure sufficient KV cache.
- Compliance and Safety (EU Banking): For compliance, the system must operate with a 'human in the loop' model. This is enforced by creating a sandbox environment that allows setting granular rules (e.g., allowing `grab` but blocking `Gradle`) to prevent autonomous agent use and protect against rogue use.

## Practical implications

- Self-hosting open models is a viable, cost-effective alternative to paying high API costs from frontier labs.
- Focusing on lead time (process bottlenecks) rather than just coding time can yield greater engineering efficiency gains.
- Implementing strict sandboxing and 'human in the loop' controls is critical for maintaining regulatory compliance in highly regulated industries (e.g., banking).

## Topics

LLM Self-Hosting, Performance Engineering, Generative AI, Cloud Architecture, Fintech Compliance, David Abitbol, Revolut

Source: https://www.youtube.com/watch?v=Ok6QdZHCcI8