Don't Approve the AI Model. Approve the App - Doneyli De Jesus, Langfuse
Summary
This talk details a framework for deploying AI agents in highly regulated environments, such as top-five banks. The core principle established is that institutions certify the *application*, not the underlying model. The methodology involves establishing strict deployment gates—requiring on-premises, gated, and air-gapped setups—and proving the application's readiness using a 'golden data set.' Evaluation is multi-dimensional, scoring traces across metrics like groundness, numerical accuracy, and regulatory compliance, allowing teams to improve performance (e.g., raising numerical accuracy from 60% to 95%) by integrating deterministic tools and human expert feedback, thereby reducing evaluation time from weeks to minutes.
Key takeaways
-
Certify Applications, Not Models
0:03
In regulated industries, the focus shifts from general model benchmarks (like Opus 5 leading benchmarks) to proving that the specific application built for internal use cases is reliable and safe for production deployment. (3:01)
-
Mandatory Deployment Constraints
0:03
Financial institutions require solutions to be deployed on-premises, gated (preventing data leakage/training on external providers), and air-gapped to maintain data control and compliance. (3:18)
-
The Evaluation Framework
0:06
The process requires building a 'golden data set' (test cases) and operationalizing testing to generate traces. Scores are calculated across multiple dimensions (e.g., accuracy, regulatory compliance) and must pass domain-expert-defined thresholds to clear the production gate. (6:00, 13:48)
-
Improving Performance with Tools
0:07
A model can be highly grounded (e.g., 97% grounded) but fail on specific tasks like math. By providing deterministic tools (e.g., a Python code tool), the application's overall accuracy can be significantly improved (e.g., from 60% to 95%). (7:20, 11:32)
Technical details
-
Deployment Architecture
3s
Solutions must adhere to three main requirements: staying on-premises (repurposing data centers for GPU infrastructure), being gated (preventing data exposure to model providers), and requiring air-gapped deployment. (3:18)
-
Evaluation Methodology
11s
The evaluation framework uses a 'golden data set' (key-value pairs with inputs and expected outputs) to generate traces. Scoring is multi-pronged, combining: 1) Deterministic scores (code-based, non-LLM), 2) Path evaluation (checking the agent's logical route), and 3) LLM judge scores (for qualitative aspects). (11:32)
-
Gate Definition and Iteration
15s
Gate thresholds are set by domain experts (e.g., risk personnel) and must be defined per use case, as a single average score is insufficient. Failures (failure modes) are collected in an 'annotation queue' for human review and feedback, which then feeds back into the golden data set for iterative improvement. (13:48, 15:53)
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.