AI Engineer

Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo

Published 2026-11-09 · Duration 1:50:34

Summary

The presentation argues that for high-stakes domains like financial analysis, the primary bottleneck when using AI agents is not speed, but trust. To build reliable AI systems, speakers introduce five core design principles—Shared Domain, Consistency, Evidence, Accountability, and Verification. These principles guide the construction of a 'harness' that enforces structured, deterministic, and traceable processes, ensuring that the system's outputs are reliable and auditable, especially when dealing with money.

Download summary

Key takeaways

  1. Trust is the Bottleneck, Not Speed 7:02

    For financial applications, the challenge is not generating fast answers, but generating answers that are trustworthy. The focus must shift from maximizing tokens to building robust, reliable systems.

  2. The Five Principles of Trust 14:05

    Trust must be engineered into the system using five principles: Shared Domain (ensuring model and user speak the same language), Consistency (using code for calculations), Evidence (showing receipts/data lineage), Accountability (maintaining human oversight), and Verification (using external reference points).

  3. Code for Determinism 1:00:48

    Calculations and any process that must yield a consistent result should be implemented in code (e.g., Python scripts) rather than relying on the language model's internal arithmetic capabilities, which are non-deterministic.

  4. The Role of the Harness 20:13

    A 'harness' is a structured framework designed to guide the agent through a problem-solving process, forcing the user to address critical steps and assumptions, thereby building trust and preventing the system from becoming a 'black box.'

More digests on AI Agents

Technical details

  • Shared Domain 2040s

    The system must ensure the user's domain knowledge aligns with the model's understanding. The harness guides the user to challenge assumptions and clarify ambiguous terms, preventing the model from making incorrect inferences.

  • Consistency and Determinism 3648s

    To ensure reliable results, deterministic calculations must be implemented via code (e.g., Python scripts) and unit tested. Relying on the LLM's internal reasoning for arithmetic leads to non-deterministic and often incorrect outputs.

  • Evidence and Traceability 5550s

    The system must provide a clear audit trail, showing the source of every piece of information (the 'receipts'). This includes labeling inputs as facts (e.g., bank statements), assumptions (agent reasoning), or interpretations. Traceability must extend to code changes.

  • Accountability

    The human user remains ultimately accountable for the agent's output. The harness must incorporate intentional 'gates' (e.g., reviewing tasks instead of individual changes) to ensure the user reviews critical decisions and assumptions, preventing the system from becoming a black box.

  • Verification

    Verification requires a separate, external point of reference (a counterfactual) to check the agent's conclusions against. This prevents the system from being biased by its own internal data or assumptions.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.