AI Engineer

Your Financial Agent Needs to Show Its Work - Anirudh Veeraragavan, Fieldguide

Published 2026-11-07 · Duration 13:28

Summary

In financial auditing, the defensibility of an AI agent's decision-making process is paramount. The speaker argues that merely achieving a correct answer is insufficient; the system must provide a complete, auditable trail of how the decision was reached. This requires implementing four core controls—Authorization, Execution Control, Evidence Provenance, and Evaluation Integrity—to ensure that the agent's output can be fully reconstructed and verified by a human auditor.

Download summary

Key takeaways

  1. The Need for Defensibility in AI Agents

    In finance, the opinion (the product) is only as good as the basis behind it. The focus must shift from whether the agent is correct or wrong to demonstrating the integrity of the decision-making process. This pattern of 'decision assurance' applies beyond audit to domains like credit and Anti-Money-Laundering (AML).

  2. Authorization Control 7:07

    Authorization must be defined not just by tool call permission, but by the object's state at the time of the call. An agent must not overwrite work that a human has already reviewed and signed off on, even if the underlying evidence changes.

  3. Execution Control 8:57

    To ensure reproducibility, the system must maintain a detailed record of every step (inputs, outputs) taken during the agent's run. This allows auditors to compare runs across different points in time and identify the exact step where a discrepancy occurred.

  4. Evidence Provenance 12:13

    A citation must be an 'identity,' meaning it must include a specific ID, version, and a hash of the document's contents, rather than just a document name. This prevents ambiguity and allows for quality constraints on the source of evidence.

  5. Evaluation Integrity

    The gold data or evaluation set used to train or test the agent must itself be audited. A high overall score (e.g., 97%) can mask significant underlying failures or methodological changes in the evaluation data.

More digests on AI Agents

Technical details

  • Financial Audit Use Case 100s

    The example used is a bank reconciliation, where the agent compares a client's book cash balance to the bank's cash balance, reconciling differences using items like deposits in transit and outstanding checks.

  • Decision Assurance Pattern

    The agent must be programmed to recognize its limitations and stop, escalating the decision to a human when certain exceptional cases arise (e.g., when a new run disagrees with a previous run, or when evidence fails to conform to established rules).

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.