# Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial Services

## Executive summary

This discussion details the architectural and strategic decisions behind building specialized AI products, specifically ChatGPT for Financial Services. The core message is that while foundational models (like ChatGPT) provide significant power, successful enterprise adoption requires building a 'secret sauce' (the missing 20%) on top of the model using plugin primitives. Key challenges include integrating proprietary data, ensuring human oversight (Human-in-the-Loop), and training models on subjective human judgment and 'taste,' which is often more difficult than solving factual errors.

## Key takeaways

- Enterprise AI requires proprietary data bundling.: While standard Market Connectors (MCPs) exist, the speaker noted that testing revealed high error rates (some exceeding 40%). Therefore, the product decision was made to bring proprietary data in-house to control the experience end-to-end, arguing that the added value of bundled data outweighs the cost of existing licenses.
- The '20%' gap requires plugin architecture.: The speaker asserted that foundational models (like Codex/ChatGPT) can get a product 80% of the way there, but the remaining 20%—the 'secret sauce'—is crucial for market success. This 20% is built using the model's plugin architecture and primitives, which allows for custom data, UI changes, or unique integrations.
- Safety mandates a Human-in-the-Loop (HITL) approach.: For financial products, safety is paramount. The current product is designed as a productivity tool for knowledge workers, operating under the deep assumption that a human is always in the loop, responsible for auditing and reviewing the model's work. This mimics existing hierarchical review chains (junior analyst $\rightarrow$ associate $\rightarrow$ VP).
- Training models on 'Taste' is key to advanced capability.: The most challenging area for AI improvement is not factual error (hallucination), but subjective human judgment and 'taste' (e.g., slide aesthetics, clean system design). This is trained using Reinforcement Learning from Human Feedback (RLHF), specifically by having humans rate pairs of outputs (pairwise signal) to determine which is superior.

## Technical details

- Model Training and Data Strategy: Financial reasoning data is integrated upstream into the main model (e.g., GPT-6/GPT-6.1), ensuring the intelligence is accessible for external developers and startups, not just first-party products. The process involves post-training and fine-tuning on specialized financial datasets.
- Plugin Architecture and Extensibility: The product build relies heavily on the ChatGPT plugin architecture and primitives. This approach allows developers to build specialized functionality by leveraging horizontal capabilities and adding proprietary 'secret sauce' (data, UI, or integration) on top of the core model.
- Evaluation (Evals) for Finance: Building a good evaluation benchmark is extremely difficult because any eval is merely a proxy for the skill being measured. Furthermore, evals are prone to being 'reward hacked' if the grading mechanism is flawed. The team is working on building a specialized 'investment banking eval' that predicts the ability to automate the work of a junior analyst.

## Practical implications

- When building enterprise AI, focus less on the foundational model's raw power and more on the proprietary data layer and workflow orchestration (the 'secret sauce').
- Design products around existing professional review chains (e.g., junior analyst $\rightarrow$ associate $\rightarrow$ VP) to ensure safety and trust, making the model behave like a co-worker.
- For model improvement, prioritize training on human preference (taste) using pairwise comparison signals, as this is often the bottleneck for advanced capabilities.
- When developing benchmarks, assume that the evaluation metrics are imperfect proxies and must be constantly validated against real-world professional workflows.

## Topics

Generative AI, Financial Technology (FinTech), Large Language Models (LLMs), Plugin Architecture, Machine Learning Operations (MLOps), Reinforcement Learning from Human Feedback (RLHF), ChatGPT for Financial Services, Joseph’s AI Engineer keynote

Source: https://www.youtube.com/watch?v=v-d1WTqIULs