Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial Services
Summary
This discussion details the architectural and strategic decisions behind building specialized AI products, specifically ChatGPT for Financial Services. The core message is that while foundational models (like ChatGPT) provide significant power, successful enterprise adoption requires building a 'secret sauce' (the missing 20%) on top of the model using plugin primitives. Key challenges include integrating proprietary data, ensuring human oversight (Human-in-the-Loop), and training models on subjective human judgment and 'taste,' which is often more difficult than solving factual errors.
Key takeaways
-
Enterprise AI requires proprietary data bundling.
8:49
While standard Market Connectors (MCPs) exist, the speaker noted that testing revealed high error rates (some exceeding 40%). Therefore, the product decision was made to bring proprietary data in-house to control the experience end-to-end, arguing that the added value of bundled data outweighs the cost of existing licenses.
-
The '20%' gap requires plugin architecture.
17:40
The speaker asserted that foundational models (like Codex/ChatGPT) can get a product 80% of the way there, but the remaining 20%—the 'secret sauce'—is crucial for market success. This 20% is built using the model's plugin architecture and primitives, which allows for custom data, UI changes, or unique integrations.
-
Safety mandates a Human-in-the-Loop (HITL) approach.
22:16
For financial products, safety is paramount. The current product is designed as a productivity tool for knowledge workers, operating under the deep assumption that a human is always in the loop, responsible for auditing and reviewing the model's work. This mimics existing hierarchical review chains (junior analyst $\rightarrow$ associate $\rightarrow$ VP).
-
Training models on 'Taste' is key to advanced capability.
27:35
The most challenging area for AI improvement is not factual error (hallucination), but subjective human judgment and 'taste' (e.g., slide aesthetics, clean system design). This is trained using Reinforcement Learning from Human Feedback (RLHF), specifically by having humans rate pairs of outputs (pairwise signal) to determine which is superior.
Technical details
-
Model Training and Data Strategy
260s
Financial reasoning data is integrated upstream into the main model (e.g., GPT-6/GPT-6.1), ensuring the intelligence is accessible for external developers and startups, not just first-party products. The process involves post-training and fine-tuning on specialized financial datasets.
-
Plugin Architecture and Extensibility
1060s
The product build relies heavily on the ChatGPT plugin architecture and primitives. This approach allows developers to build specialized functionality by leveraging horizontal capabilities and adding proprietary 'secret sauce' (data, UI, or integration) on top of the core model.
-
Evaluation (Evals) for Finance
Building a good evaluation benchmark is extremely difficult because any eval is merely a proxy for the skill being measured. Furthermore, evals are prone to being 'reward hacked' if the grading mechanism is flawed. The team is working on building a specialized 'investment banking eval' that predicts the ability to automate the work of a junior analyst.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.