# Insights | Introduction to LangSmith (9/12)

## Executive summary

This lesson introduces LangSmith Insights, a feature used to expand the search for unknown issues in agent traffic. Unlike score-based evaluators, Insights groups conversations into emergent categories, allowing teams to discover new areas of customer engagement, such as unsupported requests or ordering patterns, which can then be converted into datasets for further analysis.

## Key takeaways

- Insights vs. Evaluators: Evaluators answer questions that can be quantified (e.g., 'Is this negative sentiment?'), while Insights asks about unknown categories (e.g., 'What are people talking about when they're upset?'). Both pull signal from production traffic.
- Insights Workflow: The Insights report process involves defining the scope (e.g., 'how are customers engaging with the agent?') and the agent's function (e.g., 'customer ordering agent for the LangSlice pizzeria').
- Category Lifecycle: Any identified category or subcategory can be sent directly to a dataset or an annotation queue, enabling a continuous improvement loop: Insights -> Category -> Dataset -> Evaluator -> Production.

## Technical details

- Insights Report Structure: The resulting report includes an executive summary, patterns (with percentage coverage), and clickable references to the underlying traces.
- Feature Requirements: Insights requires a minimum of a LangSmith Plus plan to run.
- Auto Flow Process: The auto flow requires input on what the team wants to learn, what the agent does, and optionally, how the traces are structured.

## Practical implications

- Identify previously unknown failure modes or customer pain points that standard metrics might miss.
- Systematically categorize customer interactions (e.g., 'delivery minimum complaints') to prioritize development efforts.
- Automate the transition from raw trace data to structured datasets ready for targeted evaluation.

## Topics

LangSmith, Agent Engineering, AI Observability, Trace Analysis, CI/CD, LangSmith Essentials course

Source: https://www.youtube.com/watch?v=HVXydoGAWsQ
