Inside Hermes Agent Traces with NeMo Relay | Nemotron Labs
Summary
The session details how agent traces, using Hermes Agent and NeMo Relay, transform opaque tool-use workflows into structured, inspectable telemetry. This is critical for enterprise adoption of agent fleets, enabling AI teams to perform robust AI evaluations, optimize models, and maintain compliance. NeMo Relay acts as an open-source observability layer that captures model calls, tool usage, and runtime state while supporting advanced governance features like PII redaction and data fencing for personal versus shared company IP.
Key takeaways
-
Agent Traces vs. System Logs
1:30
Unlike traditional system logs used for root cause analysis, agent telemetry captures the full lifecycle of an agent's turn, including every model call and tool execution. This data serves as valuable AI evaluation and training data for improving models and operational processes. (0:01:30)
-
Operational Model Shift in Enterprise Agents
2:15
Moving from a personal agent (where the user is the operator) to a managed agent fleet introduces a split between the user and the operator. In large enterprises, the operational model further splits, requiring a dedicated 'AI team' responsible for tuning skills and models, necessitating advanced telemetry for measurement and troubleshooting. (0:02:15)
-
NeMo Relay's Multi-Layer Observability
3:10
NeMo Relay provides four layers of agent observability: basic split/messaging health checks (L1), agent analytics (measuring token calls, LLM calls, turns without storing content), full trajectories (L4), and full audit/forensic storage (L3). This allows companies to tailor their data retention policies based on compliance needs. (0:03:10)
-
Governance and Data Fencing
3:45
The system supports data fencing: personal agents can be configured to strip out all content and identifiable information, retaining only analytics and diagnostics, while shared agents (company IP) can store full content for auditing and knowledge capture. (0:03:45)
More digests on Agentic Workflows
Technical details
-
NeMo Relay Functionality
255s
NeMo Relay is an open-source NVIDIA library integrated into the agent's execution path. It performs three core functions: **Observe** (records every step, model call, and tool), **Govern** (enforces policies, such as preventing specific commands like `rm-rf`, and automatically redacting PII using regex), and **Optimize** (enables caching and model routing). (0:04:15)
-
Agent Fleet Architecture
235s
The 'Team Gateway' acts as a central router, directing incoming messages to the appropriate personal or shared agent within the fleet. This architecture is designed to manage complex, multi-user agent interactions. (0:03:55)
-
Hermes Agent Capabilities
115s
Hermes is highlighted for its self-learning capability; it improves and creates its own skills over time as it is used, making it a unique differentiator compared to platforms relying solely on plugins. (0:01:55)
Mentioned resources
- Hermes Agent
- NeMo Relay
- LangSmith
- Phoenix
- OpenTelemetry
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.