NVIDIA Developer

Accelerate Visual AI Agents on NVIDIA Jetson with VSS 3.3 and Cosmos

Published 2026-11-18 · Duration 55:19

Summary

This livestream demonstrates how to accelerate the development and deployment of visual AI agents on the NVIDIA Jetson edge platform using NVIDIA Blueprint for Video Search and Summarization (VSS) 3.3 and Cosmos. The workflow leverages coding agents and specialized skills (Jetson Device Skills, BSP Skills) to transform natural-language requests into configured, scalable AI applications, enabling complex tasks like multi-camera fusion search and automated assembly inspection.

Download summary

Key takeaways

  1. Agentic Development Workflow on Jetson 2:33

    A developer defines an application via a simple prompt to a coding agent (e.g., Open Claw, Codex). The agent then accesses specialized skills (Jetson, VSS, DeepStream) to build and deploy the multi-model pipeline, significantly expediting a process that might take weeks into a matter of hours.

  2. VSS as a Scalable Blueprint 6:28

    VSS acts as a vision agent blueprint that connects live camera feeds, recorded video, and user requests into a scalable agent capable of Observe, Reason, and Act. It provides both 'always-on' tools (real-time analysis, tracking) and 'on-demand' tools (search, summarization, report generation).

  3. Multi-Model Fusion and Efficiency 9:58

    The system achieves efficiency by using lighter-weight models (like DDMnet or D1) as fast, always-on screeners to identify candidate events, which are then passed to resource-intensive VLMs (like Cosmos) for deeper, contextual reasoning, thus reducing VLM processing load and freeing up GPU capacity.

  4. Model Agnosticism and Deployment 13:11

    VSS is designed to be service agnostic. Developers can connect their own VLMs or embedding models, and the system supports running components locally on the Jetson (e.g., Jetson Nano 8 GB) or pointing them to remote cloud endpoints.

More digests on Visual AI

Technical details

  • SOP/Assembly Procedure Checking 863s

    This workflow uses DDMnet (Dense Difference Maps) to examine visual feature changes over time, identifying activity boundaries. These segments are then processed by a fine-tuned NVIDIA Cosmos model, which identifies the specific assembly action (e.g., 'installing a fan'). The SOP checker compares the recognized sequence against a predefined checklist to flag missing steps.

  • Traffic Monitoring Pipeline 1198s

    The traffic monitoring demo uses a Decision Model (e.g., Liquid AI's D1 2B) as a fast, always-on screener. D1 scores the monitoring question ('Is there a queue?') every two seconds. Only if the 'Yes' score reaches a threshold (e.g., 85%) for three consecutive checks is a candidate event created for NVIDIA Cosmos to perform detailed verification.

  • Resource Constraints and Scaling 1341s

    The number of camera streams a Jetson can handle depends heavily on the resolution, frame rate, and codec. For example, 1080p at 60 FPS is estimated to handle around 24 streams. VSS itself does not use VRAM; the required VRAM for Cosmos depends on the model's quantization, ranging from 2 GB (smallest quantization) up to 30-70 GB (super models).

  • Security and Extensibility 1673s

    VSS blueprints operate within an 'Open Shell' secure sandbox, which dictates the policy of what the agents can access (APIs, CLIs, MCPs). The framework is designed to allow easy swapping of different VLMs via coding agents or provided documentation.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.