NVIDIA Developer

How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills

Published 2026-11-17 · Duration 6:13

Summary

NVIDIA Cosmos Data Factory automates the creation of training-ready datasets for Physical AI, particularly for rare events (e.g., falls). The system uses a coding agent to orchestrate a pipeline, which can generate synthetic data from a single seed image, automatically annotate the results, and package the output into a structured dataset. This process significantly reduces the manual effort required to build large-scale, labeled datasets.

Download summary

Key takeaways

  1. Automated Data Pipeline Setup

    The coding agent can set up the entire data pipeline using a single prompt, selecting the appropriate blueprint or composing a custom workflow from core skills.

  2. Generating Rare Event Data 0:03

    The process demonstrates generating rare events (like a worker fall) using Cosmos 3 from a single seed image, which is then processed by Event Video Generation.

  3. Orchestration and Labeling 0:04

    The workflow is managed by an Airflow DAG running on Kubernetes. This process automatically handles model deployment, video generation, and comprehensive auto-labeling (detections, tracks, captions, Q&A).

  4. Scalability and Efficiency 0:05

    The system is highly scalable; one server with eight RTX PRO 6000 GPUs can generate approximately 384 labeled clips per day.

More digests on Physical AI

Technical details

  • NVIDIA Cosmos Data Factory Overview 0s

    A collection of five reference blueprints and six composable skills designed to curate, augment, generate, and label physical AI data at scale. It is accessed via a coding agent.

  • Reference Blueprints 0s

    Five blueprints include: Autonomous Vehicle Dataset Generation (weather/lighting variations), Image Attribute Augmentation (clothing/accessories), Video Data Augmentation (transforming existing footage), Event Video Generation (creating rare events from a single image), and Defect Image Generation (creating defects on parts).

  • Core Skills 0s

    Six core skills are: Curation and retrieval (finding clips/images), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), AnomalyGen (creating defect images), Auto labeling (annotating detections, tracks, captions), and Orchestration (running the blueprint on a cluster).

  • Workflow Execution 3s

    The process involves using Cosmos 3 to generate a seed image, triggering the Event Video Generation DAG in Airflow (running on Kubernetes). The agent manages the entire sequence, from model execution to final labeled dataset packaging.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.