How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills
Summary
NVIDIA Cosmos Data Factory automates the creation of training-ready datasets for Physical AI, particularly for rare events (e.g., falls). The system uses a coding agent to orchestrate a pipeline, which can generate synthetic data from a single seed image, automatically annotate the results, and package the output into a structured dataset. This process significantly reduces the manual effort required to build large-scale, labeled datasets.
Key takeaways
-
Automated Data Pipeline Setup
The coding agent can set up the entire data pipeline using a single prompt, selecting the appropriate blueprint or composing a custom workflow from core skills.
-
Generating Rare Event Data
0:03
The process demonstrates generating rare events (like a worker fall) using Cosmos 3 from a single seed image, which is then processed by Event Video Generation.
-
Orchestration and Labeling
0:04
The workflow is managed by an Airflow DAG running on Kubernetes. This process automatically handles model deployment, video generation, and comprehensive auto-labeling (detections, tracks, captions, Q&A).
-
Scalability and Efficiency
0:05
The system is highly scalable; one server with eight RTX PRO 6000 GPUs can generate approximately 384 labeled clips per day.
Technical details
-
NVIDIA Cosmos Data Factory Overview
0s
A collection of five reference blueprints and six composable skills designed to curate, augment, generate, and label physical AI data at scale. It is accessed via a coding agent.
-
Reference Blueprints
0s
Five blueprints include: Autonomous Vehicle Dataset Generation (weather/lighting variations), Image Attribute Augmentation (clothing/accessories), Video Data Augmentation (transforming existing footage), Event Video Generation (creating rare events from a single image), and Defect Image Generation (creating defects on parts).
-
Core Skills
0s
Six core skills are: Curation and retrieval (finding clips/images), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), AnomalyGen (creating defect images), Auto labeling (annotating detections, tracks, captions), and Orchestration (running the blueprint on a cluster).
-
Workflow Execution
3s
The process involves using Cosmos 3 to generate a seed image, triggering the Event Video Generation DAG in Airflow (running on Kubernetes). The agent manages the entire sequence, from model execution to final labeled dataset packaging.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.