How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills
Summary
NVIDIA Cosmos Data Factory enables the creation of training-ready datasets for Physical AI by automating the curation, augmentation, and generation of synthetic data. Using a coding agent, users can transform a single image into a complete, labeled dataset, even for rare events like warehouse falls, by leveraging blueprints and composable skills like Event Video Generation and Autolabeling, all orchestrated via Airflow on Kubernetes.
Key takeaways
-
Solving the Rare Event Problem
Cosmos Data Factory allows the generation of rare events (e.g., falls, fires) starting from a single seed image, eliminating the need for extensive real-world footage capture.
-
Agent-Driven Pipeline Construction
A coding agent can read a repository, recommend the appropriate blueprint (e.g., Event Video Generation), and execute the entire data pipeline using minimal prompts.
-
Modular Data Generation
The system offers five reference blueprints (e.g., Autonomous vehicle data set generation, Defect image generation) and six composable skills (e.g., Curation and retrieval, Augmentation, Autolabeling) that can be combined into custom workflows.
More digests on Synthetic Data Generation
Technical details
-
Event Video Generation Workflow
314s
The process begins with Cosmos 3 generating a seed image. The Event Video Generation DAG (running on Kubernetes/Airflow) then uses a Vision Language Model to convert the scene and event into a prompt, generating clips. Autolabeling adds detections, tracks, and captions, resulting in a labeled dataset folder per clip.
-
Core Skills and Functionality
0s
The six composable skills include Curation and retrieval (finding clips), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), Anomaly Gen (creating realistic defects), Autolabeling (annotating with detections/captions), and Orchestration (running the blueprint on the cluster).
-
Scalability and Performance
On a larger machine, one server with eight RTX Pro 6000 GPUs can generate approximately 384 labeled clips per day at a cost of roughly $2.50 per clip.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.