# How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills

## Executive summary

NVIDIA Cosmos Data Factory automates the creation of training-ready datasets for Physical AI, particularly for rare events (e.g., falls). The system uses a coding agent to orchestrate a pipeline, which can generate synthetic data from a single seed image, automatically annotate the results, and package the output into a structured dataset. This process significantly reduces the manual effort required to build large-scale, labeled datasets.

## Key takeaways

- Automated Data Pipeline Setup: The coding agent can set up the entire data pipeline using a single prompt, selecting the appropriate blueprint or composing a custom workflow from core skills.
- Generating Rare Event Data: The process demonstrates generating rare events (like a worker fall) using Cosmos 3 from a single seed image, which is then processed by Event Video Generation.
- Orchestration and Labeling: The workflow is managed by an Airflow DAG running on Kubernetes. This process automatically handles model deployment, video generation, and comprehensive auto-labeling (detections, tracks, captions, Q&A).
- Scalability and Efficiency: The system is highly scalable; one server with eight RTX PRO 6000 GPUs can generate approximately 384 labeled clips per day.

## Technical details

- NVIDIA Cosmos Data Factory Overview: A collection of five reference blueprints and six composable skills designed to curate, augment, generate, and label physical AI data at scale. It is accessed via a coding agent.
- Reference Blueprints: Five blueprints include: Autonomous Vehicle Dataset Generation (weather/lighting variations), Image Attribute Augmentation (clothing/accessories), Video Data Augmentation (transforming existing footage), Event Video Generation (creating rare events from a single image), and Defect Image Generation (creating defects on parts).
- Core Skills: Six core skills are: Curation and retrieval (finding clips/images), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), AnomalyGen (creating defect images), Auto labeling (annotating detections, tracks, captions), and Orchestration (running the blueprint on a cluster).
- Workflow Execution: The process involves using Cosmos 3 to generate a seed image, triggering the Event Video Generation DAG in Airflow (running on Kubernetes). The agent manages the entire sequence, from model execution to final labeled dataset packaging.

## Practical implications

- Build engineers can use this framework to rapidly prototype and scale data pipelines for specialized AI use cases (e.g., industrial defect detection, rare event capture).
- The modular nature (blueprints and core skills) allows for building custom pipelines without relying solely on reference implementations.
- The integration with Kubernetes and Airflow provides a robust, scalable, and auditable orchestration layer for data generation.

## Topics

Physical AI, Synthetic Data Generation, Data Labeling, Data Pipeline Orchestration, Computer Vision, Cosmos Data Factory GitHub, Technical Blog

Source: https://www.youtube.com/watch?v=MT9w658LxVk
