# How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills

## Executive summary

NVIDIA Cosmos Data Factory enables the creation of training-ready datasets for Physical AI by automating the curation, augmentation, and generation of synthetic data. Using a coding agent, users can transform a single image into a complete, labeled dataset, even for rare events like warehouse falls, by leveraging blueprints and composable skills like Event Video Generation and Autolabeling, all orchestrated via Airflow on Kubernetes.

## Key takeaways

- Solving the Rare Event Problem: Cosmos Data Factory allows the generation of rare events (e.g., falls, fires) starting from a single seed image, eliminating the need for extensive real-world footage capture.
- Agent-Driven Pipeline Construction: A coding agent can read a repository, recommend the appropriate blueprint (e.g., Event Video Generation), and execute the entire data pipeline using minimal prompts.
- Modular Data Generation: The system offers five reference blueprints (e.g., Autonomous vehicle data set generation, Defect image generation) and six composable skills (e.g., Curation and retrieval, Augmentation, Autolabeling) that can be combined into custom workflows.

## Technical details

- Event Video Generation Workflow: The process begins with Cosmos 3 generating a seed image. The Event Video Generation DAG (running on Kubernetes/Airflow) then uses a Vision Language Model to convert the scene and event into a prompt, generating clips. Autolabeling adds detections, tracks, and captions, resulting in a labeled dataset folder per clip.
- Core Skills and Functionality: The six composable skills include Curation and retrieval (finding clips), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), Anomaly Gen (creating realistic defects), Autolabeling (annotating with detections/captions), and Orchestration (running the blueprint on the cluster).
- Scalability and Performance: On a larger machine, one server with eight RTX Pro 6000 GPUs can generate approximately 384 labeled clips per day at a cost of roughly $2.50 per clip.

## Practical implications

- Build engineers can drastically reduce the time and cost associated with data acquisition by generating synthetic, high-quality training data for specialized AI models.
- The modular nature of the skills allows for the creation of highly customized data pipelines that go beyond the predefined reference blueprints.
- The integration of tools like Airflow and Kubernetes provides a robust, scalable framework for managing complex data generation workflows.

## Topics

Synthetic Data Generation, Physical AI, Data Labeling Automation, Data Curation, AI Training Pipelines, NVIDIA Cosmos Data Factory GitHub, Technical Blog

Source: https://www.youtube.com/watch?v=D47jNWtzPsI
