NVIDIA Developer

How to Build Training-Ready Datasets with NVIDIA Cosmos Data Factory Skills

Published 2026-11-18 · Duration 6:13

Summary

NVIDIA Cosmos Data Factory enables the creation of training-ready datasets for Physical AI by automating the curation, augmentation, and generation of synthetic data. Using a coding agent, users can transform a single image into a complete, labeled dataset, even for rare events like warehouse falls, by leveraging blueprints and composable skills like Event Video Generation and Autolabeling, all orchestrated via Airflow on Kubernetes.

Download summary

Key takeaways

  1. Solving the Rare Event Problem

    Cosmos Data Factory allows the generation of rare events (e.g., falls, fires) starting from a single seed image, eliminating the need for extensive real-world footage capture.

  2. Agent-Driven Pipeline Construction

    A coding agent can read a repository, recommend the appropriate blueprint (e.g., Event Video Generation), and execute the entire data pipeline using minimal prompts.

  3. Modular Data Generation

    The system offers five reference blueprints (e.g., Autonomous vehicle data set generation, Defect image generation) and six composable skills (e.g., Curation and retrieval, Augmentation, Autolabeling) that can be combined into custom workflows.

More digests on Synthetic Data Generation

Technical details

  • Event Video Generation Workflow 314s

    The process begins with Cosmos 3 generating a seed image. The Event Video Generation DAG (running on Kubernetes/Airflow) then uses a Vision Language Model to convert the scene and event into a prompt, generating clips. Autolabeling adds detections, tracks, and captions, resulting in a labeled dataset folder per clip.

  • Core Skills and Functionality 0s

    The six composable skills include Curation and retrieval (finding clips), Augmentation (creating visual variations), Simulation (rendering labeled images from 3D scenes), Anomaly Gen (creating realistic defects), Autolabeling (annotating with detections/captions), and Orchestration (running the blueprint on the cluster).

  • Scalability and Performance

    On a larger machine, one server with eight RTX Pro 6000 GPUs can generate approximately 384 labeled clips per day at a cost of roughly $2.50 per clip.

Mentioned resources

Channel & topics

Watch on YouTube · Back to latest

This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.