Build Training-Ready Datasets in Two Prompts Featuring Encord and Voxel51 | Cosmos Labs
Summary
This presentation details how NVIDIA Cosmos Data Factory enables the creation of high-quality, training-ready datasets for physical AI applications using a streamlined, two-prompt workflow. The system integrates data curation, augmentation, generation (via EVG), and validation (via Cosmos Evaluator) to address the challenge of capturing rare, critical edge cases in fields like autonomous driving and robotic inspection.
Key takeaways
-
The Two-Prompt Workflow
The process is simplified into two main steps: the first prompt sets up the environment and downloads models, and the second prompt runs the chosen data flow (e.g., generating specific longtail events).
-
Data Quality is Paramount
The performance of physical AI models is directly tied to the quality of the data. The system uses agentic validation to check for attributes (e.g., 'is there a fire?'), semantic plausibility, and structural integrity, preventing the use of 'poisoned' data.
-
Handling Rare Events
Cosmos Data Factory addresses the scarcity of real-world critical scenarios (e.g., nighttime pedestrian crossing, specific manufacturing defects) by using tools like Cosmos Data Set Search and EVG to synthetically augment and generate variations.
Technical details
-
Cosmos Data Factory Architecture
0s
The platform is modular, built on NVIDIA tools and libraries, and consists of three layers: Tools (e.g., Cosmos Curator), Core Skills (e.g., labeling, simulation, anomaly generation), and Blueprints (end-to-end workflows, such as Autonomous Vehicle Dataset Generation).
-
Data Generation and Validation
0s
The Event Video Generation (EVG) skill takes a seed image and a descriptive prompt to create videos (e.g., 'boxes falling,' 'fire breaking out'). The Cosmos Evaluator then checks these outputs for physical plausibility and hallucinations, allowing for automatic retries.
-
Scalability and Deployment
0s
The workflows can be run at the exabyte scale, utilizing tools like MILV for data querying and can be configured to run on various compute platforms, including remote endpoints or Kubernetes/Airflow setups.
Mentioned resources
- NVIDIA Cosmos Data Factory
- Cosmos 3
- Cosmos Curator
- Cosmos Data Set Search
- Cosmos Evaluator
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.