# How to Fine-Tune Models Beyond Cosmos With NVIDIA Cosmos Framework

## Executive summary

This tutorial demonstrates the use of the NVIDIA Cosmos Framework to fine-tune models beyond the Cosmos family, specifically Qwen3-VL, for robot video question answering (RoboVQA). The process is fully automated by a coding agent, which executes a defined skill workflow—including setup, data download, preflight checks, baseline evaluation, training (1 epoch), and post-training scoring—on an NVIDIA GB300. The fine-tuned model showed improved performance on a held-out clip.

## Key takeaways

- Non-Cosmos Model Fine-Tuning: The Cosmos Framework supports training models outside the Cosmos family, demonstrated here with Qwen3-VL using the RoboVQA dataset.
- Agent-Driven Workflow Execution: A coding agent can execute the entire fine-tuning process (setup, download, pre-flight, training, scoring) from a single prompt by utilizing the defined skill.
- Training and Evaluation Results: The post-training accuracy was reported as 62.8% (within a 62-63% range), showing improvement over the base model when tested on a held-out robot video clip.

## Technical details

- Model and Hardware Specifications: The fine-tuning utilized Qwen3-VL and was executed on an NVIDIA GB300, with the demonstrated training run taking approximately two hours.
- Cosmos Skill Workflow: The 'Cosmos 3 Qwen3-VL RoboVQA SFT' skill manages a sequence of steps: setup, download, pre-flight, baseline scoring, training, and post-training scoring.
- Fine-Tuning Methodology: The process involves Supervised Fine-Tuning (SFT) using the RoboVQA dataset to improve the model's ability to answer questions about robot videos.

## Practical implications

- The framework enables highly automated MLOps pipelines where a single prompt can trigger complex, multi-stage model training and evaluation workflows.
- It expands the utility of the Cosmos Framework beyond its native models, allowing for specialized fine-tuning of external models (e.g., Qwen3-VL) for domain-specific tasks like physical AI/robotics.

## Topics

AI, AgenticAI, PhysicalAI, ModelFineTuning, RoboVQA, GitHub Repository (COOKBOOK.md), Technical Blog

Source: https://www.youtube.com/watch?v=79lv9HPXbEY
