A toolkit for vision-language dataset formats — JSON-schema specs plus CLI/Python tools to validate and convert between them.
VLM dataset workflows have a contract-drift problem. Annotation pipelines emit data in one shape, training pipelines expect another, and the glue between them is ad-hoc adapter code that silently goes stale. Field renames, new optional values, schema-vs-data mismatches — these surface as training-time bugs rather than at the producer / consumer boundary where they belong.
What DAFT is:
- Schemas for vision-language dataset shapes — both annotation (what producers emit) and training (what consumers expect).
- A CLI + validator so anyone holding a dataset can check it against its schema before handing it off.
- Converters between annotation and training shapes — explicit, deterministic, with optional flags for media handling.
- A reference Python adapter that plugs one of the training shapes into cosmos-rl SFT.
New formats, validators, converters, and adapters are welcome; the same registration pattern that wires the built-ins works for your own extensions.
Value, by audience:
| For… | DAFT gives you… |
|---|---|
| Producers (annotation pipelines, human annotators) | Target one of these schemas and your output is consumable by any downstream tool that speaks the same schema. |
| Consumers (training pipelines, researchers) | Validate your input dataset before launching a training run. If it passes, your loader contract holds. |
# Install (direct from git)
pip install git+https://github.com/NVIDIA-TAO/tao-daft.git
# Install (from wheel)
pip install nvidia-tao-daft
# Verify
tao-daft --helpFor runnable examples, see examples/ and the
CLI reference.
| Area | What's there | Link |
|---|---|---|
| Formats | Format registry, per-format specs (metropolis-v3.0, cosmos-reason-v1.0, tao-vl-reason-v1.0), versioning policy | formats |
| CLI | tao-daft validate / convert reference |
cli |
| Validators | Validation engine | validators |
| Converters | Conversion pairs and pair-specific options | converters |
| Datasets | Training-loop adapters (cosmos-rl) | datasets |
| Examples | Working datasets per format | examples |
nvidia-tao-daft/
├── examples/datasets/ # Working datasets, one subdir per format
│
├── tests/ # Test suite (schemas, validators, converters, CLI, doc consistency)
│
└── src/nvidia_tao_daft/
├── cli/ # tao-daft entry point (validate, convert)
├── formats/ # Format specifications + JSON schemas
├── validators/ # Validation engine
├── converters/ # Format converters (pairs/)
└── datasets/ # Training-loop adapters
Python 3.10 – 3.13. Runtime dependencies: jsonschema, pydantic.
Dev dependencies: see pyproject.toml.
See CONTRIBUTING.md for the DCO sign-off requirement, and
CONTRIBUTORS.md for the people behind this project.