Old Version of Dia Model

[anlcakmak19]

Hi everyone, I have a question I couldn’t quite figure out.

There are two Dia models on Hugging Face:

Older: https://huggingface.co/nari-labs/Dia-1.6B
New: https://huggingface.co/nari-labs/Dia-1.6B-0626

From what I understand, the first one is the older version. There are community fine-tuning scripts and FastAPI server implementations based on it. However, that model is slower and doesn’t support batch inference.
Fine tuning scripts: https://github.com/stlohrey/dia-finetuning
TTS server: https://github.com/devnen/Dia-TTS-Server/tree/main

The newer model seems to have a different config.json format, and I’m having trouble updating the old code to make it compatible — I keep getting errors. When I try to run inference with an older fine-tuned checkpoint using the new config files, it generates audio but then continues with silence until it reaches max_new_tokens.

How can I convert my old model to be compatible with the new version? Or how can I run inference on my fine-tuned model using the new Dia version?

[Alth3c0w]

I think the new model might be a different architecture entirely. I also have a lot of old fine-tuned models that I would like to use with the transformers implementation, to lower vram usage. But, whenever I use the old models with the new code it always errors out. I'm wondering if it's possible to convert the older models to the transformers architecture somehow, but I haven't found a way yet.

[anlcakmak19]

Yeah, same issue as you, @Alth3c0w . I fine-tuned the older model because there was a prepared pipeline for it. But I can’t run this model with the new Dia repo — it causes some errors. There are two options: either we write a new script to fine-tune the new model, or we adapt the older model to work with the new Dia repo. I’m a bit stuck. I wish the contributors would help.

[Sedherthe]

Doesn't it work if you back to the older code checkpoint and try inferencing?

[anlcakmak19]

@Sedherthe Yeah, I can run it using the older commit checkpoints, but the new codebase/architecture gives much better inference speed. That’s why I needed to use the old model with the new code. I’m investigating how I can further improve the inference speed.

[Sedherthe]

Ah okay, got it! @anlcakmak19
What are the latency and speeds that you are getting right now - with the newer model? And with the older code?

[anlcakmak19]

@Sedherthe I don’t really remember—I worked on this model about a month ago, and I couldn’t solve the inference speed issue. For my use case, I wasn’t able to meet the required first-chunk latency limits. When I have some free time, I’ll try to find my old benchmarks and share them here. My goal in migrating to the new codebase/architecture was to try adding Flash Attention and VLLM support, but at the moment I’m working on a different model.

[Alth3c0w]

From my own testing, on an RTX 3060 12GB, the speed and latency of the new code is about the same, within margin of error compared to the old implementation. VRAM usage, however, is much lower, around 4-5GB, versus the old inference code peaking at around 10 to almost 12GB in some cases. This is only testing using the base Dia model, however, since the finetunes don't currently work with the new code.