How to try my own fine-tuning experiments

[minghehe-nobug]

Hello!
I'm new to multimodal training.
Inspired by this exciting project, I hope to try my own fine-tuning experiments on interleaved data.

Currently, I have downloaded the pro-trained model (3B) and completed the inference process. But can anyone help me how to write the parameters for the "torchrun" script?

The challenge for me is how to change the two parameters "--laion_shards" and "--mmc4_shards" to my own.
And how to modify the original code without using "LAION-2B" (the data set is too large)?

Thanks!

[Roberyan]

I want to fine-tune the model, too. Can I ask how much GPU resource is needed. I could have 4x A5000 GPUs, are they enough to complete it?