Questions about the training resources

[ventr1c]

Hi,

Thanks for your excellent work. I noticed that the original paper indicates that 32 H100 GPUs are used for training. Could I ask whether a node with 8 H200 is available for the reproduction? Which factors cause these resource costs? Thanks for your clarification!

[wangyu-ustc]

Hi,

Thanks for the interest! I apologize that I don't have the experience on H200 yet, I would think 8 H200 should do the work, I guess it is worth giving it a try? My experience was that I was able to run the experiments with 16 H100 GPUs, it is just a bit slower than 32 GPUs.