Could you share the pretraining config？

[chuanye-Wang]

Hi, thanks for releasing LingBot-VA.

I would like to understand the exact configuration used for the official LingBot-VA pretraining stage.

Could you share the pretraining config or the key hyperparameters used in the official pretraining run?

In particular, I am interested in:

1. Training setup

   • number of GPUs / GPU type
   • distributed strategy, e.g. FSDP/DDP
   • precision, e.g. bf16/fp16

2. Batch and optimization

   • per-GPU batch size
   • gradient accumulation steps
   • total number of optimizer steps
   • learning rate and scheduler
   • warmup steps
   • optimizer settings

3. Model/memory settings

   • whether activation checkpointing was used
   • whether all transformer blocks were checkpointed
   • attention backend or other speed/memory settings

4. Data and sequence settings

   • datasets used during pretraining
   • sampling ratio between datasets, if multiple datasets were used
   • chunk size / window size / sequence length settings
   • image/video latent resolution or number of frames

This would be very helpful for understanding the official pretraining recipe and expected training cost.

Thanks!