Exp 18b moe training#28

Open

vukrosic wants to merge 7 commits intomainfrom

exp_18b_moe_training

Commits on Oct 9, 2025

Enhance README with detailed instructions on experiment setup and usage
vukrosic
committed
rename move
vukrosic
committed
show tokens
vukrosic
committed
Enhance MoE training script with GPU configuration support and checkpoint loading functionality. Added command line arguments for GPU selection and resuming from checkpoints, allowing for flexible …
vukrosic
committed
Update MoE model configuration for reduced parameters and added test mode functionality. Adjusted model architecture to fit within B200 memory constraints, modified batch size and sequence length, …
vukrosic
committed
Update MoE model configuration and enhance test mode functionality. Adjusted model parameters for improved performance, including increased hidden dimensions and expert count. Modified test mode se…
vukrosic
committed
update
vukrosic
committed