Dnarayanan/latent moe#2594

Draft

pablo-garay wants to merge 6 commits intoNVIDIA:mainfrom

pablo-garay:dnarayanan/latent_moe

Commits on Nov 22, 2025

Changes to support latent MoEs

venmugil
authored and
deepakn94
committed
Modify throughput equations for hybrid models to account for --moe-latent-size
deepakn94
committed
Move fc2_latent_proj into combine method to make sure tensors are the right shape
deepakn94
committed

Commits on Dec 5, 2025

Move fc1_latent_proj into router_and_preprocess method to make sure the MoE routing happens in hidden dimension and correct tensor shape is captured in token dispatcher
venmugil
committed
Merge branch 'main' into dnarayanan/latent_moe
deepakn94
authored

Commits on Dec 9, 2025

API compat: ignore ParameterMovedBreakage for __init__ methods
pablo-garay
committed