This folder selectively collects the design docs of latest MoE features from NVIDA/Megatron-LM.
| Date | Feature | Commit Link | Design Doc |
|---|---|---|---|
| Aug 27, 2025 | Support Expert Parallel A2A Overlapping - (03) Support EP A2A overlap for interleaved PP and MTP | 4b30ec5 | MR-3074 |
| Aug 19, 2025 | Support recomputation for FP8 layernorm/moe_act/shared_experts | 781e765 | MR-3465 |
| Aug 17, 2025 | Add MoE router fusion | c08d89b | MR-3809 |
| Aug 15, 2025 | Fixes and updates for external cudagraph | 2b6b46b | MR-3631 |
| Aug 11, 2025 | Support CP and recompute for MTP | 08abeed | MR-3330 |
| Aug 01, 2025 | Support Expert Parallel A2A Overlapping - (02) Support EP A2A overlap at PP=1 | ae1c882 | MR-3470 |
| June 16, 2025 | Support Expert Parallel A2A Overlapping - (01) Add TransformerLayer Submodule Callables | 8333bd5 | MR-3217 |
| June 13, 2025 | Flexible Asymmetric Virtual Pipeline Parallelism with Custom Pipeline Layout | 77732c3 | MR-2795 |
| Mar 23, 2025 | Multi-Token Prediction Support | 09ebca7 | MR-2628 |