README.md

orphan	true

Megatron Discussions

This directory contains in-depth guides, tutorials, and discussions about optimizing and using Megatron for various use cases.

Available Guides

Training Guides

Previous News

• [2025/05] Megatron Core v0.11.0 brings new capabilities for multi-data center LLM training (blog).
• [2024/07] Megatron Core v0.7 improves scalability and training resiliency and adds support for multimodal training (blog).
• [2024/06] Megatron Core added support for Mamba-based models. Review the paper An Empirical Study of Mamba-based Language Models and code example.
• [2024/01 Announcement] NVIDIA has released the core capabilities in Megatron-LM into Megatron Core in this repository. Megatron Core expands upon Megatron-LM's GPU-optimized techniques with more cutting-edge innovations on system-level optimizations, featuring composable and modular APIs.

Contributing

To contribute a guide or tutorial, follow this structure:

1. Create a new directory: docs/discussions/your-guide-name/
2. Add your main guide: docs/discussions/your-guide-name/your-guide-name.md
3. Create an images directory: docs/discussions/your-guide-name/images/
4. Update this README.md with a link to your guide

Each guide should be self-contained with its own images and supporting files.