Thanks for this great work that help a lot to ramp up training LLAMA easily.
Seems it hasn't been updated for a long time. Is it because this contribution is already absorbed by Megatron-LM, so that LLAMA could get equivalently trained by Megatron-LM directly?