Skip to content

fix: fix mcore sequence packing#1937

Closed
yuki-97 wants to merge 1 commit intomainfrom
yukih/fix-mcore-seq-packing
Closed

fix: fix mcore sequence packing#1937
yuki-97 wants to merge 1 commit intomainfrom
yukih/fix-mcore-seq-packing

Conversation

@yuki-97
Copy link
Contributor

@yuki-97 yuki-97 commented Feb 12, 2026

As title.

Fix nightly test sft-llama3.1-8b-1n8g-megatron-seqpack.

if mbs is None:
    mbs = self.cfg["train_micro_batch_size"]

# before fix
# (
#     data_iterator,
#     num_microbatches,
#     micro_batch_size,
#     seq_length,
#     padded_seq_length,
# ) = get_microbatch_iterator(...)

# after fix
(
    data_iterator,
    num_microbatches,
    mbs,
    seq_length,
    padded_seq_length,
) = get_microbatch_iterator(...)

losses_reduced = forward_backward_func(
    ...
    micro_batch_size=mbs,
    ...
)

Signed-off-by: Yuki Huang <yukih@nvidia.com>
@yuki-97 yuki-97 requested review from a team as code owners February 12, 2026 16:47
@yuki-97 yuki-97 requested a review from yfw February 12, 2026 16:47
@yuki-97 yuki-97 added the CI:L1 Run doctests, unit tests, and functional tests label Feb 12, 2026
Copy link
Contributor

@yfw yfw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing!

@yuki-97
Copy link
Contributor Author

yuki-97 commented Feb 13, 2026

Closed since #1654 will fix this as well.

@yuki-97 yuki-97 closed this Feb 13, 2026
auto-merge was automatically disabled February 13, 2026 11:14

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI:L1 Run doctests, unit tests, and functional tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants