Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix eval max length #297

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Some-random
Copy link
Contributor

@Some-random Some-random commented Feb 12, 2025

Although in theory we should allocate a big buffer for reasoning models to play with before giving out final answers, in practice, our generation can never be that long (in fact, the max_completion_length parameter for model training is usually clipped at 1024). So it doesn't make sense to use 32768 as the max length for generation.

Changing 32768 to 4096 would also resolve some issues because 32768 is too long as the default length for some models

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Evaluation script fails to work for Qwen-Math models.
1 participant