feat(benchmark): add support for GSM8K benchmark, solves #143 #144
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
I have added GSM8K benchmark.
What are you adding?
Changes Made
src/openbench/evals/gsm8k.pysrc/openbench/datasets/gsm8k.pysrc/openbench/config.pysrc/openbench/_registry.pyTesting
pytest)pre-commit run --all-files)Checklist
Related Issues
Closes #143
Additional Context