fix: Add SONAR implementation and resolve missing models #2014
+272
−11
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add ModelMeta for SONAR it also ensure that an "Unknown" number of parameters is not filtered.
Should resolve:
#1979
#1976
This seems to have been caused by the restructering of calls on the leaderboard.
This however leads to the side-effect that selecting number of parameters can't remove "Unknown". A solution is to add a selection box.
@x-tabdeveloping I still can't get the SONAR model to show. I was hoping you had an idea for this?
Code Quality
make lint
to maintain consistent style.Documentation
Testing
make test-with-coverage
.make test
ormake test-with-coverage
to ensure no existing functionality is broken.Adding a model checklist
andmteb.get_model(model_name, revision)
mteb.get_model_meta(model_name, revision)
I have tested the implementation works on a representative set of tasks.