Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploy evaluation stack in our Hugging Face space #25

Open
deanwampler opened this issue Dec 12, 2024 · 0 comments
Open

Deploy evaluation stack in our Hugging Face space #25

deanwampler opened this issue Dec 12, 2024 · 0 comments
Labels
evaluators Implementations of evaluations, including benchmarks and datasets leaderboards Leaderboards deployed to HF or other places reference stack All tools for the reference stack.

Comments

@deanwampler
Copy link
Contributor

Configure a HF space with the evaluation stack (lm-eval-harness + unitxt). Most likely baseline is the HF demo here: https://huggingface.co/demo-leaderboard-backend
@deanwampler deanwampler converted this from a draft issue Dec 12, 2024
@deanwampler deanwampler changed the title Evaluation stack Deploy evaluation stack in our Hugging Face space Dec 12, 2024
@deanwampler deanwampler moved this to Todo in FA2: TSEI Tasks Dec 12, 2024
@deanwampler deanwampler added reference stack All tools for the reference stack. evaluators Implementations of evaluations, including benchmarks and datasets leaderboards Leaderboards deployed to HF or other places labels Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
evaluators Implementations of evaluations, including benchmarks and datasets leaderboards Leaderboards deployed to HF or other places reference stack All tools for the reference stack.
Projects
FA2: TSEI Tasks
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
1 participant
@deanwampler