Skip to content

Issues: mattpocock/evalite

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

30 Open 31 Closed
30 Open 31 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Builtin cooldown based on tokens
#124 opened Mar 14, 2025 by tomhonour
2
Return partial results if task errors
#123 opened Mar 14, 2025 by tomhonour
1
No License
#122 opened Mar 7, 2025 by Swifilaboroka
2
Would be nice to have easier way to work with files. or some alternative for large prompts and expected data
#117 opened Feb 23, 2025 by da1z
1
Track latency and allow passing to scorers and graphs
#111 opened Feb 7, 2025 by mattpocock
option to disable cli result output
#105 opened Jan 30, 2025 by ShiboSoftwareDev
modify eval metadata
#102 opened Jan 28, 2025 by ShiboSoftwareDev
Copy Code button on code blocks
#101 opened Jan 26, 2025 by seveibar
1
an option to launch the server and view the results page without running any tests
#100 opened Jan 25, 2025 by ShiboSoftwareDev
post scorers step
#99 opened Jan 25, 2025 by ShiboSoftwareDev
2
I shouldn't have to scroll to the top to hide the Trace menu
#98 opened Jan 25, 2025 by ShiboSoftwareDev
1
Syntax highlighting
#97 opened Jan 25, 2025 by ShiboSoftwareDev
1
option to collapse rows in the table
#96 opened Jan 25, 2025 by ShiboSoftwareDev
2
Issues running evalite alongside vitest workspaces
#95 opened Jan 16, 2025 by HalfdanJ
Allow for passing --score-threshold for failing CI if below threshold
#93 opened Jan 14, 2025 by mattpocock
Add Human Evaluation from inside Evalite
#92 opened Jan 2, 2025 by mattpocock
Handle images and files traced from AI SDK
#86 opened Dec 31, 2024 by mattpocock
3 tasks
CLI table is sometimes too wide
#84 opened Dec 31, 2024 by mattpocock
Only part of the code is in a code block
#75 opened Dec 19, 2024 by ShiboSoftwareDev
2
eval loading state should be true as soon as it's data method is called
#73 opened Dec 18, 2024 by ShiboSoftwareDev
1
In custom columns an option to keep/hide the original columns
#71 opened Dec 17, 2024 by ShiboSoftwareDev
4
Add only option to data entries
#61 opened Dec 12, 2024 by AriPerkkio
4
Make the index a summary page where you can view a historical view of all runs
#50 opened Dec 10, 2024 by mattpocock
1
Track how long scoring took
#42 opened Dec 9, 2024 by mattpocock Initial Release
Is there any way to see if the Vercel AI SDK is using the cache?
#40 opened Dec 9, 2024 by dschlabach
4
ProTip! Follow long discussions with comments:>50.