Unitxt 1.21.0
What's Changed
- add 'show more' button for imports from unitxt modules by @dafnapension in #1651
- Update head qa dataset by @elronbandel in #1658
- Update few slow datasets by @elronbandel in #1663
- MLCommons AILuminate card and related artifacts by @bnayahu in #1662
- Granite guardian: add raw prompt to the result by @martinscooper in #1671
- Add positional bias summary to the response by @martinscooper in #1640
- Return float instead float32 in granite guardian metric by @martinscooper in #1669
- add qa template exact output by @OfirArviv in #1674
- LLM Judge: add prompts to the result by default by @martinscooper in #1670
- Safety eval updates by @bnayahu in #1668
- Add inference engine caching by @eladven in #1645
- BugFix: Handle cases where all sample scores are the same (yields nan) by @elronbandel in #1660
- CrossInferenceProvider: add more models by @martinscooper in #1676
- Implement get_engine_id were missing by @martinscooper in #1679
- Revisit base dependencies (specifically remove ipadic and absl-py) by @elronbandel in #1681
- Fix LoadHF.load_dataset() when mem-caching is off by @yhwang in #1683
- HFPipelineInferenceEngine - add loaded tokenizer to pipeline by @eladven in #1677
- Add default cache folder to .gitgnore by @martinscooper in #1687
- Fix a bug in loading without trust remote code by @elronbandel in #1684
- Add sacrebleu[ja] to test dependencies by @elronbandel in #1685
- Let evaluator name to be a string by @martinscooper in #1665
- Fix: AzureOpenAIInferenceEngine fails if api_version is not set by @martinscooper in #1680
- Fix some bugs in inference engine tests by @elronbandel in #1682
- Improved output message when using inference cache by @yoavkatz in #1686
- Changed API of Key Value Extraction task to use Dict and not List[Tuple] (NON BACKWARD COMPATIBLE CHANGE) by @yoavkatz in #1675
- Support for asynchronous requests for watsonx.ai chat by @pawelknes in #1666
- add tags information - url by @BenjSz in #1691
- Fixes to GraniteGuardian metric,, safety evals cleanups by @bnayahu in #1690
- Add docstring to LLMJudge classes by @martinscooper in #1652
- Remove src.lock by @elronbandel in #1692
- Text2sql metrics update and optional caching by @oktie in #1672
- Llm judge use cross provider by @martinscooper in #1673
- Improve LLM as Judge consistency by @martinscooper in #1688
- Update version to 1.21.0 by @elronbandel in #1693
New Contributors
Full Changelog: 1.20.0...1.21.0
Contributors
eladven, yhwang, and 9 other contributors