-
Notifications
You must be signed in to change notification settings - Fork 10
Closed
Labels
benchmark/evalRelated to benchmark/evalsRelated to benchmark/evalsdevRelated to general dev tasksRelated to general dev tasksenhancementNew feature or requestNew feature or request
Description
- Support Gemini models: Refactor the
parse_with_geminifunction such that the API call goes throughcreate_response - Create tests for the functionality. Can create a separate file (e.g.
tests/test_schema_parser.py) - Verify and fix errors (likely with using
max_tokens) when using o-series models from OpenAI - Check if there are any benchmarks for a similar functionality. If not, start building a dataset for evaluation.
- Add support for
dataclassobjects, potentially with additional attributes forsample_values(examples of possible values) andalternate_keys(examples of alternative names for each attribute to support robustness across unstandardized documents).
We can create separate issues for any of the above if needed.
cc: @Vaishnav2804
Metadata
Metadata
Assignees
Labels
benchmark/evalRelated to benchmark/evalsRelated to benchmark/evalsdevRelated to general dev tasksRelated to general dev tasksenhancementNew feature or requestNew feature or request