[Feature: New model] API Generation #6
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…lows for the embeddings model to be from a different endpoint
Codecov ReportAttention:
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #6 +/- ##
===========================================
- Coverage 93.74% 83.68% -10.07%
===========================================
Files 35 38 +3
Lines 1103 1330 +227
===========================================
+ Hits 1034 1113 +79
- Misses 69 217 +148
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
ehooo
reviewed
Jan 16, 2024
chucheria
added a commit
that referenced
this pull request
Jan 31, 2024
* Add new models: OpenAI GPT3.5-Turbo and Azure OpenAI. Azure OpenAI allows for the embeddings model to be from a different endpoint * Parser for the API generation and correction response * add models, prompts and add to tests * Changes: - Change in base.py from prompts, formatting the prompt - Change in test_prompts adding a new symbol to delete * add data for examples --------- Co-authored-by: Miguel Lopez <[email protected]>
chucheria
added a commit
that referenced
this pull request
Feb 2, 2024
* Add new models: OpenAI GPT3.5-Turbo and Azure OpenAI. Azure OpenAI allows for the embeddings model to be from a different endpoint * Parser for the API generation and correction response * add models, prompts and add to tests * Changes: - Change in base.py from prompts, formatting the prompt - Change in test_prompts adding a new symbol to delete * add data for examples --------- Co-authored-by: Miguel Lopez <[email protected]>
chucheria
added a commit
that referenced
this pull request
Mar 1, 2024
* initial commit * [Feature: New model] API Generation (#6) * Add new models: OpenAI GPT3.5-Turbo and Azure OpenAI. Azure OpenAI allows for the embeddings model to be from a different endpoint * Parser for the API generation and correction response * add models, prompts and add to tests * Changes: - Change in base.py from prompts, formatting the prompt - Change in test_prompts adding a new symbol to delete * add data for examples --------- Co-authored-by: Miguel Lopez <[email protected]> * task_builder.py: - New task added. New prompts added: - json-summarizer in spanish and english json_summarizer.py: - New model class for JSON summarization task. TODO: Fix the example and delete commented code __init__.py: - Added the new model in the init script promptmeteo/parsers/__init__.py: - New option for new type of task (json summarization). - TODO: We must change it to use a new parser for JSON treatment. * JSON parser added * Changes: prompts. New prompt for anthropic claude added and name changing for gpt 3.5 models. pyproject.toml. Added boto3 in the requirements to connect with aws. __init__.py. Name changing of the task json_info_extraction.py: - New file replacing json_summarizer.py with the new naming /models/__init__.py: - New model provider added for Bedrock /models/bedrock.py: - Class for Bedrock models. - Only anthropic claude v2 integrated - By the moment, huggingface embeddings /parsers/__init__.py: - Name changing of the parser /tasks/task_builder.py: - Name changing and summarization task included for new development. * __init__.py: - Added Summarizer class json_info_extraction.py: - Changes in the example and code comments. - json_fields parameter removed due to it is implicit in fields_description summarizer.py: - New class for Summarization task parsers/__init__.py: - New parser added for summarization task (dummy parser) new prompts for anthropic claude summarization and minor changes in prompts tests/tools/dictionary_checker: - 'sample' word added in spanish dictionary because is a keyword for injecting. * Minor changes in example documentation * Removing comments * Minor change: - Change for region selection * Changes: - test_models.py: Added model Bedrock for unit test - test_parsers.py: Added unit test for json parser. - json_parser.py: Change in description * Merge branch 'main' into integracion-summary * Changes: json_info_extraction.py: - Adapted to new Base model building structure summarizer.py: - Adapted to new Base model building structure. pyproject.toml: - boto3 added * prompts added and readme * Changes: prompts/base.py: - Change to allow to fill 'domain' field in prompt even if there is no 'prompt domain' param (same with prompt detail) - Removal of JSON info extraction task * Changes in vocabulary in prompts and minor bug in prompts/base * Minor changes in prompt/base, new words added to spanish dictionary and .prompts modification * api_formatter no changes * Changes: - Added in BedrockLLM model the option of kwargs argument to allow to select different arguments for boto3 client - Change in test for models the region argument * minor error in test model * minor --------- Co-authored-by: Angel Delgado Panadero <[email protected]> Co-authored-by: Bea <[email protected]> Co-authored-by: Miguel Lopez <[email protected]> Co-authored-by: Bea <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introducing Two New Code Generation Models:
API Generation Models
We're excited to unveil two powerful models designed to streamline code generation: the API Generator and API Formatter.
1. API Generator:
2. API Formatter:
Both models leverage the concept that valuable information from examples should be harnessed for effective code generation. Rigorously tested under GPT3.5, a new parser is introduced to ensure the output is exclusively in YAML format.
Noteworthy Changes:
However, it's crucial to mention that the
test_prompt_spellingcurrently does not pass for the API Formatter due to the absence of the{__EXAMPLES__}label in its prompt.Model Deprecation: Transition to gpt-3.5-turbo-instruct
In response to OpenAI's deprecation of the instruct model
text-davinci, we have updated our model to usegpt-3.5-turbo-instruct. The transition includes appropriate modifications, including updated examples. However, it is advised to review the results, as there may be variations from the previous model.