Skip to content

Commit

Permalink
rerank nimble notebooks (#149)
Browse files Browse the repository at this point in the history 
* add nimble notebooks

* update notebook
  • Loading branch information
@alekhya-n
alekhya-n authored Jul 15, 2024
1 parent 6482319 commit 1b9d982
Show file tree
Hide file tree
Showing 2 changed files with 593 additions and 0 deletions.
299 changes: 299 additions & 0 deletions notebooks/sagemaker/rerank_v3_notebooks/Deploy rerank nimble english v3.0 model.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,299 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Deploy cohere-rerank-nimble-english Model Package from AWS Marketplace \n",
"\n",
"\n",
"Cohere builds a collection of Large Language Models (LLMs) trained on a massive corpus of curated web data. Powering these models, our infrastructure enables our product to be deployed for a wide range of use cases. The use cases we power include generation (copy writing, etc), summarization, classification, content moderation, information extraction, semantic search, and contextual entity extraction\n",
"\n",
"This sample notebook shows you how to deploy [cohere-rerank-nimble-english](https://aws.amazon.com/marketplace/pp/prodview-xwsyvhz7rkjqe) using Amazon SageMaker.\n",
"\n",
"> **Note**: This is a reference notebook and it cannot run unless you make changes suggested in the notebook.\n",
"\n",
"## Pre-requisites:\n",
"1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.\n",
"1. Ensure that IAM role used has **AmazonSageMakerFullAccess**\n",
"1. To deploy this ML model successfully, ensure that:\n",
" 1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used: \n",
" 1. **aws-marketplace:ViewSubscriptions**\n",
" 1. **aws-marketplace:Unsubscribe**\n",
" 1. **aws-marketplace:Subscribe** \n",
" 2. or your AWS account has a subscription to [cohere-rerank-nimble-english](https://aws.amazon.com/marketplace/pp/prodview-xwsyvhz7rkjqe). If so, skip step: [Subscribe to the model package](#1.-Subscribe-to-the-model-package)\n",
"\n",
"## Contents:\n",
"1. [Subscribe to the model package](#1.-Subscribe-to-the-model-package)\n",
"2. [Create an endpoint and perform real-time inference](#2.-Create-an-endpoint-and-perform-real-time-inference)\n",
" 1. [Create an endpoint](#A.-Create-an-endpoint)\n",
" 2. [Create input payload](#B.-Create-input-payload)\n",
" 3. [Perform real-time inference](#C.-Perform-real-time-inference)\n",
" 4. [Delete the endpoint](#H.-Delete-the-endpoint)\n",
"3. [Clean-up](#4.-Clean-up)\n",
" 1. [Delete the model](#A.-Delete-the-model)\n",
" 2. [Unsubscribe to the listing (optional)](#B.-Unsubscribe-to-the-listing-(optional))\n",
" \n",
"\n",
"## Usage instructions\n",
"You can run this notebook one cell at a time (By using Shift+Enter for running a cell)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Subscribe to the model package"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To subscribe to the model package:\n",
"1. Open the model package listing page [cohere-rerank-nimble-english](https://aws.amazon.com/marketplace/pp/prodview-xwsyvhz7rkjqe)\n",
"1. On the AWS Marketplace listing, click on the **Continue to subscribe** button.\n",
"1. On the **Subscribe to this software** page, review and click on **\"Accept Offer\"** if you and your organization agrees with EULA, pricing, and support terms. \n",
"1. Once you click on **Continue to configuration button** and then choose a **region**, you will see a **Product Arn** displayed. This is the model package ARN that you need to specify while creating a deployable model using Boto3. Copy the ARN corresponding to your region and specify the same in the following cell."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade cohere-aws\n",
"# if you upgrade the package, you need to restart the kernel\n",
"\n",
"from cohere_aws import Client\n",
"import boto3"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"cohere_package = \"UPDATE_THIS_HERE\"\n",
"model_package_map = {\n",
" \"us-east-1\": f\"arn:aws:sagemaker:us-east-1:865070037744:model-package/{cohere_package}\",\n",
" \"us-east-2\": f\"arn:aws:sagemaker:us-east-2:057799348421:model-package/{cohere_package}\",\n",
" \"us-west-1\": f\"arn:aws:sagemaker:us-west-1:382657785993:model-package/{cohere_package}\",\n",
" \"us-west-2\": f\"arn:aws:sagemaker:us-west-2:594846645681:model-package/{cohere_package}\",\n",
" \"ca-central-1\": f\"arn:aws:sagemaker:ca-central-1:470592106596:model-package/{cohere_package}\",\n",
" \"eu-central-1\": f\"arn:aws:sagemaker:eu-central-1:446921602837:model-package/{cohere_package}\",\n",
" \"eu-west-1\": f\"arn:aws:sagemaker:eu-west-1:985815980388:model-package/{cohere_package}\",\n",
" \"eu-west-2\": f\"arn:aws:sagemaker:eu-west-2:856760150666:model-package/{cohere_package}\",\n",
" \"eu-west-3\": f\"arn:aws:sagemaker:eu-west-3:843114510376:model-package/{cohere_package}\",\n",
" \"eu-north-1\": f\"arn:aws:sagemaker:eu-north-1:136758871317:model-package/{cohere_package}\",\n",
" \"ap-southeast-1\": f\"arn:aws:sagemaker:ap-southeast-1:192199979996:model-package/{cohere_package}\",\n",
" \"ap-southeast-2\": f\"arn:aws:sagemaker:ap-southeast-2:666831318237:model-package/{cohere_package}\",\n",
" \"ap-northeast-2\": f\"arn:aws:sagemaker:ap-northeast-2:745090734665:model-package/{cohere_package}\",\n",
" \"ap-northeast-1\": f\"arn:aws:sagemaker:ap-northeast-1:977537786026:model-package/{cohere_package}\",\n",
" \"ap-south-1\": f\"arn:aws:sagemaker:ap-south-1:077584701553:model-package/{cohere_package}\",\n",
" \"sa-east-1\": f\"arn:aws:sagemaker:sa-east-1:270155090741:model-package/{cohere_package}\",\n",
"}\n",
"\n",
"region = boto3.Session().region_name\n",
"if region not in model_package_map.keys():\n",
" raise Exception(\"UNSUPPORTED REGION\")\n",
"\n",
"model_package_arn = model_package_map[region]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Create an endpoint and perform real-time inference"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"If you want to understand how real-time inference with Amazon SageMaker works, see [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-hosting.html)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### A. Create an endpoint"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"co = Client(region_name=region)\n",
"co.create_endpoint(arn=model_package_arn, endpoint_name=\"cohere-rerank-nimble-english-v3\", instance_type=\"ml.g4dn.xlarge\", n_instances=1)\n",
"\n",
"# If the endpoint is already created, you just need to connect to it\n",
"# co.connect_to_endpoint(endpoint_name=\"cohere-rerank-nimble-english-v3\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Once endpoint has been created, you would be able to perform real-time inference."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### B. Create input payload"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"documents = [\n",
" {\"Title\":\"Incorrect Password\",\"Content\":\"Hello, I have been trying to access my account for the past hour and it keeps saying my password is incorrect. Can you please help me?\"},\n",
" {\"Title\":\"Confirmation Email Missed\",\"Content\":\"Hi, I recently purchased a product from your website but I never received a confirmation email. Can you please look into this for me?\"},\n",
" {\"Title\":\"Questions about Return Policy\",\"Content\":\"Hello, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.\"},\n",
" {\"Title\":\"Customer Support is Busy\",\"Content\":\"Good morning, I have been trying to reach your customer support team for the past week but I keep getting a busy signal. Can you please help me?\"},\n",
" {\"Title\":\"Received Wrong Item\",\"Content\":\"Hi, I have a question about my recent order. I received the wrong item and I need to return it.\"},\n",
" {\"Title\":\"Customer Service is Unavailable\",\"Content\":\"Hello, I have been trying to reach your customer support team for the past hour but I keep getting a busy signal. Can you please help me?\"},\n",
" {\"Title\":\"Return Policy for Defective Product\",\"Content\":\"Hi, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.\"},\n",
" {\"Title\":\"Wrong Item Received\",\"Content\":\"Good morning, I have a question about my recent order. I received the wrong item and I need to return it.\"},\n",
" {\"Title\":\"Return Defective Product\",\"Content\":\"Hello, I have a question about the return policy for this product. I purchased it a few weeks ago and it is defective.\"}\n",
"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### C. Perform real-time inference"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = co.rerank(documents=documents, query='What emails have been about returning items?', rank_fields=[\"Title\",\"Content\"], top_n=5)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### D. Visualize output"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f'Documents: {response}') "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4. Clean-up"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### A. Delete the model"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you have successfully performed a real-time inference, you do not need the endpoint any more. You can terminate the endpoint to avoid being charged."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"co.delete_endpoint()\n",
"co.close()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### B. Unsubscribe to the listing (optional)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"If you would like to unsubscribe to the model package, follow these steps. Before you cancel the subscription, ensure that you do not have any [deployable model](https://console.aws.amazon.com/sagemaker/home#/models) created from the model package or using the algorithm. Note - You can find this information by looking at the container name associated with the model. \n",
"\n",
"**Steps to unsubscribe to product from AWS Marketplace**:\n",
"1. Navigate to __Machine Learning__ tab on [__Your Software subscriptions page__](https://aws.amazon.com/marketplace/ai/library?productType=ml&ref_=mlmp_gitdemo_indust)\n",
"2. Locate the listing that you want to cancel the subscription for, and then choose __Cancel Subscription__ to cancel the subscription.\n",
"\n"
]
}
],
"metadata": {
"instance_type": "ml.t3.medium",
"kernelspec": {
"display_name": "deploy-cohere-rerank-english",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Loading

0 comments on commit 1b9d982

Please sign in to comment.