Skip to content

Commit 98f3383

Browse files
lyakhovvvllyakho
lyakhovv
and
vllyakho
authored
How to connect from SageMaker to Keyspaces (#19)
* Introducing SageMaker connection examples using Sig4V and server-generated credentials * Introducing SageMaker connection examples using Sig4V and server-generated credentials. Grammar/style changes to Readme and notebook text. * Added links to SigV4 for reference * 1. Added license short text to the notebook 2. Moved SageMaker directory under Python * Changes URL for SageMaker examples Co-authored-by: Vadim Lyakhovich <[email protected]>
1 parent bbe53a9 commit 98f3383

File tree

6 files changed

+542
-1
lines changed

6 files changed

+542
-1
lines changed

‎CHANGELOG.md

+1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
# CHANGELOG
22
[0.0.1] Iniital Release
3+
[0.0.2] Added SageMaker section

‎README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ This repo contains examples in each language supported by Amazon Keyspaces. This
2525
- [Ruby Examples](https://github.com/aws-samples/amazon-keyspaces-examples/tree/main/ruby)
2626
- [Python Examples](https://github.com/aws-samples/amazon-keyspaces-examples/tree/main/python)
2727
- [Scala Examples](https://github.com/aws-samples/amazon-keyspaces-examples/tree/main/scala)
28+
- [SageMaker (Python) Examples](https://github.com/aws-samples/amazon-keyspaces-examples/tree/main/python/datastax-v3/SageMaker)
29+
30+
2831

2932
## Learning Resources <a name="Learning"></a>
3033
While this is an excellent learning resource for Amazon Keyspaces, there are other resources that can be referenced to assist with your learning/development process.
@@ -39,4 +42,3 @@ If you have created an Amazon Keyspaces learning resource and would like it to b
3942
# License <a name="License"></a>
4043

4144
This library is licensed under the MIT-0 License.
42-

‎python/datastax-v3/SageMaker/connection-service-specific-credentials /README.md

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
2+
3+
## Connecting to Amazon Keyspaces from SageMaker Notebook with Python
4+
5+
6+
This code shows how to connect to Amazon Keyspaces from SageMaker using [service-specific credentials](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.credentials.html).
7+
8+
Service-specific credentials aren’t the only way to authenticate and authorize access to Amazon Keyspaces resources. We recommend using AWS authentication plugin for Cassandra drivers .
9+
10+
11+
### Prerequisites<a name="Prerequisites"></a></a>
12+
The Notebook execution role must include permissions to access Amazon Keyspaces and [Secret Manager](https://aws.amazon.com/secrets-manager/).
13+
14+
* To access Amazon Keyspaces database - use AmazonKeyspacesReadOnlyAccess or AmazonKeyspacesFullAccess managed policies. Use the _least privileged approach_ for your production application.
15+
See more at
16+
[AWS Identity and Access Management for Amazon Keyspaces](https://docs.aws.amazon.com/keyspaces/latest/devguide/security-iam.html).
17+
18+
* To use AWS Secret Manager, the Notebooks execution role must include [SecretsManagerReadWrite](https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_available-policies.html) managed policy.
19+
20+
21+
22+
### Security Configuration
23+
24+
1. Generate [Keyspaces Service-Specific Credentials](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.credentials.html)
25+
26+
27+
Example of a service-specific credential
28+
29+
```
30+
"ServiceSpecificCredential": {
31+
"CreateDate": "2019-10-09T16:12:04Z",
32+
"ServiceName": "cassandra.amazonaws.com",
33+
"ServiceUserName": "keyspace-user1-at-11122223333",
34+
"ServicePassword": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
35+
"ServiceSpecificCredentialId": "ACCAYFI33SINPGJEBYESF",
36+
"UserName": " keyspace-user1",
37+
"Status": "Active"
38+
}
39+
}
40+
```
41+
42+
2. Store ServiceUserName and ServicePassword in the SecretManager. As a best practice, we don't want to store credentials as a plain text in the SageMaker Notebooks.
43+
44+
In this example I'm using
45+
_Keyspaces_Server_Generated_credential_ as a Secret Name and _keyspaces_generated_id_ and _keyspaces_generated_pw_ fields to store Keyspaces ID and password.
46+
47+
48+
49+
50+
#### Note:
51+
Amazon Keyspaces is available in the following [AWS Regions](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.endpoints.html).
52+
53+
This notebook was tested with conda_python3 kernel and should work with Python 3.x.
54+
55+
### Running the sample
56+
* Import Notebook into SageMaker

‎python/datastax-v3/SageMaker/connection-service-specific-credentials /SageMaker_Keyspaces_with_server_credentials.ipynb

+220
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "0fb7f3db",
6+
"metadata": {},
7+
"source": [
8+
"*Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved* \n",
9+
"\n",
10+
"*SPDX-License-Identifier: MIT-0* "
11+
]
12+
},
13+
{
14+
"cell_type": "markdown",
15+
"id": "b78e5991",
16+
"metadata": {},
17+
"source": [
18+
"---"
19+
]
20+
},
21+
{
22+
"cell_type": "markdown",
23+
"id": "81e637d6",
24+
"metadata": {},
25+
"source": [
26+
"## Connecting to Amazon Keyspaces using server-side credentials \n",
27+
"\n",
28+
"This code shows how to connect to Amazon Keyspaces from SageMaker using an [service-specific credentials](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.credentials.html) for an existing AWS Identity and Access Management (IAM) user.\n"
29+
]
30+
},
31+
{
32+
"cell_type": "markdown",
33+
"id": "8a1a923c",
34+
"metadata": {},
35+
"source": [
36+
"Before we start we need to generate the Keyspaces credential and use SecretManager to securly store credentials. \n",
37+
"\n",
38+
"\n",
39+
"1. Generate [Keyspaces Service-Specific Credentials](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.credentials.html)\n",
40+
"\n",
41+
"\n",
42+
"The following is an example of a service-specific credential .\n",
43+
"\n",
44+
"```\n",
45+
"\"ServiceSpecificCredential\": {\n",
46+
" \"CreateDate\": \"2019-10-09T16:12:04Z\",\n",
47+
" \"ServiceName\": \"cassandra.amazonaws.com\",\n",
48+
" \"ServiceUserName\": \"keyspace-user1-at-11122223333\",\n",
49+
" \"ServicePassword\": \"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\",\n",
50+
" \"ServiceSpecificCredentialId\": \"ACCAYFI33SINPGJEBYESF\",\n",
51+
" \"UserName\": \" keyspace-user1\",\n",
52+
" \"Status\": \"Active\"\n",
53+
" }\n",
54+
"}\n",
55+
"```\n",
56+
"\n",
57+
"2. Store ServiceUserName and ServicePassword in the SecretManager. As a best practice we don't want to store credential in SageMaker Notebooks in plain text. \n",
58+
"\n",
59+
" In this example I'm using \n",
60+
"*Keyspaces_Server_Generated_credential* as a Secret name and _keyspaces_generated_id_ and _keyspaces_generated_pw_ fields to store Keyspaces ID and password. \n"
61+
]
62+
},
63+
{
64+
"cell_type": "code",
65+
"execution_count": null,
66+
"id": "a91e2ab0",
67+
"metadata": {},
68+
"outputs": [],
69+
"source": [
70+
"#Import \n",
71+
"from cassandra.cluster import Cluster\n",
72+
"from ssl import SSLContext, PROTOCOL_TLSv1_2 , CERT_REQUIRED\n",
73+
"from cassandra.auth import PlainTextAuthProvider\n",
74+
"\n",
75+
"ssl_context = SSLContext(PROTOCOL_TLSv1_2 )"
76+
]
77+
},
78+
{
79+
"cell_type": "code",
80+
"execution_count": null,
81+
"id": "68e63b3f",
82+
"metadata": {},
83+
"outputs": [],
84+
"source": [
85+
"#Download certificate \n",
86+
"!curl https://certs.secureserver.net/repository/sf-class2-root.crt -O "
87+
]
88+
},
89+
{
90+
"cell_type": "code",
91+
"execution_count": null,
92+
"id": "a714c7dc",
93+
"metadata": {},
94+
"outputs": [],
95+
"source": [
96+
"# Load certificate \n",
97+
"ssl_context.load_verify_locations('sf-class2-root.crt') "
98+
]
99+
},
100+
{
101+
"cell_type": "code",
102+
"execution_count": null,
103+
"id": "265d59f0",
104+
"metadata": {},
105+
"outputs": [],
106+
"source": [
107+
"#Getting Credential from Secert Manager \n",
108+
"#Need to have a SecretsManagerReadWrite Policy\n",
109+
"\n",
110+
"import boto3\n",
111+
"import base64\n",
112+
"from botocore.exceptions import ClientError\n",
113+
"import json\n",
114+
"\n",
115+
"secret_name = \"Keyspaces_Server_Generated_credential\"\n",
116+
"region_name = \"us-west-1\"\n",
117+
"\n",
118+
"# Create a Secrets Manager client\n",
119+
"session = boto3.session.Session()\n",
120+
"client = session.client(\n",
121+
" service_name='secretsmanager',\n",
122+
" region_name=region_name\n",
123+
")\n",
124+
"\n",
125+
"\n",
126+
"get_secret_value_response = client.get_secret_value(\n",
127+
" SecretId=secret_name\n",
128+
")\n",
129+
"\n",
130+
"# Decrypts secret using the associated KMS CMK.\n",
131+
"# Depending on whether the secret is a string or binary, one of these fields will be populated.\n",
132+
"if 'SecretString' in get_secret_value_response:\n",
133+
" secret = json.loads(get_secret_value_response['SecretString'])\n",
134+
" ServiceUserName = secret['keyspaces_generated_id']\n",
135+
" ServicePassword = secret['keyspaces_generated_pw']\n",
136+
" print(\"id:\",ServiceUserName, \", pw:\",ServicePassword)\n",
137+
"else:\n",
138+
" decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])\n",
139+
" \n",
140+
" "
141+
]
142+
},
143+
{
144+
"cell_type": "code",
145+
"execution_count": null,
146+
"id": "2a98e386",
147+
"metadata": {},
148+
"outputs": [],
149+
"source": [
150+
"auth_provider = PlainTextAuthProvider(username=ServiceUserName, password=ServicePassword)"
151+
]
152+
},
153+
{
154+
"cell_type": "code",
155+
"execution_count": null,
156+
"id": "bad5758b",
157+
"metadata": {},
158+
"outputs": [],
159+
"source": [
160+
"cluster = Cluster(['cassandra.us-west-1.amazonaws.com'], ssl_context=ssl_context, auth_provider=auth_provider, port=9142)\n"
161+
]
162+
},
163+
{
164+
"cell_type": "code",
165+
"execution_count": null,
166+
"id": "6b22f21d",
167+
"metadata": {},
168+
"outputs": [],
169+
"source": [
170+
"session = cluster.connect()\n",
171+
"cql = \"select * from system_schema.keyspaces \"\n",
172+
"r = session.execute(cql)\n",
173+
"print(r.current_rows)"
174+
]
175+
},
176+
{
177+
"cell_type": "code",
178+
"execution_count": null,
179+
"id": "eb124f18",
180+
"metadata": {},
181+
"outputs": [],
182+
"source": [
183+
"# Load rows to Panda DataFrame \n",
184+
"from pandas import DataFrame\n",
185+
"\n",
186+
"df = DataFrame(r)\n",
187+
"print(df)"
188+
]
189+
},
190+
{
191+
"cell_type": "code",
192+
"execution_count": null,
193+
"id": "3c9b9169",
194+
"metadata": {},
195+
"outputs": [],
196+
"source": []
197+
}
198+
],
199+
"metadata": {
200+
"kernelspec": {
201+
"display_name": "conda_python3",
202+
"language": "python",
203+
"name": "conda_python3"
204+
},
205+
"language_info": {
206+
"codemirror_mode": {
207+
"name": "ipython",
208+
"version": 3
209+
},
210+
"file_extension": ".py",
211+
"mimetype": "text/x-python",
212+
"name": "python",
213+
"nbconvert_exporter": "python",
214+
"pygments_lexer": "ipython3",
215+
"version": "3.6.13"
216+
}
217+
},
218+
"nbformat": 4,
219+
"nbformat_minor": 5
220+
}

‎python/datastax-v3/SageMaker/connection-sigv4/README.md

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
## Connecting to Amazon Keyspaces from SageMaker Notebook with Python
2+
3+
This code shows how to connect to Amazon Keyspaces from SageMaker using an authentication plugin for temporary credentials. This plugin enables IAM users, roles, and federated identities to add authentication information to Amazon Keyspaces API requests using the [AWS Signature Version 4 process (SigV4)](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)
4+
5+
In this example, we do NOT need to generate Keyspaces service-specific credentials.
6+
7+
8+
9+
### Prerequisites
10+
11+
The Notebook execution role must include permissions to access Amazon Keyspaces and Assume the role.
12+
13+
* To access Amazon Keyspaces database - use AmazonKeyspacesReadOnlyAccess or AmazonKeyspacesFullAccess managed policies. Use the _least privileged approach_ for your production application.
14+
See more at
15+
[AWS Identity and Access Management for Amazon Keyspaces](https://docs.aws.amazon.com/keyspaces/latest/devguide/security-iam.html).
16+
17+
* To assume the role, you need to have [sts:AssumeRole action](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) permissions.
18+
```
19+
{
20+
"Version": "2012-10-17",
21+
"Statement": [
22+
{
23+
"Action": [
24+
"sts:AssumeRole"
25+
],
26+
"Effect": "Allow",
27+
"Resource": "*"
28+
}
29+
]
30+
}
31+
```
32+
33+
#### Note:
34+
Amazon Keyspaces is available in the following [AWS Regions](https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.endpoints.html).
35+
36+
This notebook was tested with conda_python3 kernel and should work with Python 3.x.
37+
38+
39+
40+
### Running the sample
41+
* Import Notebook into SageMaker

0 commit comments

Comments
 (0)