Skip to content
This repository was archived by the owner on Jul 3, 2024. It is now read-only.

Commit a59338c

Browse files
committed
Complete readme, update screenshots, assets, examples.
* Code pattern is complete except for flow diagram
1 parent c72d5a3 commit a59338c

15 files changed

+723
-186
lines changed

README.md

Lines changed: 17 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
*** WORK-IN-PROGRESS ***
21
# Score streaming data with a machine learning model
32

43
In this code pattern, we will be streaming online shopping data and using the data to track the products that each customer has added to their cart. We will build a k-means clustering model with scikit-learn to group customers according to the contents of their shopping carts. The cluster assignment can be used to predict additional products to recommend.
@@ -32,17 +31,13 @@ Using the Streams Flows editor, we will create a streaming application with the
3231
1. [Verify access to your IBM Streams instance on Cloud Pak for Data](#1-Verify-access-to-your-IBM-Streams-instance-on-Cloud-Pak-for-Data)
3332
1. [Create a new project in Cloud Pak for Data](#2-create-a-new-project-in-cloud-pak-for-data)
3433
1. [Build and store a model](#2-build-and-store-a-model)
35-
36-
1. [Create a Streams Flow in Cloud Pak for Data](#6-create-a-streams-flow-in-cloud-pak-for-data)
37-
1. [Create a Streams Flow with Kafka as source](#7-create-a-streams-flow-with-kafka-as-source)
38-
1. [Use Streams Flows option to generate a notebook](#8-use-streams-flows-option-to-generate-a-notebook)
39-
1. [Run the generated Streams Flow notebook](#9-run-the-generated-streams-flow-notebook)
34+
1. [Create and run a Streams Flow application](#4-create-and-run-a-streams-flow-application)
4035

4136
### 1. Verify access to your IBM Streams instance on Cloud Pak for Data
4237

4338
Once you login to your `Cloud Pak for Data` instance, ensure that your administrator has provisioned an instance of `IBM Streams`, and has given your user has access to the instance.
4439

45-
To see the available services, click on the `Services` icon. Search for `Streams`. You should see an `Enabled` indicator for Streams. `Watson Studio` and `Watson Machine Learning` also need to be enabled to build and deploy the model.
40+
To see the available services, click on the `Services` icon. Search for `Streams`. You should see an `Enabled` indicator for `Streams`. `Watson Studio` and `Watson Machine Learning` also need to be enabled to build and deploy the model.
4641

4742
![catalog_streams.png](doc/source/images/catalog_streams.png)
4843

@@ -60,9 +55,7 @@ Click on `New project +`. Then select `Create an empty project` and enter a uniq
6055

6156
### 2. Build and store a model
6257

63-
We will build a model using a Jupyter notebook and scikit-learn. We're using a k-means classifier to group customers based on the contents of their shopping carts. Later, we will use that model to predict which group a customer is most likely to go in so that we can anticipate additional products to recommend.
64-
65-
Once we have built and stored the model, it will be available for deployment so that it can be used in our streaming application.
58+
We will build a model using a Jupyter notebook and scikit-learn. We're using a k-means classifier to group customers based on the contents of their shopping carts. Once we have built and stored the model, it will be available for deployment so that it can be used in our streaming application.
6659

6760
#### Import the notebook into your project
6861

@@ -79,13 +72,15 @@ Fill in the following information:
7972
```url
8073
https://raw.githubusercontent.com/IBM/ibm-streams-with-ml-model/master/notebooks/shopping_cart_kmeans_cluster_model.ipynb
8174
```
82-
![new-notebook](doc/source/images/new-notebook.png)
75+
* Click the `Create notebook` button.
8376

84-
Click the `Create notebook` button.
77+
![new-notebook](doc/source/images/new-notebook.png)
8578

8679
#### Edit the notebook
8780

88-
When you import the notebook you will be put in edit mode. Before running the notebook, you need to configure one thing. Edit the `WML Credentials` cell to set the `url` to the URL you use to access Cloud Pak for Data.
81+
When you import the notebook you will be put in edit mode. Before running the notebook, you need to configure one thing:
82+
83+
* Edit the `WML Credentials` cell to set the `url` to the URL you use to access Cloud Pak for Data.
8984

9085
![wml_creds.png](doc/source/images/wml_creds.png)
9186

@@ -95,24 +90,24 @@ Select `Cell > Run All` to run the notebook. If you prefer, you can use the `Run
9590

9691
### 2. Associate the deployment space with the project
9792

98-
The notebook created a deployment space named "Shopping Cart k-means Model" and stored the model there.
93+
The notebook created a deployment space named `ibm_streams_with_ml_model_deployment_space` and stored the model there.
9994

10095
Inside your new project, select the `Settings` tab and click on `Associate a deployment space +`.
10196

10297
![deployment_space.png](doc/source/images/deployment_space.png)
10398

10499
* Use `Existing` tab
105-
* Select the `streams_ml_deployment_space` which was just created
100+
* Select the `ibm_streams_with_ml_model_deployment_space` which was just created
106101
* Click `Associate`
107102

108103
### 3. Deploy the model
109104

110-
* In your project, click on the newly associated deployment space named `streams_ml_deployment_space`.
111-
* Select the `Assets` tab and click on the model named `Shopping Cart k-means Model`.
105+
* In your project, click on the newly associated deployment space named `ibm_streams_with_ml_model_deployment_space`.
106+
* Select the `Assets` tab and click on the model named `Shopping Cart Cluster Model`.
112107
* Click on the `Create deployment` button.
113108
* Select `Online`, provide a deployment name, and click `Create`.
114109

115-
### 6. Create a Streams Flow
110+
### 4. Create and run a Streams Flow application
116111

117112
From the project panel, click the `Add to project +` button. Choose the `Streams flow` tile from the list of options.
118113

@@ -136,6 +131,8 @@ From the `Sources` list, select and drag the `Sample Data` operator onto the can
136131

137132
![add-sample-data-source](doc/source/images/add-sample-data-source.png)
138133

134+
> NOTE: As you've probably already noticed, red error indicators tell you when required settings are missing. For example, some settings are required. You will also see that some operators require a source and/or target connection. When an operator has a red spot on it, you can hover over it to see what the errors are.
135+
139136
Click on the canvas object to see its associated properties. From the list of available data types in the `Topic` drop-down list, select `Clickstream`.
140137

141138
![set-sample-data-topic](doc/source/images/set-sample-data-topic.png)
@@ -150,20 +147,13 @@ From the `Processing and Analytics` list, select and drag the `Filter` operator
150147

151148
![add-filter-target](doc/source/images/add-filter-target.png)
152149

153-
#### Notice error indicators
154-
155-
As you've probably already noticed, red error indicators tell you when required settings are missing. For example, we've seen settings that were required. You will also see that some operators require a source and/or target connection.
156-
157-
When an operator has a red spot on it, you can hover over it to see what the erros are.
158-
159150
#### Add a Code operator
160151

161152
* From the `Processing and Analytics` list, select and drag the `Code` operator onto the canvas
162153
* Connect the `Filter` operator's target to this `Code` operator's source (using drag-and-drop like we did earlier)
163154
* Click on the Code operator to see its associated properties. Select `Python 3.6` as the `Coding Language`.
164155
* In the `Code` property, paste in the following code:
165156
```python
166-
YOU MUST EDIT THE SCHEMA and add all attributes that you are returning as output.
167157
#
168158
# Preinstalled Python packages can be viewed from the Settings pane.
169159
# In the Settings pane you can also install additional Python packages.
@@ -181,7 +171,7 @@ When an operator has a red spot on it, you can hover over it to see what the err
181171
def init(state):
182172
# do something once on flow initialization and save in the state object
183173
state['keep_columns'] = ['Baby Food','Diapers','Formula','Lotion','Baby wash','Wipes','Fresh Fruits','Fresh Vegetables','Beer','Wine','Club Soda','Sports Drink','Chips','Popcorn','Oatmeal','Medicines','Canned Foods','Cigarettes','Cheese','Cleaning Products','Condiments','Frozen Foods','Kitchen Items','Meat','Office Supplies','Personal Care','Pet Supplies','Sea Food','Spices']
184-
state['empty_cart'] = [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
174+
state['empty_cart'] = [0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5]
185175
state['customer_carts'] = {}
186176
pass
187177

@@ -215,7 +205,7 @@ When an operator has a red spot on it, you can hover over it to see what the err
215205

216206
The output now contains just the customer_id an array indicating which products are in the cart. This is the format we needed to pass to our model.
217207

218-
> Notice: We used the Code operator specifically to arrange our shopping cart data for scoring, but if you take another look at the Code operator you'll see that it is a very powerful operator where you can put in the Python code to do whatever manipulation you need in your streaming application.
208+
> NOTE: We used the Code operator specifically to arrange our shopping cart data for scoring, but if you take another look at the Code operator you'll see that it is a very powerful operator where you can put in the Python code to do whatever manipulation you need in your streaming application.
219209
220210
#### Add a WML Deployment
221211

-36.3 KB
Loading

doc/source/images/add-notebook.png

46.3 KB
Loading
-873 Bytes
Loading
21.3 KB
Loading
-9.15 KB
Loading
-2.86 KB
Loading
-133 KB
Loading

doc/source/images/new-notebook.png

-11.1 KB
Loading

doc/source/images/new-project.png

15.3 KB
Loading
-44.3 KB
Loading
-27.6 KB
Loading

doc/source/images/wml_creds.png

-50.6 KB
Loading

0 commit comments

Comments
 (0)