Skip to content

Commit e42af47

Browse files
authored
Merge pull request #210 from IBM/adding-private-path-example
Adding private path example
2 parents 0cb1ef0 + 8d2bace commit e42af47

27 files changed

+1408
-157
lines changed

.gitignore

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11
*.out
22
node_modules
33
.DS_Store
4-
.vscode
4+
.vscode
5+
6+
# draw.io temp files
7+
.$*.bkp
8+
.$*.dtmp
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
FROM registry.access.redhat.com/ubi9/nodejs-20:latest AS build-env
1+
FROM registry.access.redhat.com/ubi9/nodejs-22:latest AS build-env
22
WORKDIR /opt/app-root/src
33
COPY --chown=default:root job/* .
44
RUN npm install
55

66
# Use a small distroless image for as runtime image
7-
FROM gcr.io/distroless/nodejs20-debian12
7+
FROM gcr.io/distroless/nodejs22
88
COPY --from=build-env /opt/app-root/src /app
99
WORKDIR /app
1010
ENTRYPOINT ["job.mjs"]

cloudant-change-listener/job/package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44
"description": "Change event listener for cloudant DB",
55
"main": "job.mjs",
66
"engines": {
7-
"node": ">=20",
8-
"npm": "^10"
7+
"node": "^22",
8+
"npm": "^11"
99
},
1010
"scripts": {
1111
"test": "echo \"Error: no test specified\" && exit 1"

gallery/README.md

Lines changed: 18 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,15 @@ OK
5353
Read the project guid from the CLI output and store it in a local variable.
5454
We'll need it later on to configure the bucket.
5555
```
56-
> export CE_PROJECT_GUID=$(ibmcloud ce project current --output json|jq -r '.guid')
57-
> echo "CE_PROJECT_GUID: $CE_PROJECT_GUID"
56+
$ export CE_PROJECT_GUID=$(ibmcloud ce project current --output json|jq -r '.guid')
57+
$ echo "CE_PROJECT_GUID: $CE_PROJECT_GUID"
5858
5959
CE_PROJECT_GUID: 91efff97-1001-4144-997a-744ec8009303
60+
61+
$ export CE_PROJECT_CRN=$(ibmcloud ce project get --name gallery --output json|jq -r '.crn')
62+
$ echo "CE_PROJECT_CRN: $CE_PROJECT_CRN"
63+
64+
CE_PROJECT_CRN: crn:v1:bluemix:public:codeengine:eu-de:a/7658687ea07db8386963ebe2b8f1897a:91efff97-1001-4144-997a-744ec8009303::
6065
```
6166

6267
Once the project has become active, you are good to proceed with the next step.
@@ -103,7 +108,7 @@ OK
103108
Service instance gallery-cos was created.
104109
105110
Name: gallery-cos
106-
ID: crn:v1:bluemix:public:cloud-object-storage:global:a/7658687ea07db8396963ebe2b8e1897d:c0f324be-33fd-4989-a4af-376a13abb316::
111+
ID: crn:v1:bluemix:public:cloud-object-storage:global:a/7658687ea07db8386963ebe2b8f1897a:c0f324be-33fd-4989-a4af-376a13abb316::
107112
GUID: c0f324be-33fd-4989-a4af-376a13abb316
108113
Location: global
109114
State: active
@@ -233,10 +238,14 @@ Utilize local build capabilities, which is able to take your local source code a
233238
```
234239
$ ibmcloud ce fn create --name change-color \
235240
--build-source . \
236-
--runtime nodejs-20 \
241+
--runtime nodejs-22 \
237242
--memory 4G \
238243
--cpu 1 \
239-
--env BUCKET=$BUCKET
244+
--env TRUSTED_PROFILE_NAME=ce-gallery-to-cos \
245+
--env COS_BUCKET=$BUCKET \
246+
--env COS_REGION=$REGION \
247+
--trusted-profiles-enabled \
248+
--visibility project
240249
241250
Preparing function 'change-color' for build push...
242251
Creating function 'change-color'...
@@ -255,33 +264,14 @@ Run 'ibmcloud ce function get -n change-color' to see more details.
255264
https://change-color.172utxcdky5l.eu-de.codeengine.appdomain.cloud
256265
```
257266

258-
In order to allow the function to read and write to the bucket, we'll need to create a binding between the COS instance and the function to expose the Object Storage credentials to the functions code. As we already created such credentials for the application, we'll want to make sure to re-use it, as opposed to create new ones.
259-
260-
List all service credentials of the Object Storage instance:
261-
```
262-
$ ibmcloud resource service-keys --instance-id $COS_INSTANCE_ID
267+
In order to allow the function to read and write to the bucket, we'll need to create an IAM trusted profile between the COS instance and the function to expose the Object Storage credentials to the functions code.
263268

264-
Retrieving all service keys in resource group default under account John Does's Account as [email protected]...
265-
OK
266-
Name State Created At
267-
gallery-ce-service-binding-prw1t active Fri Sep 8 07:56:19 UTC 2023
268269
```
270+
$ ibmcloud iam trusted-profile-create ce-gallery-to-cos
269271
270-
Extract the name of the service access secret, that has been created for the app
271-
```
272-
$ export COS_SERVICE_CREDENTIAL=$(ibmcloud resource service-keys --instance-id $COS_INSTANCE_ID --output json|jq -r '.[0].name')
273-
$ echo "COS_SERVICE_CREDENTIAL: $COS_SERVICE_CREDENTIAL"
274-
```
272+
$ ibmcloud iam trusted-profile-link-create ce-gallery-to-cos --name ce-fn-change-color --cr-type CE --link-crn ${CE_PROJECT_CRN} --link-component-type function --link-component-name change-color
275273
276-
Finally expose the COS credentials to the function by binding the service access secret to the function
277-
```
278-
$ ibmcloud ce function bind --name change-color \
279-
--service-instance gallery-cos \
280-
--service-credential $COS_SERVICE_CREDENTIAL
281-
282-
Binding service instance...
283-
Status: Done
284-
OK
274+
$ ibmcloud iam trusted-profile-policy-create ce-gallery-to-cos --roles "Writer" --service-name cloud-object-storage --service-instance ${COS_INSTANCE_ID} --resource-type bucket --resource ${BUCKET}
285275
```
286276

287277
In order to complete this step, we'll update the app and make it aware that there is a function that allows to change the colors of individual images.

gallery/app/app.js

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,20 @@ const { open, readFile, writeFile, readdir, unlink } = require("fs/promises");
66

77
const basePath = __dirname; // serving files from here
88

9-
const GALLERY_PATH = process.env.MOUNT_LOCATION || "/app/tmp";
9+
let GALLERY_PATH = "/app/tmp";
10+
11+
// if the optional env var 'MOUNT_LOCATION' is not set, but a bucket has been mounted to /mnt/bucket assume it is a COS mount
12+
let isCosEnabled = false;
13+
if (process.env.MOUNT_LOCATION || existsSync("/mnt/bucket")) {
14+
isCosEnabled = true;
15+
GALLERY_PATH = process.env.MOUNT_LOCATION || "/mnt/bucket";
16+
}
1017

1118
function getFunctionEndpoint() {
1219
if (!process.env.COLORIZER) {
1320
return undefined;
1421
}
15-
return `https://${process.env.COLORIZER}.${process.env.CE_SUBDOMAIN}.${process.env.CE_DOMAIN}`;
22+
return `http://${process.env.COLORIZER}.${process.env.CE_SUBDOMAIN}.function.cluster.local`;
1623
}
1724

1825
async function invokeColorizeFunction(imageId) {
@@ -67,7 +74,7 @@ async function handleHttpReq(req, res) {
6774

6875
if (existsSync(GALLERY_PATH)) {
6976
enabledFeatures.fs = {
70-
cos: !!process.env.MOUNT_LOCATION,
77+
cos: isCosEnabled,
7178
};
7279
}
7380
if (process.env.COLORIZER) {
@@ -183,7 +190,6 @@ async function handleHttpReq(req, res) {
183190
console.log(`Error deleting gallery content: ${err}`);
184191
res.statusCode = 503;
185192
res.end(`Error deleting gallery content: ${err}`);
186-
187193
}
188194
return;
189195
}

gallery/function/cos-service.js

Lines changed: 69 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -6,23 +6,38 @@
66
* disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
77
******************************************************************************/
88

9-
const ibm = require("ibm-cos-sdk");
9+
const { ContainerAuthenticator } = require("ibm-cloud-sdk-core");
10+
const { Readable } = require('node:stream');
1011

12+
const responseToReadable = (response) => {
13+
const reader = response.body.getReader();
14+
const rs = new Readable();
15+
rs._read = async () => {
16+
const result = await reader.read();
17+
if (!result.done) {
18+
rs.push(Buffer.from(result.value));
19+
} else {
20+
rs.push(null);
21+
return;
22+
}
23+
};
24+
return rs;
25+
};
1126
class CosService {
12-
cos;
1327
config;
28+
authenticator;
1429

1530
constructor(config) {
1631
const fn = "constructor";
1732
this.config = config;
18-
this.cos = new ibm.S3(config);
19-
console.debug(
20-
`${fn}- initialized! instance: '${config.serviceInstanceId}'`
21-
);
22-
}
2333

24-
getServiceInstanceId() {
25-
return this.config.serviceInstanceId;
34+
// create an authenticator based on a trusted profile
35+
this.authenticator = new ContainerAuthenticator({
36+
iamProfileName: config.trustedProfileName,
37+
});
38+
console.log(
39+
`CosService init - region: '${this.config.cosRegion}', bucket: ${this.config.cosBucket}, trustedProfileName: '${this.config.trustedProfileName}'`
40+
);
2641
}
2742

2843
getContentTypeFromFileName(fileName) {
@@ -60,61 +75,64 @@ class CosService {
6075
/**
6176
* https://ibm.github.io/ibm-cos-sdk-js/AWS/S3.html#putObject-property
6277
*/
63-
createObject(bucket, id, dataToUpload, mimeType, contentLength) {
78+
async createObject(id, dataToUpload, mimeType, contentLength) {
6479
const fn = "createObject ";
6580
console.debug(`${fn}> id: '${id}', mimeType: '${mimeType}', contentLength: '${contentLength}'`);
6681

67-
return this.cos
68-
.putObject({
69-
Bucket: bucket,
70-
Key: id,
71-
Body: dataToUpload,
72-
ContentType: mimeType,
73-
ContentLength: contentLength,
74-
})
75-
.promise()
76-
.then((obj) => {
77-
console.debug(`${fn}< done`);
78-
return true;
79-
})
80-
.catch((err) => {
81-
console.error(err);
82-
console.debug(`${fn}< failed`);
83-
throw err;
84-
});
85-
}
82+
// prepare the request to create the object files in the bucket
83+
const requestOptions = {
84+
method: "PUT",
85+
body: dataToUpload,
86+
headers: {
87+
"Content-Type": mimeType,
88+
"Content-Length": contentLength,
89+
},
90+
};
8691

87-
/**
88-
* https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-node#node-examples-list-objects
89-
*/
90-
getBucketContents(bucketName, prefix) {
91-
const fn = "getBucketContents ";
92-
console.debug(`${fn}> bucket: '${bucketName}', prefix: '${prefix}'`);
93-
return this.cos
94-
.listObjects({ Bucket: bucketName, Prefix: prefix })
95-
.promise()
96-
.then((data) => {
97-
console.debug(`${fn}< done`);
98-
if (data != null && data.Contents != null) {
99-
return data.Contents;
100-
}
101-
})
102-
.catch((err) => {
103-
console.error(err);
104-
console.debug(`${fn}< failed`);
105-
return undefined;
106-
});
92+
// authenticate the request
93+
await this.authenticator.authenticate(requestOptions);
94+
95+
// perform the request
96+
const response = await fetch(
97+
`https://s3.direct.${this.config.cosRegion}.cloud-object-storage.appdomain.cloud/${this.config.cosBucket}/${id}`,
98+
requestOptions
99+
);
100+
101+
if (response.status !== 200) {
102+
console.error(`Unexpected status code: ${response.status}`);
103+
throw new Error(`Failed to upload image: '${response.status}'`);
104+
}
105+
return;
107106
}
108107

109108
/**
110109
* https://ibm.github.io/ibm-cos-sdk-js/AWS/S3.html#getObject-property
111110
* @param id
112111
*/
113-
getObjectAsStream(bucket, id) {
112+
async getObjectAsStream(id) {
114113
const fn = "getObjectAsStream ";
115114
console.debug(`${fn}> id: '${id}'`);
116115

117-
return this.cos.getObject({ Bucket: bucket, Key: id }).createReadStream();
116+
// prepare the request to list the files in the bucket
117+
const requestOptions = {
118+
method: "GET",
119+
};
120+
121+
// authenticate the request
122+
await this.authenticator.authenticate(requestOptions);
123+
124+
// perform the request
125+
return fetch(
126+
`https://s3.direct.${this.config.cosRegion}.cloud-object-storage.appdomain.cloud/${this.config.cosBucket}/${id}`,
127+
requestOptions
128+
).then((response) => {
129+
if (!response.ok) {
130+
console.error(`${fn}< HTTP error, status = ${response.status}`);
131+
throw new Error(`HTTP error, status = ${response.status}`);
132+
}
133+
console.debug(`${fn}< receiving response as readable stream`);
134+
return responseToReadable(response);
135+
});
118136
}
119137
}
120138

0 commit comments

Comments
 (0)