Skip to content

Commit 05bc060

Browse files
committed
update amazon ecommerce example
1 parent a6643a7 commit 05bc060

File tree

1 file changed

+31
-23
lines changed

1 file changed

+31
-23
lines changed

examples/ecommerce.mdx

+31-23
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,18 @@ A breakdown of the script is below
6666
The first step is to define all the types
6767

6868
```ts
69+
interface Chunk {
70+
chunk_html: string;
71+
link: string;
72+
tracking_id: string;
73+
tag_set: string[];
74+
metadata: {
75+
[key: string]: string;
76+
};
77+
time_stamp: string;
78+
upsert_by_tracking_id: boolean;
79+
}
80+
6981
interface LanguageTaggedValue {
7082
language_tag: string;
7183
value: string;
@@ -150,7 +162,7 @@ if (imageHashMap == null) {
150162

151163
### Processing a Singular Chunk
152164

153-
For each row of json we construct a `CreateChunkData` object, each of which can be pushed into Trieve separately or in bulk. We use the `imageHashMap` object constructed from before to get the `image_url` for each.
165+
For each row of json we construct a `Chunk` object, each of which can be pushed into Trieve separately or in bulk. We use the `imageHashMap` object constructed from before to get the `image_url` for each.
154166

155167
```ts
156168
function processLine(line: string) {
@@ -206,7 +218,7 @@ function processLine(line: string) {
206218
metadata.image_url = image_url;
207219
metadata.price = price;
208220

209-
const chunkData: CreateChunkData = {
221+
const chunkData: Chunk = {
210222
chunk_html: searchableString.trim(),
211223
link: `https://${item.domain_name}/dp/${item.item_id}`,
212224
tracking_id: item.item_id,
@@ -219,25 +231,9 @@ function processLine(line: string) {
219231
}
220232
```
221233

222-
223-
### Instantiating the Trieve Client
224-
225-
We have to first instantiate the client with our credentials before we can use it:
226-
227-
```ts
228-
const trieveApiKey = Bun.env.TRIEVE_API_KEY ?? "";
229-
const trieveDatasetId = Bun.env.TRIEVE_DATASET_ID ?? "";
230-
231-
const trieveApiConfig = new Configuration({
232-
apiKey: trieveApiKey,
233-
basePath: "https://api.trieve.ai",
234-
});
235-
236-
```
237-
238234
### Iterating and sending chunks to Trieve
239235

240-
The last step is we iterate through all the json files, process each line into a CreateChunkData object. We use `chukApi.createChunk` to create chunks, in this case we are sending chunks in a batch size of 120 which is the max for this route.
236+
The last step is we iterate through all the json files, process each line into a Chunk object. We use the `/chunk` endpoint to create chunks, in this case we are sending chunks in a batch size of 120 which is the max for this route.
241237

242238

243239
```ts
@@ -254,19 +250,29 @@ for (const file of files) {
254250
crlfDelay: Infinity,
255251
});
256252

257-
const items: CreateChunkData = [];
253+
const items: Chunk[] = [];
258254

259255
for await (const line of rl) {
260256
try {
261-
const chunkData: CreateChunkData = processLine(line);
262-
items.push(chunkData);
257+
const chunkData: Chunk = processLine(line);
258+
const options = {
259+
method: "POST",
260+
headers: {
261+
"TR-Dataset": trieveDatasetId,
262+
Authorization: trieveApiKey,
263+
"Content-Type": "application/json",
264+
},
265+
body: JSON.stringify(chunk),
266+
};
267+
268+
await fetch("https://api.trieve.ai/api/chunk", options);
263269
} catch (error) {
264270
console.error("Error parsing JSON from line:", error);
265271
}
266272
}
267273

268274
const batchSize = 120;
269-
const chunkedItems: CreateChunkData[] = [];
275+
const chunkedItems: Chunk[][] = [];
270276
for (let i = 0; i < items.length; i += batchSize) {
271277
const chunk = items.slice(i, i + batchSize);
272278
chunkedItems.push(chunk);
@@ -284,3 +290,5 @@ for (const file of files) {
284290
}
285291
```
286292

293+
Checkout the api reference for the [upload API](/api-reference/chunk/create-or-upsert-chunk-or-chunks) to view all of the parameters that can be passed in with the chunk.
294+

0 commit comments

Comments
 (0)