feat(javascript): add `replaceAllObjectsWithTransformation` #5008

shortcuts · 2025-06-16T22:03:24Z

🧭 What and Why

🎟 JIRA Ticket: https://algolia.atlassian.net/browse/DI-3894

Changes included:

this adds the helper to replace all of the content of an index by leveraging the transformation pipeline, i've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

algolia-bot · 2025-06-17T07:36:11Z

✔️ Code generated!

Name	Link
🪓 Triggered by	`bb1df0d09b79b91cdd1cf6820c26df3d6f4f785e`
🍃 Generated commit	`1f8550f014ef27e1b4148fefc4eafa0b737e4e92`
🌲 Generated branch	`generated/feat/javascript-replaceAllObjectsWithTransformation`

📊 Benchmark results

Benchmarks performed on the method using a mock server, the results might not reflect the real-world performance.

Language	Req/s
go	1724
javascript	1586
php	1504
csharp	1236
java	1108
python	1063
ruby	780
swift	736
scala	23

shortcuts · 2025-06-17T10:03:09Z

've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

ah actually that would mean breaking the signature, I need to check if there is some usage first

…rmation

shortcuts · 2025-06-17T14:08:38Z

've inlined the chunkedPush logic but actually could be reused in the saveObjects and partialUpdateObjects methods as well, lmk

ah actually that would mean breaking the signature, I need to check if there is some usage first

breaking all of them at once in #5011

millotp

so nice, gg

scripts/cts/testServer/replaceAllObjectsWithTransformation.ts

specs/search/helpers/chunkedPush.yml

templates/javascript/clients/algoliasearch/builds/definition.mustache

millotp · 2025-06-17T14:18:58Z

templates/javascript/clients/algoliasearch/builds/definition.mustache

+   * @param chunkedPush.indexName - The `indexName` to replace `objects` in.
+   * @param chunkedPush.objects - The array of `objects` to store in the given Algolia `indexName`.
+   * @param chunkedPush.action - The `batch` `action` to perform on the given array of `objects`, defaults to `addObject`.
+   * @param chunkedPush.waitForTasks - Whether or not we should wait until every `batch` tasks has been processed, this operation may slow the total execution time of this method but is more reliable.


does waitForTasks make sense here ? should it be waitForPush ?

I copied chunkedBatch so it's usable the same way but since it's internal I can rename it if you want

millotp · 2025-06-17T14:21:33Z

templates/javascript/clients/algoliasearch/builds/definition.mustache

+      const responses: Array<WatchResponse> = [];
+
+      const objectEntries = objects.entries();
+      for (const [i, obj] of objectEntries) {


what about https://stackoverflow.com/a/8495740 ? I feel like it's more efficient and more readable

I've actually just copied chunkedBatch 🥶

I don't think it's IE compatible as well? I can't remember what version we should support but I think it's 11?

slice is available everywhere

i'll update both implems in a follow up pr!

millotp · 2025-06-17T14:23:36Z

templates/javascript/clients/algoliasearch/builds/definition.mustache

+      if (waitForTasks) {
+        for (const resp of responses) {
+          if (resp.eventID === undefined || !resp.eventID) {
+            throw new Error('received unexpected response from the push endpoint, eventID must not be undefined');


IMO this error checking should be in loop above, to early exit.
Also in that case there should be an error message in the response

I actually had to put it here to satisfy the type checking (same for Go and other languages), I can duplicate it, but this one is required

templates/javascript/clients/algoliasearch/builds/definition.mustache

…rmation

templates/javascript/clients/algoliasearch/builds/definition.mustache

millotp

test

…generated) [skip ci] Co-authored-by: Clément Vannicatte <[email protected]>

algolia/api-clients-automation#5008 Co-authored-by: algolia-bot <[email protected]> Co-authored-by: Clément Vannicatte <[email protected]>

shortcuts self-assigned this Jun 16, 2025

shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from 142a5c0 to bc5ed6c Compare June 17, 2025 07:35

feat(javascript): add replaceAllObjectsWithTransformation

405c5e4

shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from bc5ed6c to 405c5e4 Compare June 17, 2025 07:36

chore: playground

4be66cf

shortcuts marked this pull request as ready for review June 17, 2025 09:35

shortcuts requested a review from a team as a code owner June 17, 2025 09:35

shortcuts requested review from morganleroi and millotp June 17, 2025 09:35

shortcuts force-pushed the feat/javascript-replaceAllObjectsWithTransformation branch from 0ee2dc0 to 4be66cf Compare June 17, 2025 11:35

Merge branch 'main' into feat/javascript-replaceAllObjectsWithTransfo…

c4c8800

…rmation

millotp reviewed Jun 17, 2025

View reviewed changes

chore: review

e86157a

shortcuts requested a review from millotp June 17, 2025 15:08

Merge branch 'main' into feat/javascript-replaceAllObjectsWithTransfo…

d8f3b85

…rmation

shortcuts enabled auto-merge (squash) June 17, 2025 15:49

millotp reviewed Jun 18, 2025

View reviewed changes

templates/javascript/clients/algoliasearch/builds/definition.mustache Outdated Show resolved Hide resolved

chore: review

7b1a881

shortcuts requested a review from millotp June 18, 2025 13:01

millotp reviewed Jun 18, 2025

View reviewed changes

shortcuts requested a review from millotp June 18, 2025 14:06

millotp approved these changes Jun 18, 2025

View reviewed changes

shortcuts merged commit a2ebbf4 into main Jun 18, 2025
28 checks passed

shortcuts deleted the feat/javascript-replaceAllObjectsWithTransformation branch June 18, 2025 14:07

algolia-bot added a commit that referenced this pull request Jun 18, 2025

feat(javascript): add replaceAllObjectsWithTransformation (#5008) (…

0b5cf44

…generated) [skip ci] Co-authored-by: Clément Vannicatte <[email protected]>

feat(javascript): add replaceAllObjectsWithTransformation #5008

feat(javascript): add replaceAllObjectsWithTransformation #5008

Uh oh!

Conversation

shortcuts commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧭 What and Why

Changes included:

Uh oh!

algolia-bot commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Code generated!

Uh oh!

shortcuts commented Jun 17, 2025

Uh oh!

shortcuts commented Jun 17, 2025

Uh oh!

millotp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

millotp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

feat(javascript): add `replaceAllObjectsWithTransformation` #5008

feat(javascript): add `replaceAllObjectsWithTransformation` #5008

shortcuts commented Jun 16, 2025 •

edited

Loading

algolia-bot commented Jun 17, 2025 •

edited

Loading