feat: enhance start_tasks with conditional input logic and add array handling migration #216

jumski · 2025-09-16T19:52:52Z

Enhanced Map Step Input Processing

Updated start_tasks function to build step input conditionally based on step type
Implemented logic for root and dependent map steps to extract array elements
Added a new migration script to handle array elements in start_tasks
Included comprehensive tests for dependent map element extraction, large array processing, mixed JSON types, and nested arrays
Improved handling of array-based tasks and input construction for various step configurations

changeset-bot · 2025-09-16T19:52:56Z

⚠️ No Changeset found

Latest commit: e87df3c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

coderabbitai · 2025-09-16T19:52:59Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

✨ Finishing touches

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch 09-16-handle-arrays-in-start-tasks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

jumski · 2025-09-16T19:53:15Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

merge:queue - adds this PR to the back of the merge queue
hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

nx-cloud · 2025-09-16T19:59:52Z

View your CI Pipeline Execution ↗ for commit e87df3c

Command	Status	Duration	Result
`nx run-many -t build --projects client,dsl --co...`	✅ Succeeded	4s	View ↗
`nx affected -t build --configuration=production...`	✅ Succeeded	3s	View ↗
`nx affected -t lint typecheck test --parallel -...`	✅ Succeeded	6m 5s	View ↗

☁️ Nx Cloud last updated this comment at 2025-10-06 15:33:40 UTC

pkgs/core/supabase/migrations/20250916201303_pgflow_temp_handle_arrays_in_start_tasks.sql

graphite-app · 2025-09-17T04:41:15Z

pkgs/core/supabase/tests/start_tasks/map_performance.test.sql

+  (select flow_start_time_ms <= task_creation_time_ms * 1.5
+   from map_performance


The variable comparison appears to be reversed. The test is checking if array processing scales linearly, but the variables in the comparison don't match their content:

select ok( (select flow_start_time_ms <= task_creation_time_ms * 1.5 from map_performance where array_size = -1), 'Array processing MUST scale linearly (time_ratio <= size_ratio * 1.5)' );

In the sentinel row (where array_size = -1), flow_start_time_ms contains the time ratio and task_creation_time_ms contains the size ratio. The comparison should be:

select ok( (select time_ratio <= size_ratio * 1.5 from map_performance where array_size = -1), 'Array processing MUST scale linearly (time_ratio <= size_ratio * 1.5)' );

Or using the existing column names:

select ok( (select flow_start_time_ms <= task_creation_time_ms * 1.5 from map_performance where array_size = -1), 'Array processing MUST scale linearly (time_ratio <= size_ratio * 1.5)' );

Suggested change

(select flow_start_time_ms <= task_creation_time_ms * 1.5

from map_performance

(select time_ratio <= size_ratio * 1.5

from map_performance

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-09-17T04:51:50Z

pkgs/core/schemas/0120_function_start_tasks.sql

+            (SELECT jsonb_array_element(value, st.task_index)
+            FROM jsonb_each(dep_out.deps_output)
+            LIMIT 1)


Critical bug: The subquery uses LIMIT 1 without ORDER BY, making the result non-deterministic when a map step has multiple dependencies. While the comment states 'Map steps have exactly 1 dependency (enforced by add_step)', this creates a race condition if that constraint is ever violated or if there are concurrent modifications. The query should either add ORDER BY for deterministic results or add a runtime check to ensure exactly one dependency exists. This could cause map tasks to receive elements from random dependencies in multi-dependency scenarios.

Suggested change

(SELECT jsonb_array_element(value, st.task_index)

FROM jsonb_each(dep_out.deps_output)

LIMIT 1)

(SELECT jsonb_array_element(value, st.task_index)

FROM jsonb_each(dep_out.deps_output)

WHERE (SELECT COUNT(*) FROM jsonb_each(dep_out.deps_output)) = 1

LIMIT 1)

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-09-17T06:34:20Z

pkgs/core/schemas/0120_function_start_tasks.sql

+            -- Extract the element at task_index from the run's input array.
+            -- Note: If run input is not an array, this will return NULL
+            -- and the flow will fail (validated in start_flow).
+            jsonb_array_element(r.input, st.task_index)


Array bounds vulnerability - jsonb_array_element(r.input, st.task_index) will return NULL if task_index is out of bounds, but this NULL is passed directly as task input without validation. If there's a race condition or bug in task creation that results in task_index >= array_length, tasks will receive NULL input and likely fail at runtime. Add bounds checking or handle NULL case explicitly.

Suggested change

jsonb_array_element(r.input, st.task_index)

CASE

WHEN jsonb_array_length(r.input) > st.task_index THEN jsonb_array_element(r.input, st.task_index)

ELSE jsonb_build_object('error', 'Task index out of bounds')

END

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

…handling migration - Updated start_tasks function to build step input conditionally based on step type - Implemented logic for root and dependent map steps to extract array elements - Added a new migration script to handle array elements in start_tasks - Included comprehensive tests for dependent map element extraction, large array processing, mixed JSON types, and nested arrays - Improved handling of array-based tasks and input construction for various step configurations

github-actions · 2025-10-06T15:33:33Z

🔍 Preview Deployment: Website

✅ Deployment successful!

🔗 Preview URL: https://pr-216.pgflow.pages.dev

📝 Details:

Branch: 09-16-handle-arrays-in-start-tasks
Commit: 2ee6c15e2700a08b299552a1f44944b53cfbbbb7
View Logs

_Last updated: _

github-actions · 2025-10-06T15:35:19Z

🔍 Preview Deployment: Playground

✅ Deployment successful!

🔗 Preview URL: https://pr-216--pgflow-demo.netlify.app

📝 Details:

Branch: 09-16-handle-arrays-in-start-tasks
Commit: 2ee6c15e2700a08b299552a1f44944b53cfbbbb7
View Logs

_Last updated: _

graphite-app · 2025-10-07T08:28:20Z

Merge activity

Oct 7, 8:28 AM UTC: jumski added this pull request to the Graphite merge queue.
Oct 7, 8:29 AM UTC: CI is running for this pull request on a draft pull request (#232) due to your merge queue CI optimization settings.
Oct 7, 8:29 AM UTC: Merged by the Graphite merge queue via draft PR: #232.

…handling migration (#216) # Enhanced Map Step Input Processing - Updated start_tasks function to build step input conditionally based on step type - Implemented logic for root and dependent map steps to extract array elements - Added a new migration script to handle array elements in start_tasks - Included comprehensive tests for dependent map element extraction, large array processing, mixed JSON types, and nested arrays - Improved handling of array-based tasks and input construction for various step configurations

jumski marked this pull request as ready for review September 16, 2025 19:53

jumski temporarily deployed to preview September 16, 2025 20:00 — with GitHub Actions Inactive

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from 0a5205c to 63a2762 Compare September 16, 2025 20:14

graphite-app bot reviewed Sep 16, 2025

View reviewed changes

pkgs/core/supabase/migrations/20250916201303_pgflow_temp_handle_arrays_in_start_tasks.sql Outdated Show resolved Hide resolved

jumski temporarily deployed to preview September 16, 2025 20:21 — with GitHub Actions Inactive

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch 3 times, most recently from ce60c8e to a462a29 Compare September 17, 2025 04:39

graphite-app bot reviewed Sep 17, 2025

View reviewed changes

jumski temporarily deployed to preview September 17, 2025 04:46 — with GitHub Actions Inactive

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from a462a29 to 419de1b Compare September 17, 2025 04:50

graphite-app bot reviewed Sep 17, 2025

View reviewed changes

jumski temporarily deployed to preview September 17, 2025 04:57 — with GitHub Actions Inactive

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from 419de1b to 781e413 Compare September 17, 2025 06:31

graphite-app bot reviewed Sep 17, 2025

View reviewed changes

jumski temporarily deployed to preview September 17, 2025 06:38 — with GitHub Actions Inactive

jumski force-pushed the 09-16-make-initial-tasks-nullable branch from 2b71eb5 to dd1dde0 Compare October 5, 2025 18:59

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from 781e413 to 53ca34b Compare October 5, 2025 18:59

jumski mentioned this pull request Oct 5, 2025

docs: improve documentation and READMEs to explain map/array steps #228

Closed

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from 53ca34b to 99d14b5 Compare October 5, 2025 19:20

jumski force-pushed the 09-16-make-initial-tasks-nullable branch from dd1dde0 to 2adddf3 Compare October 5, 2025 19:20

jumski temporarily deployed to preview October 5, 2025 19:34 — with GitHub Actions Inactive

jumski mentioned this pull request Oct 6, 2025

fix: remove temp migrations and prepare stack to be merged #230

Closed

jumski force-pushed the 09-16-handle-arrays-in-start-tasks branch from 99d14b5 to e87df3c Compare October 6, 2025 15:25

jumski force-pushed the 09-16-make-initial-tasks-nullable branch from 2adddf3 to 4665802 Compare October 6, 2025 15:25

jumski temporarily deployed to preview October 6, 2025 15:32 — with GitHub Actions Inactive

jumski mentioned this pull request Oct 6, 2025

fix: typecheck tests using strict tsc mode #231

Closed

graphite-app bot closed this Oct 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: enhance start_tasks with conditional input logic and add array handling migration #216

feat: enhance start_tasks with conditional input logic and add array handling migration #216

Uh oh!

jumski commented Sep 16, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

coderabbitai bot commented Sep 16, 2025 •

edited

Loading

Review skipped

Other AI code review bot(s) detected

Uh oh!

jumski commented Sep 16, 2025 •

edited

Loading

Uh oh!

nx-cloud bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

graphite-app bot Sep 17, 2025

Uh oh!

graphite-app bot Sep 17, 2025

Uh oh!

graphite-app bot Sep 17, 2025

Uh oh!

github-actions bot commented Oct 6, 2025

Uh oh!

github-actions bot commented Oct 6, 2025

Uh oh!

graphite-app bot commented Oct 7, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		(select flow_start_time_ms <= task_creation_time_ms * 1.5
		from map_performance

-            jsonb_array_element(r.input, st.task_index)
+            CASE
+                WHEN jsonb_array_length(r.input) > st.task_index THEN jsonb_array_element(r.input, st.task_index)
+                ELSE jsonb_build_object('error', 'Task index out of bounds')
+            END

feat: enhance start_tasks with conditional input logic and add array handling migration #216

feat: enhance start_tasks with conditional input logic and add array handling migration #216

Uh oh!

Conversation

jumski commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Enhanced Map Step Input Processing

Uh oh!

changeset-bot bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

coderabbitai bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Other AI code review bot(s) detected

Uh oh!

jumski commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

nx-cloud bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

graphite-app bot Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 6, 2025

🔍 Preview Deployment: Website

Uh oh!

github-actions bot commented Oct 6, 2025

🔍 Preview Deployment: Playground

Uh oh!

graphite-app bot commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jumski commented Sep 16, 2025 •

edited

Loading

changeset-bot bot commented Sep 16, 2025 •

edited

Loading

coderabbitai bot commented Sep 16, 2025 •

edited

Loading

jumski commented Sep 16, 2025 •

edited

Loading

nx-cloud bot commented Sep 16, 2025 •

edited

Loading

graphite-app bot commented Oct 7, 2025 •

edited

Loading