# 🚀 Task Spawning for Map Steps #213

jumski · 2025-09-12T11:04:28Z

Summary

Implements task spawning functionality for map steps, enabling parallel task creation based on array input length. This PR enhances start_ready_steps() to generate N tasks for map steps while maintaining backward compatibility with single steps.

What's New

✅ Dynamic Task Spawning: Map steps spawn N tasks based on initial_tasks count
✅ Empty Array Handling: Zero-length arrays auto-complete with [] output
✅ Batch Operations: Efficient task creation using generate_series()
✅ Task Indexing: Proper task_index assignment (0 to N-1) for ordered execution

Implementation Details

Enhanced start_ready_steps() to handle both single and map step types
Added CROSS JOIN LATERAL generate_series() for efficient batch task creation
Special handling for empty arrays - direct transition to completed state
Maintains existing behavior for single steps (1 task with index 0)

Testing

Comprehensive test coverage for task spawning scenarios
Tests for empty array auto-completion
Validation of task_index values and message queue entries
Performance testing script for large arrays

Next Steps

This completes the task spawning phase. Next PRs will implement:

Array element extraction in start_tasks()
Dependent map support in complete_task()
Output aggregation in maybe_complete_run()

Part of the map infrastructure implementation (PR #4 of 8)

changeset-bot · 2025-09-12T11:04:33Z

⚠️ No Changeset found

Latest commit: 407d48d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

coderabbitai · 2025-09-12T11:04:35Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

✨ Finishing touches

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch 09-12-task-spawning

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

jumski · 2025-09-12T11:04:45Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

merge:queue - adds this PR to the back of the merge queue
hotfix:queue - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

nx-cloud · 2025-09-12T11:11:08Z

🤖 Nx Cloud AI Fix Eligible

An automatically generated fix could have helped fix failing tasks for this run, but Self-healing CI is disabled for this workspace. Visit workspace settings to enable it and get automatic fixes in future runs.

_{To disable these notifications, a workspace admin can disable them in workspace settings.}

View your CI Pipeline Execution ↗ for commit 407d48d

Command	Status	Duration	Result
`nx affected -t lint typecheck test --parallel -...`	❌ Failed	5m 56s	View ↗

☁️ Nx Cloud last updated this comment at 2025-10-06 15:41:54 UTC

pkgs/core/schemas/0100_function_start_ready_steps.sql

github-actions · 2025-10-05T19:28:49Z

🔍 Preview Deployment: Website

✅ Deployment successful!

🔗 Preview URL: https://pr-213.pgflow.pages.dev

📝 Details:

Branch: 09-12-task-spawning
Commit: 1d1d70ce585d59fd2789aa5cb91de7e772b173c0
View Logs

_Last updated: _

github-actions · 2025-10-05T19:30:30Z

🔍 Preview Deployment: Playground

✅ Deployment successful!

🔗 Preview URL: https://pr-213--pgflow-demo.netlify.app

📝 Details:

Branch: 09-12-task-spawning
Commit: 1d1d70ce585d59fd2789aa5cb91de7e772b173c0
View Logs

_Last updated: _

…low management - Introduced collect_perf_data.sh for performance testing of large array handling - Updated start_ready_steps function to handle empty map steps and initialize task states - Added migration script to modify start_ready_steps for correct task spawning - Created tests for map step message queueing, delayed message scheduling, and task spawning - Ensured proper handling of initial_tasks, task indices, and step status transitions - Included tests for both map and single steps to verify correct task creation and message dispatching

graphite-app · 2025-10-07T08:28:18Z

Merge activity

Oct 7, 8:28 AM UTC: jumski added this pull request to the Graphite merge queue.
Oct 7, 8:29 AM UTC: CI is running for this pull request on a draft pull request (#232) due to your merge queue CI optimization settings.
Oct 7, 8:29 AM UTC: Merged by the Graphite merge queue via draft PR: #232.

## Summary Implements task spawning functionality for map steps, enabling parallel task creation based on array input length. This PR enhances `start_ready_steps()` to generate N tasks for map steps while maintaining backward compatibility with single steps. ## What's New - ✅ **Dynamic Task Spawning**: Map steps spawn N tasks based on `initial_tasks` count - ✅ **Empty Array Handling**: Zero-length arrays auto-complete with `[]` output - ✅ **Batch Operations**: Efficient task creation using `generate_series()` - ✅ **Task Indexing**: Proper task_index assignment (0 to N-1) for ordered execution ## Implementation Details - Enhanced `start_ready_steps()` to handle both single and map step types - Added `CROSS JOIN LATERAL generate_series()` for efficient batch task creation - Special handling for empty arrays - direct transition to completed state - Maintains existing behavior for single steps (1 task with index 0) ## Testing - Comprehensive test coverage for task spawning scenarios - Tests for empty array auto-completion - Validation of task_index values and message queue entries - Performance testing script for large arrays ## Next Steps This completes the task spawning phase. Next PRs will implement: - Array element extraction in `start_tasks()` - Dependent map support in `complete_task()` - Output aggregation in `maybe_complete_run()` Part of the map infrastructure implementation (PR #4 of 8)

jumski mentioned this pull request Sep 12, 2025

feat: add map and array steps for parallel processing #207

Closed

This was referenced Sep 12, 2025

feat: add map step type in sql #208

Closed

feat: add root map step support #209

Closed

jumski changed the title ~~feat: Add scripts and tests for task spawning and step execution in flow management~~ # 🚀 Task Spawning for Map Steps Sep 12, 2025

jumski marked this pull request as ready for review September 12, 2025 12:44

jumski force-pushed the 09-12-task-spawning branch 2 times, most recently from 9bd624e to 66e3c3b Compare September 12, 2025 13:24

jumski temporarily deployed to preview September 12, 2025 13:31 — with GitHub Actions Inactive

jumski force-pushed the 09-12-task-spawning branch 2 times, most recently from ce696e8 to 850de93 Compare September 12, 2025 15:09

jumski temporarily deployed to preview September 12, 2025 15:16 — with GitHub Actions Inactive

jumski force-pushed the 09-12-task-spawning branch from 850de93 to 05e2388 Compare September 12, 2025 21:37

graphite-app bot reviewed Sep 12, 2025

View reviewed changes

pkgs/core/schemas/0100_function_start_ready_steps.sql Show resolved Hide resolved

jumski temporarily deployed to preview September 12, 2025 21:44 — with GitHub Actions Inactive

jumski mentioned this pull request Sep 15, 2025

feat: implement cascade completion of taskless steps #214

Closed

jumski force-pushed the 09-12-task-spawning branch from 05e2388 to f462cf0 Compare September 15, 2025 17:54

jumski force-pushed the 9-11-root-map-support branch from c0fc2a1 to 2e5052c Compare September 15, 2025 17:54

jumski temporarily deployed to preview September 15, 2025 18:08 — with GitHub Actions Inactive

jumski force-pushed the 9-11-root-map-support branch from 2e5052c to b813159 Compare October 5, 2025 18:59

jumski force-pushed the 09-12-task-spawning branch from f462cf0 to fc686ff Compare October 5, 2025 18:59

jumski mentioned this pull request Oct 5, 2025

docs: improve documentation and READMEs to explain map/array steps #228

Closed

jumski force-pushed the 9-11-root-map-support branch from b813159 to f02b6ae Compare October 5, 2025 19:20

jumski force-pushed the 09-12-task-spawning branch from fc686ff to cc142fb Compare October 5, 2025 19:20

jumski temporarily deployed to preview October 5, 2025 19:28 — with GitHub Actions Inactive

jumski mentioned this pull request Oct 6, 2025

fix: remove temp migrations and prepare stack to be merged #230

Closed

jumski force-pushed the 9-11-root-map-support branch from f02b6ae to 025ae50 Compare October 6, 2025 15:25

jumski force-pushed the 09-12-task-spawning branch from cc142fb to 407d48d Compare October 6, 2025 15:25

jumski mentioned this pull request Oct 6, 2025

fix: typecheck tests using strict tsc mode #231

Closed

graphite-app bot closed this Oct 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

# 🚀 Task Spawning for Map Steps #213

# 🚀 Task Spawning for Map Steps #213

Uh oh!

jumski commented Sep 12, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Sep 12, 2025 •

edited

Loading

Uh oh!

coderabbitai bot commented Sep 12, 2025 •

edited

Loading

Review skipped

Other AI code review bot(s) detected

Uh oh!

jumski commented Sep 12, 2025 •

edited

Loading

Uh oh!

nx-cloud bot commented Sep 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

github-actions bot commented Oct 5, 2025

Uh oh!

github-actions bot commented Oct 5, 2025

Uh oh!

graphite-app bot commented Oct 7, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

# 🚀 Task Spawning for Map Steps #213

# 🚀 Task Spawning for Map Steps #213

Uh oh!

Conversation

jumski commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's New

Implementation Details

Testing

Next Steps

Uh oh!

changeset-bot bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

coderabbitai bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Other AI code review bot(s) detected

Uh oh!

jumski commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

nx-cloud bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 Nx Cloud AI Fix Eligible

Uh oh!

Uh oh!

github-actions bot commented Oct 5, 2025

🔍 Preview Deployment: Website

Uh oh!

github-actions bot commented Oct 5, 2025

🔍 Preview Deployment: Playground

Uh oh!

graphite-app bot commented Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jumski commented Sep 12, 2025 •

edited

Loading

changeset-bot bot commented Sep 12, 2025 •

edited

Loading

coderabbitai bot commented Sep 12, 2025 •

edited

Loading

jumski commented Sep 12, 2025 •

edited

Loading

nx-cloud bot commented Sep 12, 2025 •

edited

Loading

graphite-app bot commented Oct 7, 2025 •

edited

Loading