Skip to content

Fix CANCEL_IN_PROGRESS replacement during retry backoff - #4903

Draft
Scalahansolo wants to merge 1 commit into
hatchet-dev:mainfrom
Scalahansolo:fix/workflow-replacement-retry-backoff
Draft

Scalahansolo wants to merge 1 commit into
hatchet-dev:mainfrom
Scalahansolo:fix/workflow-replacement-retry-backoff

Conversation

@Scalahansolo

Copy link
Copy Markdown

Keep filled CANCEL_IN_PROGRESS slots attached to the current retry during backoff, preserving replacement ownership and ordering. Remove pending retries atomically when their retained slots are cancelled, and reuse retained slots when retries become eligible. Includes a reversible database migration.

Adds database regressions for workflow and single-task declarations, future and already-due retries, cancellation before asynchronous delivery, and ordinary retry admission on another key. All four cases fail without the change; the concurrency integration suite and concurrency unit tests pass with it. Migration rollback was also exercised with a pending retained retry.

Draft: the standalone first-step and durable-wait → later-step reproductions pass on v0.105.16 with the SQL fix backported. End-to-end validation on current main is blocked by a separate first-task dispatch failure that also occurs with this migration rolled back. Concurrent retry-admission interleavings and chained-policy retry coverage still need validation before this is ready.

Fixes #4900

@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the Hatchet Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added the engine Related to the core Hatchet engine label Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine Related to the core Hatchet engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Workflow CANCEL_IN_PROGRESS leaves superseded retry-backoff work runnable

1 participant