Skip to content

ci(fork-release-smoke): wait out boot1 teardown before seeding the scratch DB - #333

Open
claudegoogl-sudo wants to merge 2 commits into
masterfrom
ci/fork-release-smoke-seed-fix
Open

ci(fork-release-smoke): wait out boot1 teardown before seeding the scratch DB#333
claudegoogl-sudo wants to merge 2 commits into
masterfrom
ci/fork-release-smoke-seed-fix

Conversation

@claudegoogl-sudo

@claudegoogl-sudo claudegoogl-sudo commented Sep 8, 2026

Copy link
Copy Markdown
Owner

What Changed

  • Commit 2 (seed the paperclip database, not postgres): the app creates and migrates the embedded database named paperclip (proven by boot1 logs); the seed connected to postgres where no app tables exist.
  • fork-release-smoke.yml seed step: after boot1's teardown, poll pg_ctl -D <db> status until the old embedded postmaster reports not-running, clear the stale postmaster.pid/postmaster.opts, then start the seed cluster with pg_ctl -w -t 60 start (fail with the pg-seed.log tail on error).
  • Upload sandbox/pg-seed.log alongside the other smoke artifacts.

Why Needed

  • Run 34239890154 failed in the seed phase: preflight's teardown stops the scratch Postgres with SIGTERM and returns while the drain is still in flight (boot.log shows the SIGTERM at 14:48:02.2Z); the seed's pg_ctl start fired at 14:48:02.25Z against a live data dir → pg_ctl: another server might be runningcould not start server.
  • Install + boot1 phases passed on that run (checksums/closure/exports PASS, 317 packages from the release URL graph, GET / 200).

Verification

  • Re-dispatch of the smoke workflow from this branch: GREEN — run 34241843769 (install from published URLs → boot1 migrations 0000..0249 → seeded stranded wake → boot2 → sweep resolved it, startup stranded deferred-wake sweep promoted or resolved wakes {scanned:1, resolved:1}).
  • Artifacts from the failed run (install/boot1 evidence): https://github.com/claudegoogl-sudo/paperclip/actions/runs/34239890154

Risk Assessment

  • Workflow-only change; release tree and tag untouched. Failure mode shifts from race to deterministic wait (60 s cap + explicit failure with logs).

Rollback Plan

  • Revert this commit; the smoke workflow's previous behavior returns.

Migration/Compatibility Impact

  • None (CI workflow only).

Model Used

  • Paperclip prime_local adapter (provider-routed model; session does not expose exact backing model id)

Coder and others added 2 commits September 8, 2026 14:56
…ratch DB

The smoke failed on run 34239890154: preflight's teardown stops the
embedded Postgres with SIGTERM and returns while the drain is still in
flight, so pg_ctl start hit a live data dir ("another server might be
running") 8s later. Wait for pg_ctl status to report not-running, clear
the stale pid/opts files, then start with -w -t 60. Also upload
sandbox/pg-seed.log with the other smoke artifacts for diagnosis.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Boot1 log proves the app creates and migrates the embedded database
named 'paperclip' (248 migrations 0000..0249); the seed connected to the
default 'postgres' database where no app tables exist.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant