Skip to content

feat(release): publish nightly and stable releases through npm trusted publishing - #3380

Merged
thymikee merged 5 commits into
fix/nightly-ready-versionsfrom
feat/release-workflow
Oct 11, 2026
Merged

thymikee merged 5 commits into
fix/nightly-ready-versionsfrom
feat/release-workflow

Conversation

@thymikee

@thymikee thymikee commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Summary

.github/workflows/release.yml now owns every npm publish, through npm trusted publishing. No npm token or one-time password is involved.

  • Nightly: the schedule publishes X.Y.Z-nightly.<date>.<run> under the nightly dist-tag when main has changed, its CI passed, and no nightly shipped that UTC day. The workflow tags the commit nightly/v…, and the npm dist-tag serves as the durable record.
  • Stable: a repository admin publishes a GitHub release with a new vX.Y.Z tag targeting main. Only admins may create v* tags (chore(release): retire local npm publishing #3381 documents the ruleset). The tag's run waits up to 20 minutes for that commit's CI, then for approval in the release environment. It then:
    • publishes to latest
    • attaches the runner and helper assets to the release
    • publishes the MCP Registry entry
    • commits main's next -dev version. That commit is pushed with GITHUB_TOKEN, so it gets no CI run of its own; it counts as covered by its parent's CI because it only rewrites "version" fields.
  • Dry run: a PR that touches release tooling builds and verifies every package.
  • Token boundary: only publish (environment npm-publish) and the MCP Registry call can mint an id-token. test/ci/release-workflow.test.ts pins that, and pins the exact approval condition.

The layer above (#3388) refuses to publish until GitHub enforces these protections. 12 files.

Validation

At db9484fff6:

  • Plan, version, workflow, and package-script tests: 27/27 pass.
  • Fallow, lint, and format pass.
  • The version-only detection returned true for the real chore: mark 0.21.24 as released commit and false for a code commit.
  • Read-only plans against the live repo: the nightly skips while CI is queued, and the stable plan resolves v0.21.24.
  • Against a stub npm, the publisher published on E404, refused on a network error, and skipped versions already on npm.
  • The macOS release:prepare dry run passed at 9c6ef86.

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-3380/

Built to branch gh-pages at 2026-10-11 07:21 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.16 MB 5.16 MB 0 B
Package (unpacked) 5.16 MB 5.16 MB 0 B
Package (download) 1.55 MB 1.55 MB -3 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.3 ms 27.0 ms +0.7 ms
CLI --help 83.8 ms 84.7 ms +0.9 ms

@thymikee

Copy link
Copy Markdown
Member Author

The release checks are green at 61567a6 (33 checks, none failing, including the macOS release:prepare dry run), but one publishing gap needs a fix before merge. I did not run the new tests locally. I could not exercise the nightly, promote or stable route either, because the OIDC exchange, the deployment policy and the approval gate only show up on main after setup.

npm trust is pinned to release.yml plus the npm-publish environment, so any run of that workflow that reaches the publish job can publish. release.yml runs on pull_request and on workflow_dispatch from any ref, and each run uses that ref's own copy of the workflow and release-plan.mjs. A same-repo branch can edit the if: or the dist tag and publish --tag latest from unreviewed code. The refs/heads/main check at release-plan.mjs:760 runs inside that same copy, so it does not protect anything. The stable approval at line 106 (environment: release) only works if that environment has required reviewers. A read-only gh api query shows that neither release nor npm-publish exists today. GitHub creates a missing environment with no protection, and the setup steps in #3381 (CONTRIBUTING, lines 85-140) mention neither. Once npm trust is granted as documented, anyone with push access could publish any version to latest. If this merges before the manual setup, every stable tag publishes with no approval. The rule to satisfy: every job that can mint an npm-trusted OIDC token runs only for refs/heads/main or a vX.Y.Z tag, GitHub enforces that and not release-plan.mjs, and stable publishing needs a human approval that GitHub enforces. Please apply and document three settings: the npm-publish deployment policy allows only main and the release tag pattern; the release environment has required reviewers with self-review prevented; and a tag ruleset limits who can create or update v* tags, with GitHub Actions allowed to bypass for promote and tag-nightly. Then confirm the result with gh api repos/callstack/agent-device/environments before the npm trust is granted.

Not blocking, and fine to take or leave: the workflow test at release-workflow.test.ts:20 checks which jobs can mint an id-token but not the approval gate, so removing approve from publish.needs or the needs.approve.result == 'success' check leaves it green; if gh workflow run fails after the tag is created, re-running failed jobs re-runs promote, which then fails with 422 because the ref exists, so skip tag creation when the tag already points at the commit or document dispatching on the existing tag; and isPublished in release-publish.mjs:59 treats every npm view error as "not published", so it should return false only on E404 and rethrow the rest.

Is there a smaller design? I did not find one that keeps these properties, and the two-hop promotion (tag, then dispatch) looks justified because the stable run gets the exact promoted commit as its sha and provenance. Could #3381 land with or right after this PR, so the old local-publish path is removed and two copies do not sit on main? The merge needs the environment and tag protections in place and documented first.

@thymikee
thymikee force-pushed the feat/release-workflow branch from 61567a6 to 365abaa Compare October 10, 2026 17:27
@thymikee
thymikee removed this pull request from stack #3382 October 10, 2026 17:28
@thymikee
thymikee force-pushed the feat/release-workflow branch from 365abaa to 7b8e5c3 Compare October 10, 2026 17:30
@thymikee
thymikee added this pull request to stack #3389 October 10, 2026 17:30
@thymikee

Copy link
Copy Markdown
Member Author

Addressed. A GitHub-side check now gates publishing.

Blocking item. The plan cannot protect anything by itself, as you said. GitHub has to enforce the gates, so:

  • The repository setup check is in its own layer, feat(release): refuse to publish until GitHub enforces the release protections #3388, to keep this PR under the diff budget. Before a nightly or stable publish, the plan reads the repository's settings through the public API and refuses, naming each gap, unless:

    • npm-publish deploys only from main and v*
    • release requires a reviewer
    • an active ruleset restricts creating, updating, and deleting refs/tags/v*

    Run read-only against today's repo, it refuses with all three gaps, so merging before setup publishes nothing.

  • Tag ruleset with an Actions bypass. I couldn't confirm that the built-in Actions token can be a ruleset bypass actor, so the promote job no longer creates the stable tag. An admin runs pnpm release:promote [X.Y.Z], which pushes vX.Y.Z at the latest nightly's commit. Only the Repository admin role may bypass the v* ruleset. Nightly tags moved to nightly/v…, outside both that ruleset and the npm-publish tag policy. This also removes the promote 422 re-run case.

  • Documented in chore(release): retire local npm publishing #3381 under "One-time repository setup": both environments, the ruleset, the Actions settings, and the gh api checks to run before granting npm trust. "Prevent self-review" is documented as optional, because the admin who pushes the tag can't approve their own run without a second maintainer.

  • Applied: not yet. I tried to create the environments with gh api, and the auto-mode permission policy blocked it. The maintainer applies them by hand.

  • Remaining boundary: main doesn't require PRs, so anyone who can push to main can still change the next nightly. This is documented as accepted for now.

Non-blocking items. Both are fixed at 7b8e5c359b:

  • isPublished returns false only when npm's JSON error code is E404 and rethrows anything else. Against a stub npm, a network error failed the run without publishing.
  • The workflow test now pins approve on the release environment, and publish waiting on needs.approve.result == 'success'. Removing either fails it; both mutations were checked.

#3381 lands right after this one in the stack.

@thymikee
thymikee force-pushed the feat/release-workflow branch from 7b8e5c3 to 9c6ef86 Compare October 10, 2026 17:42
@thymikee

Copy link
Copy Markdown
Member Author

Changed the stable flow at the maintainer's request. A stable release now publishes the commit its vX.Y.Z tag points at, normally main's HEAD, created by publishing a GitHub release. It no longer promotes the latest nightly, and pnpm release:promote is gone. The stable run now also requires green ci.yml on that commit (9c6ef86d35). The admin-only v* ruleset and the release approval are unchanged.

@thymikee
thymikee marked this pull request as ready for review October 10, 2026 19:08

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/publish-mcp-registry.yml
Comment thread test/ci/release-workflow.test.ts Outdated
Comment thread .github/workflows/release.yml
Comment thread .github/workflows/release-android-snapshot-helper.yml
Comment thread .github/workflows/release-ios-runner.yml
Comment thread scripts/release-version.mjs Outdated
Comment thread scripts/release-plan.mjs
@thymikee

Copy link
Copy Markdown
Member Author

The earlier findings from 61567a6 are fixed at 9c6ef86, and I found no new blocking problems in the code.

CI shows 33 checks reported and none failing at 9c6ef86. The changed release.yml route (nightly or stable publish) cannot run on a PR, so CI only exercises the dry-run mode. I did not run the tests locally. I judged the earlier regressions by reading the pre-delta code. Repository settings (v* ruleset, release reviewers, npm-publish deployment policy) are not applied yet, per the author, and the #3388 guard is in the stacked PR, not this head. I also could not check that a GitHub release published from the UI with a new tag fires the push: tags trigger, whether immutable releases are enabled (they would reject asset uploads to an already-published release), or that the MCP Registry rejects a version with no matching npm package. There are no conflicts.

On the open threads: the "mark-released push skips CI" thread (#3380 (comment)) and the "unanchored test match" thread (#3380 (comment)) still apply. Three lower-priority threads still apply as well. These threads do not apply, so you can resolve them: the MCP registry tag check (#3380 (comment)) never limited what a writer could register, and the helper and runner version-equality threads (#3380 (comment) and #3380 (comment)) would fail every stable run because the tagged commit carries X.Y.Z-dev.

Not blocking: in the new stable flow, an admin who publishes the GitHub release first makes it public before the CI gate at release-plan.mjs:60 and the release approval run, so a release made right after a merge can fail the gate while ci.yml is still running and then point at a version that is not on npm (documenting the draft-release or tag-push route, or letting the gate poll for a bounded time, would fix it), and you can take or leave this.

To merge, please answer the thread on the mark-released push (#3380 (comment)). That bump commit is pushed with GITHUB_TOKEN, so it never gets a ci.yml run, and the stable CI gate added here, like the nightly gate, refuses any commit without one. Push with a token that starts workflows, or treat that commit as covered by its parent's CI.

@thymikee
thymikee force-pushed the feat/release-workflow branch from 9c6ef86 to e18c816 Compare October 11, 2026 06:51

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/release-plan.mjs Outdated
@thymikee
thymikee force-pushed the feat/release-workflow branch from e18c816 to 498ff88 Compare October 11, 2026 06:55
…d publishing

.github/workflows/release.yml owns every npm publish. A schedule publishes
main's HEAD under the nightly dist-tag once CI passed; dispatching stable
promotes the latest nightly's commit to X.Y.Z, then publishes the GitHub
release, its assets, and the MCP Registry entry, and moves main to the next
-dev version. Only the publish job can mint an id-token, and no npm token is
involved.
…e approval gate

The publisher treats only npm's E404 as "not published" and rethrows any other
registry failure. Promotion reuses a tag that already points at the promoted
commit, so re-running a failed dispatch succeeds. The workflow test now fails
if a stable publish stops waiting for the release environment approval.
A stable release starts from a vX.Y.Z tag on main that a repository admin
creates, normally by publishing a GitHub release targeting main. The stable
run now also requires green CI on that commit. This replaces the promote job
and the GITHUB_TOKEN tag it created. Nightly tags move to nightly/v*, outside
the admin-only v* tags and the npm-publish tag policy.
…e plan

The workflow's -dev bump is pushed with GITHUB_TOKEN, so it never gets a CI
run. A main commit with no CI run now stands on its parent's CI when it only
rewrites version fields, so nightlies and stable releases keep working after
a release. A stable tag waits up to 20 minutes for its commit's CI, the npm
nightly dist-tag also blocks a second nightly that UTC day, version patterns
reject leading zeros, and the approval test pins the publish condition
exactly.
@thymikee
thymikee force-pushed the feat/release-workflow branch from 498ff88 to db9484f Compare October 11, 2026 07:01
@thymikee

Copy link
Copy Markdown
Member Author

The two blocking threads are fixed in db9484fff6:

  • mark-released CI: a main commit with no CI run of its own now uses its parent's CI, but only if every changed line is a "version" field.
  • Exact publish.if: the test now asserts the whole normalized expression.

The other three applicable threads are fixed too: leading zeros, the npm nightly dist-tag as the "already shipped today" record, and waiting while a CI run is requested or not created yet. The MCP, helper, and runner threads are answered and resolved as not applicable.

On the non-blocking note about publishing the release before CI finishes: a stable tag now waits up to 20 minutes for its commit's CI, so a release published right after a merge waits for that merge's CI instead of failing. The release is still public before approval. CONTRIBUTING says to delete the release and tag if you reject the run, or to push the tag instead, in which case the run drafts the release itself.

@thymikee

Copy link
Copy Markdown
Member Author

The PR is ready at db9484f. The earlier findings from 9c6ef86 are fixed: the bump commit now inherits its parent's CI result, the npm nightly dist-tag check is in place, the version segment checks reject leading zeros, and the publish condition matches its test.

Not blocking: you can take or leave these. First, the tests only cover isVersionOnlyCommit, so dropping requested or missing from the unsettled set, or the bump-to-parent fallback, would keep every test green; a pure ciVerdict(runs, commit) with tests for requested, missing, bump-to-parent and exhausted-budget would pin it (release-plan.mjs#L120). Second, readRun waits for CI before the not-on-main and older-than-latest refusals, so a v* tag on an off-main commit polls for about 20 minutes before it fails; computing those two checks before the wait would fail it at once (release-plan.mjs#L191).

The review threads from cubic-dev-ai no longer apply, since each is fixed at this commit, so please resolve them: release-plan.mjs wait states, release-workflow publish condition, bump inherits parent CI, leading-zero version segments, and npm nightly dist-tag.

CI is still pending. The only check not passing is Build 0.21.25-nightly.20261011.32, which is still in progress. It is this PR's own release.yml dry-run, so it runs the changed route (plan, stamp, then release:prepare) and has no result to attribute yet. There are no conflicts.

I did not run a stable tag release or a scheduled nightly against the live GitHub and npm APIs. I checked the polling loop and the bump-to-parent recursion by reading the code only. I also assumed the commits API leaves out the ---/+++ headers in patch and lists runs newest first, and I did not check either against the live API. Before merge, that dry-run build needs to finish green.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 11, 2026
@thymikee
thymikee merged commit 22efd9e into main Oct 11, 2026
40 checks passed
@thymikee
thymikee deleted the feat/release-workflow branch October 11, 2026 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant