Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# NVRC Release Process

This document describes how to create a new NVRC release.

## Prerequisites

- Write access to the repository
- Ability to merge PRs to `main` branch

## Important: Immutable Releases

This repository has **Immutable Releases** enabled as a security measure. Once a
release is published:

- Release assets cannot be modified or added
- The tag cannot be reused, even if deleted
- Each release attempt requires a unique version number

If a release workflow fails after creating the tag, you must bump the version
and try again. There is no way to "retry" with the same version.

**Note:** Even if you disable immutable releases after a failed release attempt,
you still cannot reuse the same version number. GitHub permanently remembers
that the tag/release existed. You must always bump to a new version.

## Release Workflow

### Step 1: Bump Version in Cargo.toml

Create a PR that updates the version in `Cargo.toml`:

```toml
[package]
name = "NVRC"
version = "X.Y.Z" # Update this
```

The version in `Cargo.toml` is the **source of truth** for release tags. The
workflow automatically derives the tag name as `vX.Y.Z` from this version.

Merge the PR to `main` after review.

### Step 2: Trigger the Release Workflow

1. Go to **Actions** in the GitHub repository
2. Select **Release NVRC** workflow from the left sidebar
3. Click **Run workflow**
4. Ensure **main** branch is selected (this is the default and required)
5. Click the green **Run workflow** button

**Important:** Always run from `main` branch. The signature verification step
validates that artifacts were built from `main` using certificate constraints
like `--certificate-github-workflow-ref`.

### Step 3: Monitor the Workflow

The release workflow performs these steps:

1. **preflight** - Derives tag from `Cargo.toml`, checks no existing release,
creates git tag
2. **build-and-release** - Builds binaries for x86_64 and aarch64, signs with
Sigstore/cosign, creates tarballs
3. **create-release** - Creates a **draft** GitHub release with tarballs
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow step description states that create-release "Creates a draft GitHub release with tarballs", but the actual workflow does not use the draft option. The release is published immediately, not kept as a draft. This should be corrected to say "Creates a GitHub release with tarballs" (removing "draft").

Suggested change
3. **create-release** - Creates a **draft** GitHub release with tarballs
3. **create-release** - Creates a GitHub release with tarballs

Copilot uses AI. Check for mistakes.
4. **provenance** - Generates SLSA Level 3 provenance attestations
5. **provenance-publish** - Uploads provenance to the draft release
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The step description says "Uploads provenance to the draft release", but the release is not a draft - it's published immediately when created. Remove the word "draft" to accurately reflect the workflow behavior.

Suggested change
5. **provenance-publish** - Uploads provenance to the draft release
5. **provenance-publish** - Uploads provenance to the release

Copilot uses AI. Check for mistakes.
6. **release-notes** - Adds VERIFY.md content to release body
7. **publish-release** - Publishes the draft (makes it immutable)
Comment on lines +64 to +67
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow does not have a "publish-release" job. The actual workflow jobs are: preflight, build-and-release, create-release, provenance-hash-all, provenance, provenance-publish, release-notes, and verify-signatures. Remove this line from the step list.

Suggested change
4. **provenance** - Generates SLSA Level 3 provenance attestations
5. **provenance-publish** - Uploads provenance to the draft release
6. **release-notes** - Adds VERIFY.md content to release body
7. **publish-release** - Publishes the draft (makes it immutable)
4. **provenance-hash-all** - Computes hashes for all release artifacts for provenance
5. **provenance** - Generates SLSA Level 3 provenance attestations
6. **provenance-publish** - Uploads provenance to the draft release
7. **release-notes** - Adds VERIFY.md content to release body

Copilot uses AI. Check for mistakes.
8. **verify-signatures** - Verifies all signatures and provenance
Comment on lines +64 to +68
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow step list is missing the "provenance-hash-all" job that runs between build-and-release and provenance. While this is an internal implementation detail, for completeness the documentation should either include it or clarify that only the major user-visible steps are listed. Consider adding: "4. provenance-hash-all - Prepares hash data for provenance generation" between steps 3 and 4, and renumbering subsequent steps.

Suggested change
4. **provenance** - Generates SLSA Level 3 provenance attestations
5. **provenance-publish** - Uploads provenance to the draft release
6. **release-notes** - Adds VERIFY.md content to release body
7. **publish-release** - Publishes the draft (makes it immutable)
8. **verify-signatures** - Verifies all signatures and provenance
4. **provenance-hash-all** - Prepares hash data for provenance generation
5. **provenance** - Generates SLSA Level 3 provenance attestations
6. **provenance-publish** - Uploads provenance to the draft release
7. **release-notes** - Adds VERIFY.md content to release body
8. **publish-release** - Publishes the draft (makes it immutable)
9. **verify-signatures** - Verifies all signatures and provenance

Copilot uses AI. Check for mistakes.

The release remains a draft until all assets are uploaded, then gets published
in a single atomic operation. This ensures the immutable release contains all
required artifacts.
Comment on lines +70 to +72
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow does not create a draft release. The create-release job uses softprops/action-gh-release without the draft option, which means it creates a published release immediately. The documentation incorrectly states that "The release remains a draft until all assets are uploaded, then gets published in a single atomic operation." This should be corrected to accurately reflect that the release is created and published in the create-release step, and subsequent jobs add additional assets to the already-published release.

Copilot uses AI. Check for mistakes.

## Troubleshooting

### "Release already exists" Error

If preflight fails with "Release already exists", a previous release attempt
partially succeeded. With immutable releases enabled, you cannot reuse the
version. Bump the version in `Cargo.toml` and try again.

### "Cannot upload assets to an immutable release" Error

This occurs if the release was published before all assets were uploaded. The
workflow uses draft releases to prevent this, but if it happens:
Comment on lines +84 to +85
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This troubleshooting section states "The workflow uses draft releases to prevent this", but the workflow does not actually use draft releases. Since the release is published immediately, this error could occur if subsequent jobs (provenance-publish or release-notes) fail to upload their assets. The statement should be corrected to reflect the actual workflow behavior.

Copilot uses AI. Check for mistakes.

1. Delete the release: `gh release delete vX.Y.Z --repo NVIDIA/nvrc --yes`
2. Delete the tag: `gh api -X DELETE repos/NVIDIA/nvrc/git/refs/tags/vX.Y.Z`
3. Bump version and retry (required if immutable releases is enabled)

### Tag Creation Fails

If tag creation fails with repository rule violations, there may be org-level
restrictions. Contact the repository administrator.

### Signature Verification Fails

The verify-signatures job validates:

- Cosign keyless signatures (online via Rekor)
- Cosign bundle signatures (offline verification)
- SLSA provenance (via slsa-verifier)

Failures here indicate a problem with the signing process or certificate
constraints. Check that the workflow ran from `main` branch.

## Release Artifacts

Each release includes per-architecture:

| File | Description |
| ------------------------------------------ | ---------------------------------------- |
| `NVRC-{arch}.tar.xz` | Tarball containing binary and signatures |
| `NVRC-{arch}.tar.xz.sig` | Cosign signature for tarball |
| `NVRC-{arch}.tar.xz.cert` | Cosign certificate for tarball |
| `NVRC-{arch}.tar.xz.bundle.json` | Rekor bundle for offline verification |
| `NVRC-{arch}.intoto.jsonl` | SLSA provenance attestation |

Inside each tarball:

| File | Description |
| ------------------------------------------ | -------------------- |
| `NVRC-{arch}` | The binary |
| `NVRC-{arch}.sig` | Binary signature |
| `NVRC-{arch}.cert` | Binary certificate |
| `NVRC-{arch}.bundle.json` | Binary Rekor bundle |
| `sbom-NVRC-{arch}.spdx.json` | SBOM in SPDX format |
| `sbom-NVRC-{arch}.spdx.json.sig` | SBOM signature |
| `sbom-NVRC-{arch}.spdx.json.cert` | SBOM certificate |
| `sbom-NVRC-{arch}.spdx.json.bundle.json` | SBOM Rekor bundle |
Comment on lines +113 to +130
Copy link

Copilot AI Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The artifact naming in the table uses {arch} as a placeholder, which is ambiguous. The actual workflow uses full target triples like x86_64-unknown-linux-musl and aarch64-unknown-linux-musl. Update the table to use {target} instead of {arch} to be consistent with the workflow terminology, or clarify in the table header that {arch} refers to the full target triple (e.g., "where {arch} is x86_64-unknown-linux-musl or aarch64-unknown-linux-musl").

Suggested change
| `NVRC-{arch}.tar.xz` | Tarball containing binary and signatures |
| `NVRC-{arch}.tar.xz.sig` | Cosign signature for tarball |
| `NVRC-{arch}.tar.xz.cert` | Cosign certificate for tarball |
| `NVRC-{arch}.tar.xz.bundle.json` | Rekor bundle for offline verification |
| `NVRC-{arch}.intoto.jsonl` | SLSA provenance attestation |
Inside each tarball:
| File | Description |
| ------------------------------------------ | -------------------- |
| `NVRC-{arch}` | The binary |
| `NVRC-{arch}.sig` | Binary signature |
| `NVRC-{arch}.cert` | Binary certificate |
| `NVRC-{arch}.bundle.json` | Binary Rekor bundle |
| `sbom-NVRC-{arch}.spdx.json` | SBOM in SPDX format |
| `sbom-NVRC-{arch}.spdx.json.sig` | SBOM signature |
| `sbom-NVRC-{arch}.spdx.json.cert` | SBOM certificate |
| `sbom-NVRC-{arch}.spdx.json.bundle.json` | SBOM Rekor bundle |
| `NVRC-{target}.tar.xz` | Tarball containing binary and signatures |
| `NVRC-{target}.tar.xz.sig` | Cosign signature for tarball |
| `NVRC-{target}.tar.xz.cert` | Cosign certificate for tarball |
| `NVRC-{target}.tar.xz.bundle.json` | Rekor bundle for offline verification |
| `NVRC-{target}.intoto.jsonl` | SLSA provenance attestation |
Inside each tarball (per target triple, where `{target}` is `x86_64-unknown-linux-musl` or `aarch64-unknown-linux-musl`):
| File | Description |
| ------------------------------------------ | -------------------- |
| `NVRC-{target}` | The binary |
| `NVRC-{target}.sig` | Binary signature |
| `NVRC-{target}.cert` | Binary certificate |
| `NVRC-{target}.bundle.json` | Binary Rekor bundle |
| `sbom-NVRC-{target}.spdx.json` | SBOM in SPDX format |
| `sbom-NVRC-{target}.spdx.json.sig` | SBOM signature |
| `sbom-NVRC-{target}.spdx.json.cert` | SBOM certificate |
| `sbom-NVRC-{target}.spdx.json.bundle.json` | SBOM Rekor bundle |

Copilot uses AI. Check for mistakes.

## Verifying a Release

The release workflow automatically runs signature verification as the final step
(`verify-signatures` job) to ensure all artifacts are correctly signed before
the release is considered complete. This serves as a sanity check that the
signing and publishing process succeeded.

For manual verification of downloaded releases, see [VERIFY.md](VERIFY.md) for
instructions on verifying release signatures and provenance.
Loading