Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Release

# Builds multi-platform wheels + sdist and (optionally) publishes to PyPI /
# TestPyPI via Trusted Publishing (OIDC) — no API token is stored.
#
# Triggers:
# * push tag v* -> build, gate on provenance, publish to PyPI.
# * workflow_dispatch -> build, and publish to the chosen target:
# none (artifacts only), testpypi, or pypi.
#
# One-time setup before the first real publish (see docs/releasing.md):
# * Configure a PyPI Trusted Publisher for this repo + workflow, and a
# TestPyPI one for dry runs.
# * Create GitHub environments named `pypi` and `testpypi`.

on:
push:
tags: ["v*"]
workflow_dispatch:
inputs:
publish:
description: Where to publish the built artifacts
type: choice
options: [none, testpypi, pypi]
default: none

permissions:
contents: read

jobs:
provenance-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Check release provenance
shell: bash
run: |
if [ "${{ github.event_name }}" = "push" ]; then
# Real tagged release: require full traceability.
python scripts/check_release_provenance.py --tag "${{ github.ref_name }}"
elif [ "${{ inputs.publish }}" = "pypi" ]; then
python scripts/check_release_provenance.py
else
# Dry run / artifact-only / TestPyPI: allow placeholder provenance.
python scripts/check_release_provenance.py --allow-unknown
fi

build_wheels:
needs: provenance-check
name: Wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
steps:
- uses: actions/checkout@v4
- name: Build wheels
uses: pypa/cibuildwheel@v2.21.3
- uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.os }}
path: ./wheelhouse/*.whl

build_sdist:
needs: provenance-check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Build sdist
run: pipx run build --sdist
- uses: actions/upload-artifact@v4
with:
name: sdist
path: dist/*.tar.gz

publish_testpypi:
needs: [build_wheels, build_sdist]
if: github.event_name == 'workflow_dispatch' && inputs.publish == 'testpypi'
runs-on: ubuntu-latest
environment: testpypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
path: dist
merge-multiple: true
- uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/

publish_pypi:
needs: [build_wheels, build_sdist]
if: github.event_name == 'push' || (github.event_name == 'workflow_dispatch' && inputs.publish == 'pypi')
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
path: dist
merge-multiple: true
- uses: pypa/gh-action-pypi-publish@release/v1
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Python port of the R NNS 13.0 package.

- Distribution package: `NNS`
- Distribution package: `ovvo-nns` (`pip install ovvo-nns`)
- Import package: `nns` (`import nns`)
- Native extension: `nns._nnscore`
- Runtime dependencies: NumPy, SciPy
Expand All @@ -13,7 +13,7 @@ Python port of the R NNS 13.0 package.
## Install

```bash
pip install NNS
pip install ovvo-nns
```

## Quick Use
Expand Down
86 changes: 86 additions & 0 deletions docs/releasing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Releasing NNS-python to PyPI

NNS-python ships a native (C++17 / nanobind) extension, so a release builds
multi-platform wheels with [`cibuildwheel`](https://cibuildwheel.pypa.io) plus an
sdist, and publishes via **PyPI Trusted Publishing (OIDC)** — no API token is
stored in the repo.

The pipeline is `.github/workflows/release.yml`.

The official distribution name is **`ovvo-nns`** (`pip install ovvo-nns`, then
`import nns`). The import package stays `nns`.

## One-time setup

### 1. Claim the PyPI project name
The first Trusted-Publishing upload registers the `ovvo-nns` name to your
account/organization — no takeover of the legacy `NNS` project is required. Use
your own PyPI account (enable 2FA), and consider owning it under a PyPI
**Organization** (e.g. `OVVO-Financial`). For the very first publish, add a
**pending** Trusted Publisher (PyPI → *Your projects* → *Publishing* → *Add a
pending publisher*) for the not-yet-existing `ovvo-nns` project.

(Optional: if you later also obtain the legacy `NNS` name, you can publish it as
an alias pointing at the same `import nns` package.)

### 2. Configure Trusted Publishers
On PyPI (and TestPyPI) for project **`ovvo-nns`** → **Publishing** → add a GitHub
Actions trusted publisher:

| Field | Value |
| --- | --- |
| PyPI Project Name | `ovvo-nns` |
| Owner | `OVVO-Financial` |
| Repository | `NNS-python` |
| Workflow name | `release.yml` |
| Environment | `pypi` (and `testpypi` on TestPyPI) |

### 3. Create GitHub environments
Repo **Settings → Environments** → create `pypi` and `testpypi`. Optionally add
required reviewers on `pypi` for a manual approval gate before publish.

No `ANTHROPIC_API_KEY` / PyPI token secret is needed — Trusted Publishing uses
the workflow's OIDC identity, and PEP 740 provenance attestations are generated
automatically.

## Wheels built

`[tool.cibuildwheel]` in `pyproject.toml` builds CPython **3.11–3.13** for:

* Linux manylinux + musllinux (`x86_64`)
* macOS `x86_64` and `arm64`
* Windows `AMD64`

Each wheel is smoke-tested (`import nns._nnscore` + a numeric call). PyPy and
32-bit targets are skipped.

## Cutting a release

1. Finalize the version in `pyproject.toml` (drop the pre-release suffix when
ready, e.g. `1.0.0a0` → `1.0.0`).
2. **Make the release traceable**: set `r_commit` and `core_commit` in
`sync/nns_source.json` to the R and NNS-core commits this build corresponds
to. The provenance gate **fails a tagged release** while these are `unknown`.
3. Dry run end to end against TestPyPI:
* Actions → **Release** → *Run workflow* → `publish: testpypi`.
* Verify the artifacts and `pip install -i https://test.pypi.org/simple/ ovvo-nns`.
4. Tag and push to publish to PyPI:
```bash
git tag v1.0.0
git push origin v1.0.0
```
The tag triggers `release.yml`, which gates on provenance, builds all
wheels + sdist, and publishes to PyPI.

`workflow_dispatch` with `publish: none` builds and uploads the artifacts to the
Actions run without publishing — useful for inspecting wheels.

## Provenance gate

`scripts/check_release_provenance.py` enforces, for a tagged release:

* the tag matches the `pyproject.toml` version (`v<version>`), and
* `sync/nns_source.json` records non-placeholder `r_commit`, `core_commit`, and
`r_version`.

Dry runs pass `--allow-unknown` so placeholder provenance does not block testing.
25 changes: 24 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
[project]
name = "NNS"
name = "ovvo-nns"
version = "1.0.0a0"
description = "Python port of nonlinear nonparametric statistics from R NNS"
readme = "README.md"
requires-python = ">=3.11"
license = "GPL-3.0-only"
authors = [
{ name = "Fred Viole", email = "ovvo.open.source@gmail.com" },
{ name = "Roberto Spadim" },
{ name = "Rasheed Khoshnaw" },
]
maintainers = [
{ name = "Fred Viole", email = "ovvo.open.source@gmail.com" },
]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Science/Research",
Expand Down Expand Up @@ -104,3 +112,18 @@ python_version = "3.11"
strict = true
files = ["src/nns", "tests"]
mypy_path = ["tests"]

[tool.cibuildwheel]
# Native (C++17 / nanobind) extension: build CPython wheels across 3.11-3.13.
# scikit-build-core fetches cmake/ninja as build deps, so no system CMake needed.
build = "cp311-* cp312-* cp313-*"
skip = ["pp*", "*_i686", "*-win32", "*-musllinux_i686"]
build-frontend = "build"
# Smoke-test every built wheel: the native extension imports and computes.
test-command = 'python -c "import nns, nns._nnscore as c; print(c.lpm(2.0, 0.0, [-2.0, -1.0, 0.5, 3.0]))"'

[tool.cibuildwheel.macos]
archs = ["x86_64", "arm64"]

[tool.cibuildwheel.windows]
archs = ["AMD64"]
78 changes: 78 additions & 0 deletions scripts/check_release_provenance.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
"""Release-readiness provenance gate for NNS-python.

Validates that a release is traceable before it is published:

* the git tag (when given) matches the `pyproject.toml` project version, and
* `sync/nns_source.json` records the R and NNS-core commits this release is
built from (not the `unknown` placeholders).

Used by `.github/workflows/release.yml`. For dry runs (TestPyPI / artifact-only
builds) pass `--allow-unknown` to permit placeholder provenance.

Exits non-zero with a clear message when a real release is not traceable.
"""

from __future__ import annotations

import argparse
import json
import sys
import tomllib
from pathlib import Path

PLACEHOLDERS = {"", "unknown", None}


def project_version(pyproject: Path) -> str:
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
return str(data["project"]["version"])


def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--tag", default="", help="release tag, e.g. v1.0.0")
parser.add_argument("--manifest", type=Path, default=Path("sync/nns_source.json"))
parser.add_argument("--pyproject", type=Path, default=Path("pyproject.toml"))
parser.add_argument(
"--allow-unknown",
action="store_true",
help="permit placeholder R/core provenance (dry runs / TestPyPI)",
)
args = parser.parse_args()

problems: list[str] = []

version = project_version(args.pyproject)
tag = args.tag.lstrip("v").strip()
if tag and tag != version:
problems.append(
f"tag '{args.tag}' does not match pyproject version '{version}' "
f"(expected tag 'v{version}')"
)

manifest = json.loads(args.manifest.read_text(encoding="utf-8"))
if not args.allow_unknown:
for field in ("r_commit", "core_commit"):
if manifest.get(field) in PLACEHOLDERS:
problems.append(
f"sync/nns_source.json '{field}' is unset/placeholder "
f"('{manifest.get(field)}'); a real release must record it "
"so the published version is traceable to R + NNS-core"
)
if manifest.get("r_version") in PLACEHOLDERS:
problems.append("sync/nns_source.json 'r_version' is unset")

if problems:
print("Release provenance check FAILED:")
for p in problems:
print(f" - {p}")
return 1

print(f"Release provenance OK (version {version}, tag '{args.tag or '(none)'}').")
if args.allow_unknown:
print(" (placeholder R/core provenance allowed for this dry run)")
return 0


if __name__ == "__main__":
sys.exit(main())
64 changes: 64 additions & 0 deletions tests/tools/test_release_provenance.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
from __future__ import annotations

import json
import subprocess
import sys
from pathlib import Path

REPO_ROOT = Path(__file__).resolve().parents[2]
SCRIPT = REPO_ROOT / "scripts" / "check_release_provenance.py"
PYPROJECT = REPO_ROOT / "pyproject.toml"


def _run(args: list[str]) -> subprocess.CompletedProcess[str]:
return subprocess.run(
[sys.executable, str(SCRIPT), *args],
cwd=REPO_ROOT,
capture_output=True,
text=True,
check=False,
)


def _project_version() -> str:
import tomllib

data = tomllib.loads(PYPROJECT.read_text(encoding="utf-8"))
return str(data["project"]["version"])


def _write_manifest(path: Path, **overrides: object) -> None:
manifest: dict[str, object] = {
"r_repo": "OVVO-Financial/NNS",
"r_commit": "unknown",
"r_version": "13.0",
"core_commit": "unknown",
}
manifest.update(overrides)
path.write_text(json.dumps(manifest), encoding="utf-8")


def test_allow_unknown_passes_with_placeholder_provenance() -> None:
result = _run(["--allow-unknown"])
assert result.returncode == 0, result.stdout + result.stderr


def test_unknown_provenance_fails_real_release() -> None:
result = _run([])
assert result.returncode != 0
assert "provenance" in (result.stdout + result.stderr).lower()


def test_tag_mismatch_fails(tmp_path: Path) -> None:
manifest = tmp_path / "manifest.json"
_write_manifest(manifest, r_commit="abc123", core_commit="def456")
result = _run(["--tag", "v9.9.9", "--manifest", str(manifest)])
assert result.returncode != 0
assert "does not match" in (result.stdout + result.stderr)


def test_matching_tag_and_full_provenance_passes(tmp_path: Path) -> None:
manifest = tmp_path / "manifest.json"
_write_manifest(manifest, r_commit="abc123", core_commit="def456")
result = _run(["--tag", f"v{_project_version()}", "--manifest", str(manifest)])
assert result.returncode == 0, result.stdout + result.stderr
Loading