Repository navigation
Add release workflow - #2
Draft
GMNGeoffrey wants to merge 11 commits into
Draft
GMNGeoffrey wants to merge 11 commits into
GMNGeoffrey wants to merge 11 commits into
Conversation
Python packaging continues to struggle with environment-dependent packages. We do have a direct dependency on Triton and properly we should declare it, but doing so creates problems and doesn't really have much benefit. We get triton already through the transitive torch dependency. The package providing triton sometimes has a different name like `pytorch-triton-rocm` and then pip will clobber it with the version from PyPI. Just not declaring the triton dependency appears to be the least bad option and a common practice in the other Triton-based PyPI packages I surveyed. This allows us to remove `--no-deps` from our install instructions. The CPU CI job is one case where this has a cost: CPU torch doesn't depend on Triton, so we don't get it automatically. A real user wouldn't be using Triton in this scenario, but we install it explicitly so that we can at least test collection of the Triton-based tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Otherwise sdist and wheel builds would not include one. A symlink would be left as a dangling link (hatchling doesn't automagically turn it into a copy or anything like that). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For now we want to be able to alter these together. We may eventually disconnect them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This is a separate dist. Hatchling by default picks up everything git tracks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use `license = "Apache-2.0"` so the metadata carries License-Expression rather than the legacy free-text License field, and drop the License :: classifier, which PEP 639 deprecates alongside an expression. Require hatchling>=1.27, the first release that writes core metadata 2.4, which defines License-Expression and License-File, by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build.yml builds the sdists and wheels for both packages and validates
them:
- `twine check --strict` validating metadata
- `check-wheel-contents` for general hygiene. `--package` for extra or
missing files.
- shim and kernel packages have matching versions and the shim pins the
same.
- both wheels have a license
It runs on PRs and main to surface packaging problems before release
time.
release.yml calls build.yml and publishes its artifacts to a GitHub
release and PyPI through Trusted Publishing for the repository `pypi`
environment. It runs only on manual dispatch from a version tag (X.Y.Z
or X.Y.Z{a,b,rc}N) pointing to a commit on `main` that must match the
version declared in the pyproject.toml.
A TestPyPI option skips the GitHub release and just publishes to
TestPyPI. It accepts tags off main for ease of testing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
@Emrys-Merlin this is what I had in mind. WDYT? (draft still because I need to actually test TestPyPI and set up TrustedPublisher links) |
|
Yes, this looks good to me :-) That creates everything we need for a conda-forge release downstream. |
Start publishing with an alpha. Both packages and the shim's pin on the kernels move together, which the build workflow checks. Drop the hand-maintained __version__ attributes. Nothing reads them, and the installed version is available from importlib.metadata. If this becomes an issue we can consider adding them back. I think the big packages that do this usually generate it at build time, but that complicates simple workflows like being able to run tests from a clone without installing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Testing a release on TestPyPI, especially of an unmerged commit, should not use up the vVERSION tag the real release will need. Require a test-vVERSION tag for TestPyPI and keep vVERSION for PyPI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PyPI token exchange can create at most one new project, so a single job can't create both projects on their first release. Split publishing into one job per package. The shim job runs after the kernels job, since the shim pins the kernels' exact version and must not be published without them. This also allows us to ensure that the kernel publication is successful before trying to publish the shim and reduce the likelihood of a broken state. build.yml now builds each package into its own directory and uploads it as its own artifact, so each publish job downloads only its package. The GitHub release job downloads both. PyPI only allows one pending publisher per repository, workflow and environment. So to bootstrap with the first release: register the kernels' pending publisher, release (shim job will fail), register the shim's pending publisher once the kernels' has become a normal publisher, and re-run the failed shim job. The GitHub release job now always runs and only its release step is conditional on PyPI to maximize the amount of the workflow tested. Since no job is skipped, the publish jobs can use plain needs without conditions to work around skipped dependencies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tag prefix already says where a release goes, so stop asking for it separately. The check job now derives the index from the tag, vVERSION for PyPI and test-vVERSION for TestPyPI, and passes it on as an output that selects the environment, the upload URL and whether to create the GitHub release. Dispatching the workflow then only needs the tag, a mismatch between the tag and the choice can no longer happen, and a run from a trigger other than workflow_dispatch needs no stand-in input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
workflow_dispatch only works once the workflow is on the default branch. A push trigger runs the workflow at the pushed commit, so this lets us test a TestPyPI release before merging the PR using a test-v* tag. Drop this commit before merging. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GMNGeoffrey
had a problem deploying
to
testpypi
November 4, 2026 02:02 — with 
GitHub Actions
Failure
GitHub Actions
Failure
Member
Author
|
As expected, the first attempt at publish-shim failed because of the weird thing where you can only have a pending publisher for one project at a time. Rerun worked fine: https://github.com/AMDResearch/geometric-nn-ops/actions/runs/41757866217/. Packages now published to TestPyPI: https://test.pypi.org/project/geometric-nn-ops-torch/, https://test.pypi.org/project/hipequivariance-ops-torch/. The real thing will be only after we actually merge this, but I think that it's good to go other than reverting the temporary on-push trigger (we might eventually revive that, but for now I want releases to be a bit more manual). |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mostly some metadata cleanup and actions workflows.
build.yml builds the sdists and wheels for both packages and validates them:
twine check --strictvalidating metadatacheck-wheel-contentsfor general hygiene.--packagefor extra or missing files.It runs on PRs and main to surface packaging problems before release time.
release.yml calls build.yml and publishes its artifacts to a GitHub release and PyPI through Trusted Publishing for the repository
pypienvironment. It runs only on manual dispatch from a version tag (X.Y.Z or X.Y.Z{a,b,rc}N) pointing to a commit onmainthat must match the version declared in the pyproject.toml.A TestPyPI option skips the GitHub release and just publishes to TestPyPI. It accepts tags off main for ease of testing.
This PR also includes dropping Triton as an explicit dependency. Python packaging continues to struggle with environment-dependent packages. We do have a direct dependency on Triton and properly we should declare it, but doing so creates problems and doesn't really have much benefit.
We get triton already through the transitive torch dependency. The package providing triton sometimes has a different name like
pytorch-triton-rocmand then pip will clobber it with the version from PyPI. Just not declaring the triton dependency appears to be the least bad option and a common practice in the other Triton-based PyPI packages I surveyed. This allows us to remove--no-depsfrom our install instructions.