Skip to content

dynamically test with all supported Python versions#2256

Open
zacharyburnett wants to merge 2 commits into
spacetelescope:mainfrom
zacharyburnett:test/all_supported_pythons
Open

dynamically test with all supported Python versions#2256
zacharyburnett wants to merge 2 commits into
spacetelescope:mainfrom
zacharyburnett:test/all_supported_pythons

Conversation

@zacharyburnett

@zacharyburnett zacharyburnett commented Apr 6, 2026

Copy link
Copy Markdown
Collaborator

uses fill: to fill in toxenvs py311, py312, py313 (reading requires-python: from pyproject.toml)

analogous to spacetelescope/jwst#10428

blocked by OpenAstronomy/github-actions-workflows#388 and release v2.6.3 of the OpenAstronomy workflows

this pr is on hold pending a better solution in the OpenAstronomy workflows

Tasks

  • request a review from someone specific, to avoid making the maintainers review every PR
  • add a build milestone, i.e. 24Q4_B15 (use the latest build if not sure)
  • Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
    • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see changelog readme for instructions)
      • if your change breaks existing functionality, also add a changes/<PR#>.breaking.rst news fragment
    • update or add relevant tests
    • update relevant docstrings and / or docs/ page
    • start a regression test and include a link to the running job (click here for instructions)
      • Do truth files need to be updated ("okified")?
        • after the reviewer has approved these changes, run okify_regtests to update the truth files
  • if a JIRA ticket exists, make sure it is resolved properly

@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from 3501012 to f2054fe Compare April 6, 2026 15:50
@zacharyburnett zacharyburnett changed the title dynamically test with al supported Python versions using new fill: parameter introduced in OpenAstronomy/github-actions-workflows@2.6.0 dynamically test with all supported Python versions using new fill: parameter introduced in OpenAstronomy/github-actions-workflows@2.6.0 Apr 6, 2026
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch 2 times, most recently from f793536 to 4c3b4b9 Compare April 6, 2026 16:03
@codecov

codecov Bot commented Apr 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.00%. Comparing base (67d9afc) to head (c154650).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2256   +/-   ##
=======================================
  Coverage   81.00%   81.00%           
=======================================
  Files         159      159           
  Lines        9509     9509           
=======================================
  Hits         7703     7703           
  Misses       1806     1806           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@zacharyburnett zacharyburnett requested a review from braingram April 7, 2026 15:22
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from 4c3b4b9 to 287a249 Compare April 7, 2026 15:26
@zacharyburnett zacharyburnett marked this pull request as ready for review April 7, 2026 15:26
@zacharyburnett zacharyburnett changed the title dynamically test with all supported Python versions using new fill: parameter introduced in OpenAstronomy/github-actions-workflows@2.6.0 dynamically test with all supported Python versions using new fill: parameter introduced in OpenAstronomy/github-actions-workflows@2.6.3 Apr 7, 2026
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from 287a249 to 70b737a Compare April 7, 2026 15:30
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from 70b737a to b3e2dc7 Compare April 8, 2026 15:07
@zacharyburnett zacharyburnett marked this pull request as draft May 15, 2026 17:34
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from b3e2dc7 to 3619ada Compare June 5, 2026 19:34
@zacharyburnett zacharyburnett changed the title dynamically test with all supported Python versions using new fill: parameter introduced in OpenAstronomy/github-actions-workflows@2.6.3 dynamically test with all supported Python versions Jun 5, 2026
@zacharyburnett

zacharyburnett commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator Author

I've refactored this PR to do the following:

  • make a dynamic list of Python versions that this package supports, that we can manually update at our own pace to increase the latest version (so that when Python 3.15 comes out the CI doesn't immediately fail)
  • set static job names run tests (oldest supported Python, ubuntu-latest) and run tests with coverage (latest supported Python, ubuntu-latest), that we can then set as "required" in the branch protection rules but which will always align with our oldest and latest supported Python versions

image

@zacharyburnett

Copy link
Copy Markdown
Collaborator Author

(this PR basically copies the upcoming package template as closely as possible, so that rollout of that template will have minimal changes when it happens)

@zacharyburnett zacharyburnett marked this pull request as ready for review June 5, 2026 19:36
@zacharyburnett zacharyburnett force-pushed the test/all_supported_pythons branch from 6dd1eed to c154650 Compare June 8, 2026 19:34
@zacharyburnett

Copy link
Copy Markdown
Collaborator Author

in the repository settings, before or after merging this PR we can change the current required checks:

check / check-dependencies
test / py311-oldestdeps-xdist-cov (ubuntu-latest)
test / py314-cov (ubuntu-latest)
test / py314-pyargs (macos-latest)

to

test / check-dependencies
test / run tests (oldest supported Python, ubuntu-latest)
test / run tests with coverage (latest supported Python, ubuntu-latest)

and remove the other version-specific ones

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant