WHL: build limited-api compliant wheels

[neutrinoceros]

Based off #371, similar to Unidata/netcdf4-python#1427
I've included a commit to drop support for 3.8 and 3.9 (soon to be EOL), but it can easily be split out of this PR if desired.

[neutrinoceros]

@jswhit I would recommend running CI on #371 first, and hopefully merge it, so I'll gain "contributor" privileges on this repo and will be able to iterate much faster on fixing CI here.

[neutrinoceros]

(that said, I think I found the issue and updated the branch here)

[neutrinoceros]

Here's the current error

ImportError: dlopen(/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/cibw-run-67bztgcn/cp314-macosx_arm64/venv-test-arm64/lib/python3.14/site-packages/cftime/_cftime.abi3.so, 0x0002): symbol not found in flat namespace '_PyCode_NewWithPosOnlyArgs'

this is likely caused by some limitation (or bug) in Cython where it can still generate code that isn't strictly Limited-API-compliant. This is similar to why cp310-abi3 doesn't work, so let's try targeting cp312-abi3 instead

[neutrinoceros]

cp312-abi3 still errors (different symbol)

ImportError: dlopen(/private/var/folders/y6/nj790rtn62lfktb1sh__79hc0000gn/T/cibw-run-2th61c55/cp314-macosx_arm64/venv-test-arm64/lib/python3.14/site-packages/cftime/_cftime.abi3.so, 0x0002): symbol not found in flat namespace '__PyThreadState_UncheckedGet'

let's try cp313-abi3

[neutrinoceros]

cp313-abi3 builds fine and even passes tests on CPython 3.13, however it doesn't seem to survive the forward compatibility check on CPython 3.14:

2025-09-24T06:37:12.7224860Z + /bin/sh -c 'python -c "import cftime; print(f'"'"'cftime v{cftime.__version__}'"'"')" && python -m pytest -vv /Users/runner/work/cftime/cftime/test
2025-09-24T06:37:12.7225340Z '
2025-09-24T06:37:12.7535710Z <frozen importlib._bootstrap>:491: RuntimeWarning: compile time Python version 3.13 of module 'cftime._cftime' does not match runtime version 3.14

This again might be a sign of a limitation or bug in Cython, but I imagine it could also be a symptom of 3.14 not being completely stabilized yet (even though we're on rc3 at this point). I cannot do much more for now, I'll need to wait for the final 3.14.0 release, about 2 weeks from now, to see if anything changes.
If not, I'll report the 3 failure modes to Cython devs, and we'll see where it goes to from there.

[jswhit]

python 3.14 is released, would like to merge this PR before making a new release (if possible)

[neutrinoceros]

I can try out a new run

[neutrinoceros]

actually, it looks like cibuildwheels needs to be patched (and then upgraded here) to use the final 3.14.0 release, but the patch isn't there yet.
What's more, I'm not sure I know enough about the internals of cibuildwheel to emulate that missing patch myself, so I'd be more confortable just waiting it out (might be a couple days). Would you be confortable with waiting for required changes upstream before revisiting this ? Even then, there's a non-zero chance CI still doesn't pass against the final release.

[jswhit]

I wonder why it worked for netcdf4-python (Unidata/netcdf4-python#1427)?

[jswhit]

I'm OK with waiting, we can always issue a patch release with the limited abi wheels

[neutrinoceros]

I wonder why it worked for netcdf4-python

It worked out of the box for most packages I tried this on indeed, but not all. It depends on Cython for generating correct and compatible code, and maybe we're just hitting a bug in Cython or CPython itself (yikes). If 3.14.0 final doesn't work, I'll upstream the discussion to interested parties.

[neutrinoceros]

here's a clue; the warning we see before the interpreter crashes, namely

RuntimeWarning: compile time Python version 3.13 of module 'cftime._cftime' does not match runtime version 3.14

is generated by Cython (specifically __Pyx_check_binary_version)

[neutrinoceros]

I could be wrong but it looks to me like the information that we want to target abi3 is somehow not reaching Cython. I suspect it might be caused by the specific logic used in setup.py for this project but I'm not certain yet.

[jswhit]

I see that the cftime setup.py uses Cython.Build.cythonize, while netcdf4-python doesn't.

[neutrinoceros]

that's definitely the difference, thanks.
Now I'm convinced that what's broken is this setup.py. I do have a couple known-working packages that combine abi3 + cythonize + setuptools so I know it's feasible, however all examples I know are somewhat hacky (and already were before they moved to abi3 !), all in their unique ways, so I don't know how big an effort it'd be to reproduce this here. In particular, I just tried to emulate how h5py does it, but it's even more broken now, so that seems like a red herring.

[neutrinoceros]

I opened #375 to unblock your release process

[ocefpaf]

so that seems like a red herring.

BTW, I tried to build this locally for cp313 and ran the tests on cp314, everything works. I could not reproduce the error here :-/

[neutrinoceros]

refreshing CI with a rebase just in case it works better this time (and because I forgot what the error looked like)

[ocefpaf]

I think I found the problem. These should probably be defined outside of this if-clause. I also believe that CYTHON_TRACE is not compatible with ABI3.

[neutrinoceros]

Nice. Feel free to push to this branch if you like. I'll be AFK until tomorrow

[neutrinoceros]

Thanks a lot @ocefpaf. I've integrated your fix from #395 back here, so I expect CI to pass now.