Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change versioning policy to allow API breaking in minor versions? #2889

Open
dstansby opened this issue Mar 4, 2025 · 11 comments · May be fixed by #2910
Open

Change versioning policy to allow API breaking in minor versions? #2889

dstansby opened this issue Mar 4, 2025 · 11 comments · May be fixed by #2910

Comments

@dstansby
Copy link
Contributor

dstansby commented Mar 4, 2025

Our current versioning policy says:

If a release contains any backwards-incompatible public API changes, the major version number should be incremented (e.g., 2.3.0 -> 3.0.0).

An example of where we could currently have to increment to v4 is at #2819, and @d-v-b also says:

We were already planning on making breaking changes without incrementing the major version number -- deprecated functions like create were slated to be removed long before a 4.x release

So I think the proposal on the table is to update the versioning policy so:

  • The major version of zarr-python is tied to the Zarr format specification version it supports
  • The minor version is for breaking changes or major new features
  • The micro version is for bugfixes

Ping @zarr-developers/python-core-devs for comment, since this is a major change. Comments/suggestions/disagreements very welcome - if you just generally agree with the above proposal feel free to react with a 👍 to this post.

@d-v-b
Copy link
Contributor

d-v-b commented Mar 4, 2025

Generally speaking I don't have strong feelings here, but I do think we should ensure that we have a versioning policy that allows small breaking changes as needed. Although we released v3, I don't think all the code / APIs are fully settled, and it would be a shame to preserve loads of warts because we are afraid of breaking changes. The alternative to changing the versioning policy would be to just release 4.x, 5.x, etc as needed. I don't mind that, as long as I can push out my breaking changes :)

@joshmoore
Copy link
Member

Having lived with a couple of libraries that do the "tie the major version to some other repos major version" I'd be up for trying to do something else. It leaves you with less expressivity then one would often want.

My gut reaction would be to add an additional version digit but I assume our hands might be a bit tied by support for that (though it looks to be allowed by PEP440: https://peps.python.org/pep-0440/#public-version-identifiers)

@dstansby dstansby changed the title Change versioning policy to allow API breaking in minor versions Change versioning policy to allow API breaking in minor versions? Mar 5, 2025
@dstansby
Copy link
Contributor Author

dstansby commented Mar 5, 2025

Having four version numbers feels really weird to me - I guess because it gets more complicated, and no-one else does it? But I guess we could, and have zarr.x.y.z, where zarr is the data spec version, and x.y.z are the usual semantic version numbers. So then we'd move the current version to 3.0.0.4, and put breaking changes in a 3.1.0.0 release.

If we stick to current policy we'd have a choice between doing a v4 soon (and in general, major versions often), or maintaining a v4 branch and v3 in parallel branch with breaking changes only applied to the v4 branch, while we build up a set of such breaking changes to release at once.

@d-v-b
Copy link
Contributor

d-v-b commented Mar 5, 2025

The versioning guidelines states a conditional: if we add support for a new zarr format, then we must increment the major version of the library. But that's the only link between the library version and format version. So it's fine for us to do a major version increment without supporting a new zarr format, and the guidelines state as much:

Note however that the major version number of the Zarr library may not always correspond to the spec version number. For example, Zarr versions 2.x, 3.x, and 4.x might all implement the same version of the storage spec and thus maintain data format compatibility, although they will not maintain API compatibility.

So I don't think we need to change the versioning policy; we just need to decide when we want to make a v4 release.

@normanrz
Copy link
Member

normanrz commented Mar 5, 2025

I don't think we would even need to increment the major version if we supported a new Zarr version, if it is unnoticable by users. I think the major releases should indicate "big" changes which would imply larger changes that users need to make to migrate. Smaller breaking changes could go into minor releases with "enough" notice in terms of deprecation warnings. We should specify "big" and "enough" here.

@d-v-b
Copy link
Contributor

d-v-b commented Mar 5, 2025

I don't think we would even need to increment the major version if we supported a new Zarr version, if it is unnoticable by users.

I agree with this in principle but the current versioning guidelines say otherwise:

If a release of the Zarr library implements a different version of the storage specification, then the major version number of the Zarr library should be incremented.

It might be worth removing this statement from the guidelines. I doubt that will trouble anyone.

@TomAugspurger
Copy link
Contributor

It might be worth removing this statement from the guidelines. I doubt that will trouble anyone.

+1. IMO, the versioning of zarr-python the library should be tied to the source code, rather than the spec version.

And for folks who haven't read it, https://hynek.me/articles/semver-will-not-save-you/ is a nice thing to keep in mind when discussing semver. I don't have a super strong preference for whether we try to stick strictly to semver vs. something else, as long as

  1. we're clear about it in our versioning policy
  2. we're considerate to downstream users (both individuals and libraries building on zarr-python) by
  • preserving backwards compatibility where possible
  • giving notice of deprecations through FutureWarnings
  • giving users and libraries time to adapt to those deprecations

The last bit about time is important. We could release breaking changes daily under any versioning scheme, but doing that would be too disruptive to downstream users.

@dcherian
Copy link
Contributor

dcherian commented Mar 5, 2025

+1. IMO, the versioning of zarr-python the library should be tied to the source code, rather than the spec version.

I agree. the use of zarr_format i think makes it easy to untangle these two.

I agree with the other two proposals

@jni
Copy link
Contributor

jni commented Mar 6, 2025

for folks who haven't read it, hynek.me/articles/semver-will-not-save-you is a nice thing to keep in mind when discussing semver.

I'm a big fan of Jacob Tomlinson's EffVer ever since I read his blogpost proposing it. Matplotlib has officially adopted it. It's basically "realistic semver".

I think point 2 in Tom's post ("we're considerate to downstream users") is very important. I read @d-v-b's comments of "whatever policy lets me push my breaking changes" as overly flippant. But reading #2819, these are minor breaking changes that ~most/no users will notice. In line with EffVer, this would only be a minor version bump: downstream packages will probably not have to change anything, and those that do will have to change very little.

So, in short, my vote/recommendation:

Oh, and yeah, we shouldn't need to update the software version with the spec version, that's weird. 😅

@jhamman
Copy link
Member

jhamman commented Mar 8, 2025

Reporting out on what was discussed today at the dev meeting:

  • broad agreement that the current version update policy should be relaxed
  • minor breaking challenges (especially those that fix bugs or API flaws) are not the sort of thing that should trigger a major bump (subtext: Make create_array signatures consistent #2819 fits into this category)

Does anyone on this thread want to take a cut at the policy update in the docs?


I'm a big fan of Jacob Tomlinson's EffVer ever since I read his blogpost proposing it. Matplotlib has matplotlib/matplotlib#27702. It's basically "realistic semver".

🙌 thanks @jacobtomlinson for writing this up. Super helpful :)

@d-v-b
Copy link
Contributor

d-v-b commented Mar 8, 2025

Does anyone on this thread want to take a cut at the policy update in the docs?

i'm interested. I can find time for this in the next few days. I think this should probably also come with a public announcement, e.g. a blog post?

@d-v-b d-v-b linked a pull request Mar 14, 2025 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants