[CRD] Mark ray.io/v1alpha1 as a deprecated API version - #5122
Conversation
Add `+kubebuilder:deprecatedversion:warning` to the v1alpha1 RayCluster, RayJob, and RayService root types so the API server emits `deprecated: true` and a `deprecationWarning` for that version. `kubectl` now warns on `ray.io/v1alpha1` manifests instead of accepting them silently. v1alpha1 has been served but not stored since #1482 set v1 as the storage version in October 2023. Its Go types last changed functionally in #1788 (December 2023), and nothing in the operator imports them: the manager scheme registers only rayv1 and the generated clientset contains only v1. Users on v1alpha1 therefore get a schema frozen at its December 2023 field set, with newer fields pruned on that request path, and no signal that this is happening. This change is not breaking. It adds a warning and removes nothing. Also add a `Deprecated:` doc comment to each type so Go consumers get a staticcheck SA1019 signal, and regenerate the CRDs, the Helm chart copies, and docs/reference/api.md. Signed-off-by: Douglas Strodtman <douglas@anyscale.com>
|
cc @kevin85421 @andrewsykim for review. @alculquicondor @tenzen-y — tagging you because you were on the #1784 thread that reverted the v1alpha1 removal in December 2023. Nothing here removes anything; this only adds the deprecation warning that step never got. But since that revert was made on Kueue's behalf, you should know the state has changed: One heads-up for anyone looking at CI: the |
Thank you for tagging me here. Yes, Kueue currently doesn't rely on Ray alpha API. I'm ok with removing alpha api anytime. Please also check with other Kueue core team members |
|
Note I've also started a downstream cleanup in Ray docs: ray-project/ray#65336 @andrewsykim Appreciate your review there, and will follow up with other Ray maintainers about making sure all the tests are configured correctly. |
machichima
left a comment
There was a problem hiding this comment.
LGTM! Thank you.
Created a issue to track the deprecation and remove progress: #5124
|
Could we also update kuberay/ray-operator/apis/ray/v1alpha1/doc.go Lines 1 to 2 in 7af74cb Maybe change to // Package v1alpha1 contains API Schema definitions for the ray v1alpha1 API group.
// +groupName=ray.io
//
// Deprecated: ray.io/v1alpha1 is frozen at its December 2023 feature set. Use
// github.com/ray-project/kuberay/ray-operator/apis/ray/v1 instead.
package v1alpha1 |
Why are these changes needed?
ray.io/v1alpha1is served but not stored, and has been since #1482 madev1the storage version in October 2023. Today nothing tells users that. There's nodeprecatedflag on the version, nodeprecationWarning, and no conversion webhook, sokubectl applyof aray.io/v1alpha1manifest succeeds with no output at all.That silence is the problem, because the two versions are far apart now:
ray.io/v1exposes 63 exported types;ray.io/v1alpha1exposes 27.authOptions,tlsOptions,networkPolicy,gcsFaultToleranceOptions,historyServerOptions,deletionPolicy, the RayCluster and RayService upgrade strategies, andRayCronJob.RayClusterSpechas 14 fields in v1 and 7 in v1alpha1.None of the CRDs declare a
conversion:stanza, so the strategy isNoneand fields absent from the v1alpha1 schema are pruned on that request path. A user who applies a v1alpha1 RayCluster withtlsOptionsset gets a cluster without TLS and no error.Worth flagging that this isn't just an unfortunate side effect. The CRD versioning docs describe
strategy: Noneas being for the case where all served versions share an identical schema, since the only thing the API server rewrites is theapiVersionfield. Ours diverged a long time ago — 14 fields vs. 7 onRayClusterSpecalone. Serving two materially different schemas through a no-op converter is what produces the silent pruning, so the behavior is the predictable consequence of the current configuration rather than a bug in it.The v1alpha1 Go types also aren't wired into anything anymore.
main.goregisters onlyrayv1in the manager scheme, the generated clientset underpkg/client/.../typed/ray/contains onlyv1, and no non-generated file in the repo importsapis/ray/v1alpha1. The package exists socontroller-genstill emits the schema. Its last functional change was #1788 on 2023-12-29.This PR adds the missing signal. It doesn't remove anything.
What changed
Added
+kubebuilder:deprecatedversion:warning=...to the three v1alpha1 root types, plus a GoDeprecated:doc comment on each, then regenerated.The generated CRD change is three stanzas, one per kind:
No schema churn beyond that.
The
Deprecated:doc comments do double duty. They don't reach the CRD, becauseCRD_OPTIONSsetsmaxDescLen=0, but they do reachdocs/reference/api.md, and they give Go consumers astaticcheckSA1019 warning when they import the v1alpha1 types.Files:
ray-operator/apis/ray/v1alpha1/{raycluster,rayjob,rayservice}_types.goDeprecated:commentray-operator/config/crd/bases/ray.io_ray{clusters,jobs,services}.yamlmake manifestshelm-chart/kuberay-operator/crds/ray.io_ray{clusters,jobs,services}.yamlmake helmdocs/reference/api.mdmake api-docsThis is the deprecation step of a sequence that stalled
Framing this as "a new deprecation" undersells it. The standard removal runway for a CRD version is four steps, stated most explicitly in Gateway API's versioning policy:
Where KubeRay actually sits:
served: false, still present in the CRDStep 1 landed almost three years ago and step 2 was never taken. That's the whole reason there's no user-visible signal today, and it's why I'd frame this as finishing something rather than starting it.
Should v1alpha1 be removed instead?
I think yes, eventually, and I'd like maintainer input on the timing. I'm not proposing removal in this PR because step 3 and step 4 need a deprecation period that has never formally started — but I don't think "keep it indefinitely" is the alternative either.
The precedent says removal is normal. Kubernetes' own deprecation policy Rule #4a says alpha versions "may be removed in any release without prior deprecation notice." That policy governs core APIs rather than CRDs, so it isn't binding here, but it's the rung
v1alpha1sits on. Two comparable projects, both with larger blast radius than KubeRay:v1alpha2,v1alpha3,v1beta1v1alpha3; v1.7.0 (2024-04-16) droppedv1alpha4The spec argument is the strongest one. As noted above,
strategy: Noneis specified for versions sharing one schema, and ours haven't for years. The two ways back into spec are a conversion webhook or removal. Writing and maintaining a converter for a version nobody develops against is hard to justify, which leaves removal.But a straight removal PR still shouldn't go first, for two reasons.
First, downstream consumers. #1771 deleted v1alpha1 in December 2023 and #1784 reverted it four days later because Kueue couldn't straddle both versions; the plan recorded there was removal in 1.2.0, and we're on 1.6.2. Kueue itself is resolved —
kubernetes-sigs/kueuenow has 43 references toapis/ray/v1and zero tov1alpha1. The class of problem isn't:ray.io/v1alpha1, including manifests inawslabs/ai-on-eks,Azure-Samples/aks-labs,data-prep-kit/data-prep-kit, andproject-codeflare/multi-cluster-app-dispatcher. Some of those hits areconfig.ray.io/v1alpha1, a different and current API group, so treat that as an upper bound.koordinator-sh/koord-queue, last pushed 2026-04-27, whosepkg/jobext/handles/rayjob.goreconcilesrayv1alpha1.RayJob. Same shape as the Kueue problem in 2023. TheDeprecated:doc comments in this PR are what would give that project astaticchecksignal.Second,
status.storedVersions. Removal isn't purely a code deletion: the API server rejects removing a version fromspec.versionswhile it remains instatus.storedVersions, and that field doesn't self-clean. Any cluster that installed KubeRay before v1.0 still listsv1alpha1there until every object is rewritten and the status is patched. cert-manager handled this by making it a hard user prerequisite in their 1.6 to 1.7 upgrade notes rather than shipping automation. RayClusters are much shorter-lived than Certificates so most clusters will already be clean, but the eventual removal PR needs an upgrade note with a verification command, not just the deletion.KubeRay's own published policy suggests the cadence. The API reference page in Ray docs already carries a "KubeRay API compatibility and guarantees" section, which says maintainers "preserve the right to mark fields as deprecated and remove functionality associated with deprecated fields after a minimum of two minor releases."
That clause is scoped to fields within v1, and the section is silent on non-v1 API versions, so it doesn't settle this on its own. But two minor releases is the project's own stated interval for deprecate-then-remove, and applying it here gives a defensible timeline without importing anyone else's: deprecate in the next release, then
served: falseand removal from 1.9 onward. That's a stronger basis than the Gateway API runway I quoted above, which I'd treat as the sequence of steps rather than the schedule.So my suggestion is: land the warning now, take step 3 once it's been out for the interval above, then step 4. Happy to write either. If you'd rather go straight to removal, or name a target release now, say so and I'll follow up with that PR instead.
The one documentation gap worth naming: that compatibility section covers field stability inside v1 and says nothing about the lifecycle of API versions. Extending it to cover version deprecation and removal would have prevented this drift, but it lives in the Ray repo rather than here, so it's a separate change with a different reviewer. I'm happy to draft it if maintainers want it.
Related issue number
None. Happy to file one if you'd prefer this tracked as an issue first.
Adjacent but separate: #5091 covers the anchor collision between the v1 and v1alpha1 sections of
docs/reference/api.md. Removing v1alpha1 from generation would resolve it, which is part of why the removal question matters, but this PR doesn't touch that.Labels
doc-updates-requiredlabel.breaking-changelabel.Not breaking: this adds a warning and removes nothing. v1alpha1 remains served and every existing manifest still applies.
Checks
Manual test instructions
Confirm the generated output and that the API server surfaces the warning.
Expected:
Warning: ray.io/v1alpha1 RayCluster is deprecated; use ray.io/v1 RayCluster. v1alpha1 receives no new fields and will be removed in a future release.The object is still created.The same manifest with
apiVersion: ray.io/v1produces no warning.Verified locally:
make manifests,make helm, andmake api-docsall produce exactly the diff in this PR, andgo build ./...andgo vet ./apis/ray/v1alpha1/are clean withcontroller-gen@v0.16.5.