You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
synchronize the Helm Pool CRD with PoolStatus.Updated
make pool buffer calculations use Ready, unallocated pods while retaining Pending pods as in-flight capacity
bound scale-down batches with scaleStrategy.maxUnavailable and wait for deletions to finish
delete least-useful idle pods first and preserve stable Ready capacity
count terminating pods against PoolMax
keep status updates running while scale expectations are pending
Root cause
Six behaviors amplified one another under slow-starting runtimes: the Helm CRD pruned status.updated, buffer calculations treated Pending pods as available, the buffer target jumped to its midpoint outside the configured band, scale-down was unbounded, terminating pods disappeared from occupancy accounting, and scale-down selected the oldest Ready pods first. Together these caused a hot reconcile loop and sustained pod create/delete oscillation.
Impact
The controller now converges without repeatedly replacing usable pool capacity. Existing Pool manifests remain compatible: no CRD fields, CLI flags, or Helm values were added. The existing scaleStrategy.maxUnavailable setting now limits both scale-up and scale-down batch sizes; the default remains 25%.
Validation
KUBEBUILDER_ASSETS="..." go test ./internal/controller -count=1 (36 envtest specs)
targeted pool scaling stability unit tests
go test ./internal/utils/expectations/... -count=1
go test ./... -run '^$'
go vet ./internal/controller ./internal/utils/expectations/...
helm lint charts/opensandbox-controller
rendered Helm CRD contains .status.updated and the UPDATED printer column
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
component/k8sFor kubernetes runtimedocumentationImprovements or additions to documentationsize/LDenotes a PR that changes 100-499 lines, ignoring generated files.
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PoolStatus.UpdatedscaleStrategy.maxUnavailableand wait for deletions to finishPoolMaxRoot cause
Six behaviors amplified one another under slow-starting runtimes: the Helm CRD pruned
status.updated, buffer calculations treated Pending pods as available, the buffer target jumped to its midpoint outside the configured band, scale-down was unbounded, terminating pods disappeared from occupancy accounting, and scale-down selected the oldest Ready pods first. Together these caused a hot reconcile loop and sustained pod create/delete oscillation.Impact
The controller now converges without repeatedly replacing usable pool capacity. Existing Pool manifests remain compatible: no CRD fields, CLI flags, or Helm values were added. The existing
scaleStrategy.maxUnavailablesetting now limits both scale-up and scale-down batch sizes; the default remains 25%.Validation
KUBEBUILDER_ASSETS="..." go test ./internal/controller -count=1(36 envtest specs)go test ./internal/utils/expectations/... -count=1go test ./... -run '^$'go vet ./internal/controller ./internal/utils/expectations/...helm lint charts/opensandbox-controller.status.updatedand theUPDATEDprinter columnpnpm docs:buildgit diff --checkCloses #1423