Use node status.addresses for kubelet health-check - #331
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: kwohlfahrt The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @kwohlfahrt. Thanks for your PR. I'm waiting for a medik8s member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
📝 WalkthroughSummary by CodeRabbit
WalkthroughChangesKubelet address selection
Architecture-aware protoc download
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant SelfNodeRemediationConfig
participant Controller
participant DaemonSet
participant Manager
participant KubernetesNode
participant Kubelet
SelfNodeRemediationConfig->>Controller: Set PreferredAddressTypes
Controller->>DaemonSet: Render PREFERRED_ADDRESS_TYPES
DaemonSet->>Manager: Provide environment configuration
Manager->>KubernetesNode: Read status.addresses
Manager->>Kubelet: Try preferred addresses in order
Kubelet-->>Manager: Return health response
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/controlplane/manager.go`:
- Around line 184-205: Update isKubeletServiceRunningOnAddress to build the
kubelet endpoint with net.JoinHostPort(address, manager.kubeletPort) before
constructing the HTTPS URL, preserving valid formatting for both IPv4 and IPv6
addresses. Add a test using an IPv6 loopback listener to verify the address is
contacted successfully.
In `@Makefile`:
- Around line 409-412: Update the PROTOC_ARCH mapping to translate GOARCH values
ppc64le, s390x, and 386 to ppcle_64, s390_64, and x86_32 respectively, or make
the Makefile fail early with a clear unsupported-architecture error for unmapped
values before the protoc download runs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8eb1c779-5525-4ec1-a7d2-a6cdc73857ea
📒 Files selected for processing (11)
Makefileapi/v1alpha1/selfnoderemediationconfig_types.goapi/v1alpha1/zz_generated.deepcopy.gobundle/manifests/self-node-remediation.medik8s.io_selfnoderemediationconfigs.yamlconfig/crd/bases/self-node-remediation.medik8s.io_selfnoderemediationconfigs.yamlinstall/self-node-remediation-deamonset.yamlinternal/controller/selfnoderemediationconfig_controller.gointernal/controller/tests/config/selfnoderemediationconfig_controller_test.gointernal/controlplane/manager.gointernal/controlplane/manager_test.gointernal/controlplane/suite_test.go
kwohlfahrt
changed the title
status.addresses for kubelet health-check
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
05b17b1 to
1f44fdd
Compare
razo7
left a comment
razo7
left a comment
There was a problem hiding this comment.
Nice contribution @kwohlfahrt! The approach makes sense and it models after kube-apiserver's --kubelet-preferred-address-types which is the right pattern, backward compatibility is preserved, and test coverage is solid.
Before applying my ok-to-test please address the threads below
| Expect(len(dsContainers)).To(BeNumerically("==", 1)) | ||
| container := dsContainers[0] | ||
| envVars := getEnvVarMap(container.Env) | ||
| Expect(envVars["PREFERRED_ADDRESS_TYPES"].Value).To(Equal("InternalDNS,InternalIP")) |
There was a problem hiding this comment.
These assertions (lines 137 and 140) use bare Expect instead of g.Expect inside the Eventually(func(g Gomega) {...}) callback. When bare Expect fails, it panics immediately instead of allowing Eventually to retry.
The adjacent toleration test (line 153+) correctly uses g.Expect throughout — this block should match that pattern.
| Expect(len(dsContainers)).To(BeNumerically("==", 1)) | |
| container := dsContainers[0] | |
| envVars := getEnvVarMap(container.Env) | |
| Expect(envVars["PREFERRED_ADDRESS_TYPES"].Value).To(Equal("InternalDNS,InternalIP")) | |
| g.Expect(len(dsContainers)).To(BeNumerically("==", 1)) | |
| container := dsContainers[0] | |
| envVars := getEnvVarMap(container.Env) | |
| g.Expect(envVars["PREFERRED_ADDRESS_TYPES"].Value).To(Equal("InternalDNS,InternalIP")) |
| }) | ||
|
|
||
| It("should contact the kubelet via an IPv6 address", func() { | ||
| listener, err := net.Listen("tcp", "[::1]:0") |
There was a problem hiding this comment.
suggestion: This test will fail in CI environments without IPv6 loopback. Consider adding a skip guard:
listener, err := net.Listen("tcp", "[::1]:0")
if err != nil {
Skip("IPv6 loopback not available: " + err.Error())
}| return false | ||
| } | ||
|
|
||
| func (manager *Manager) isKubeletServiceRunningOnAddress(address string) bool { |
There was a problem hiding this comment.
pre-existing / follow-up: The http.Client created inside this function (line 213) has no Timeout, and the request uses no context.Context with deadline. If a target address accepts TCP but never responds TLS, this blocks indefinitely — and with the new multi-address iteration, a single hanging connection also blocks fallback to later address types.
Not a blocker for this PR (the no-timeout client is pre-existing on main), but worth a follow-up:
httpClient := &http.Client{
Transport: tr,
Timeout: 10 * time.Second,
}| var preferredAddressTypes []string | ||
| rawPreferredAddressTypes := os.Getenv("PREFERRED_ADDRESS_TYPES") | ||
| if rawPreferredAddressTypes != "" { | ||
| preferredAddressTypes = strings.Split(rawPreferredAddressTypes, ",") |
There was a problem hiding this comment.
nit: strings.Split doesn't trim whitespace. If someone manually patches the pod env var with "InternalIP, NodeName" (space after comma), " NodeName" won't match "NodeName".
Via the normal CRD→template path this can't happen (the join template produces no spaces), but for defensive robustness:
for i, t := range preferredAddressTypes {
preferredAddressTypes[i] = strings.TrimSpace(t)
}| return wrapWithInitError(err) | ||
| } | ||
| manager.setNodeRole(node) | ||
| manager.nodeAddresses = node.Status.Addresses |
There was a problem hiding this comment.
note: nodeAddresses is cached at init and never refreshed. If node addresses change during the pod's lifetime (IP rotation, secondary NIC), the kubelet check uses stale data.
In practice, this is fine as node addresses rarely change, and the SNR daemonset restarts on config changes. But it's worth documenting this in the CRD field description so users know a pod restart is needed to pick up address changes.
| return true | ||
| } | ||
| } else { | ||
| addressType := corev1.NodeAddressType(addressType) |
There was a problem hiding this comment.
nit: This shadows the outer addressType (string) with a same-named variable of type corev1.NodeAddressType. Functionally correct since it's scoped to the else block, but go vet -shadow would flag it. Consider renaming to nodeAddrType or similar.
2 participants
Why we need this PR
To support clusters where the node's name is not resolvable, but the addresses in
status.addressescan be used instead.Changes made
preferredAddressTypes) that allows selecting addresses fromstatus.addresses(and a specialNodeNamevalue, for the old behaviour, on by default)status.addresses, use the enabled ones when trying the kubelet health-checkMakefileto download the correctprotoconaarch64dev machinesWhich issue(s) this PR fixes
Closes #330.
Test plan
Added new unit tests for control-plane manager kubelet liveness checks, and new e2e tests for daemonset rendering.