Add IPvlan L2 driver for Alibaba Cloud HPN nodes - #206
Conversation
k8s-ci-robot
added
the
kind/feature
✅ Deploy Preview for dranet canceled.
|
|
Welcome @Bowser1704! |
k8s-ci-robot
added
needs-ok-to-test
|
Hi @Bowser1704. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
size/XXL
|
Happy to have you contributing @Bowser1704 This is very interesting as we're also working on a somewhat generic approach towards having IPVLAN as part of #63. @ngcxy is working on a POC at the moment and we can align to make sure it covers your use case too (from the looks of it, I think it would). |
|
/assign |
k8s-ci-robot
added
ok-to-test
f529806 to
fbb1e43
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Bowser1704 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
bb0eeb9 to
e351900
Compare
|
Thanks @gauravkghildiyal! I've actually implemented a more generic IPvlan driver in this PR — it's not tightly coupled to Alibaba Cloud. The cloud provider hint just signals when to enable it. Would love to align with @ngcxy's POC from #63 — if we can converge on the IPvlan approach, that would let Alibaba Cloud users onboard to dranet right away. Happy to iterate on the design together. |
k8s-ci-robot
added
the
release-note
bb28b7f to
48383e4
Compare
|
Thanks for the context. We'd very much like to get things work with Alibaba Cloud. But the approach that I think we should take with IPVLAN ( or MACVLAN and others) is also having the flexibility to share the parent interface across multiple pods and doing this in a more DRA native way (using things like DRA Consumable Capacity and others) This is something we are actively working towards and I'd highly encourage and request we wait for the work being done for #63. This is slightly opinionated, but it'd be awesome to get the IPVLAN support right from the get-go -- a modeling that allows for future extension like sharing between multiple pods. We'd appreciate your patience with this and would be happy to collaborate together so that all requirements are met. This PR is already a great demonstration of what exactly are the things we should have a solution for (in addition to things like device sharing). |
gauravkghildiyal
added
the
do-not-merge/hold
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
needs-rebase
kubernetes-prow
Bot
added
the
do-not-merge/work-in-progress
* Add SubInterfaceConfig, IPVlanConfig, and IPRangeConfig definitions to the NetworkConfig schema. * Support a list of IP ranges per subinterface, each expressed either as a CIDR block or as an explicit start/end address pair. * Define default configuration parameters and merging rules for subinterfaces, with user-provided values taking precedence and duplicates removed. * Implement validation logic for subinterface properties (IP ranges, CIDR formatting, start/end ordering, and IPVLAN modes/flags) along with unit tests.
* Implement LocalIPAM to allocate addresses from the configured IP ranges (CIDR or explicit start/end), returning one address per IP family chosen at random within the range bounds, while tracking the set of in-use addresses to prevent conflicts. * Roll back partial allocations on error, and add Release to return an address to the pool. * Add PodConfigStore.GetAllocatedIPs to collect the addresses recorded in the stored device configs, and seed LocalIPAM with them at NetworkDriver startup so previously assigned IPs are not reissued. * Add unit tests verifying IP allocation boundaries, randomized selection, conflict prevention, and lifecycle cleanups.
…laim hooks * Extend the prepareResourceClaim hook to process subinterface specs: * Assign interface name with the network type prefix. * Use LocalIPAM to automatically allocate IPs from the configured ranges when no static address is specified. * Implement source-based routing to generate custom-table routes and rules for the subinterface, when no custom routes and rules are configured. * Release allocated subinterface IPs in unprepareResourceClaim so addresses are returned to the pool on claim teardown. * Add unit tests validating IP allocation and source-based routing configuration for routes and rules.
* Plumb subinterface setup (createSubinterfaceInNS) and cleanup (nsDeleteSubinterface) into NRI runPodSandbox and stopPodSandbox lifecycle hooks. * Refactor common network configuration tasks (VRF, routes, neighbors, and rules) into a shared configureNetdevInNS helper. * Implement netlink utilities nsCreateSubinterface and nsDeleteSubinterface with support for IPVLAN creation (L2 mode, bridge flag), setting up link state, and configuring IPs. * Add integration tests (TestSubinterface_IPVlan) to verify end-to-end IPVLAN creation, MAC/MTU inheritance, IP assignment, and cleanup inside test network namespaces.
* Implement GetDeviceConfig to lookup GCE VM network interface details matching the device MAC address. * Add a shared cloudutil.IPRangeFromCIDR helper that derives explicit [start, end] allocation bounds from a CIDR, always excluding the network and broadcast addresses and supporting additional reserved counts at each end. * For IPv4, derive the range from the primary GCE Alias IP Range. * For IPv6, compute the range by appending a 16-bit marker (0xC0DE) to the parent interface's base prefix. * Add unit tests for the range helper, GCE device configuration lookups, and IPv6 prefix derivation.
Our address-inheritance fallback ran after PR241's own subinterface block, which already does `continue` (skipping the rest of the device loop) when Addresses is empty and IPRanges isn't set -- exactly our bond case. Move the inherit-from-parent fallback before the IPRanges/IPAM branch so it actually runs. Also clean up a duplicated block in subinterfaces_test.go left over from a bad edit during the merge.
…st addr
Switch the lingjun bond support to use PR241's native IPRanges/LocalIPAM
path instead of a custom address-inherit + strip/restore mechanism.
eflo reserves a fixed 16-address block (host bits
0000:000f:0000:0c00-0c0f, a /124) inside every RDMA NIC's own /64 for
that NIC's pod-facing subinterface -- disjoint from whatever address
the NIC already carries on the host. Deriving this range and handing
it to SubInterface.IPRanges means the pod's subinterface address is
never also live on the host, so there is nothing to strip or restore.
This mirrors exactly how the GCE provider derives its own IPv6
subinterface range (appending a fixed marker to the node's /64), just
with eflo's offset instead of GCE's 0xC0DE.
pkg/driver/{subinterfaces,nri_hooks,dra_hooks}.go are reverted to
pure PR241 -- no downstream changes needed there anymore.
48383e4 to
6af213c
Compare
kubernetes-prow
Bot
removed
the
needs-rebase
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Bowser1704 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
4 participants
What type of PR is this?
/kind feature
Introduce a configurable IPvlan driver that creates IPvlan slave
interfaces in the pod namespace while keeping the parent netdev on
the host. The implementation supports L2/L3/L3S modes, bridge/private/
vepa flags, and three addressing strategies (none, static,
parentIPv6PrefixPodIPv4).
What this PR does / why we need it:
Key changes:
neighbor copy options
pod netns on retry, tolerate EEXIST on address assignment
sorted link-scope-first for gateway reachability
degrading connectivity
generates the IPvlan preset for bond devices with global IPv6
Which issue(s) this PR is related to:
N/A
Special notes for your reviewer:
Does this PR introduce a user-facing change?