Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roundrobin: Delegate subchannel creation to pickfirst #7966

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

arjan-bal
Copy link
Contributor

@arjan-bal arjan-bal commented Dec 24, 2024

This PR switches round round robin to defer to pick first instead of creating and handling SubConns itself. This is part of Dualstack, and makes this a petiole. This PR makes roundrobin a wrapper around the endpoint sharding policy that implements the roundrobin logic.

RELEASE NOTES:

  • balancer/roundrobin: Round Robin uses pick first child policies to manage endpoints instead of creating SubConns.
  • balancer/roundrobin: Round Robin uses the Endpoints field instead of the Addresses field in resolver state updates.

@arjan-bal arjan-bal added Type: Feature New features or improvements in behavior Area: Resolvers/Balancers Includes LB policy & NR APIs, resolver/balancer/picker wrappers, LB policy impls and utilities. labels Dec 24, 2024
@arjan-bal arjan-bal added this to the 1.70 Release milestone Dec 24, 2024
@arjan-bal arjan-bal changed the title roundrobin: Delegate subchannel endpoint to pickfirst roundrobin: Delegate subchannel creation to pickfirst Dec 24, 2024
Copy link

codecov bot commented Dec 24, 2024

Codecov Report

Attention: Patch coverage is 89.28571% with 3 lines in your changes missing coverage. Please review.

Project coverage is 82.29%. Comparing base (9afb49d) to head (84556fc).

Files with missing lines Patch % Lines
balancer/roundrobin/roundrobin.go 86.36% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7966   +/-   ##
=======================================
  Coverage   82.29%   82.29%           
=======================================
  Files         387      387           
  Lines       39065    39067    +2     
=======================================
+ Hits        32147    32152    +5     
- Misses       5582     5602   +20     
+ Partials     1336     1313   -23     
Files with missing lines Coverage Δ
balancer/weightedroundrobin/balancer.go 83.42% <100.00%> (-0.80%) ⬇️
internal/testutils/balancer.go 83.85% <100.00%> (+1.31%) ⬆️
balancer/roundrobin/roundrobin.go 87.50% <86.36%> (-12.50%) ⬇️

... and 28 files with indirect coverage changes

@arjan-bal arjan-bal force-pushed the roundrobin-delegate-to-pf branch from 01ef7af to 3a594f6 Compare December 26, 2024 03:52
@arjan-bal arjan-bal requested a review from dfawley December 26, 2024 03:52
@arjan-bal arjan-bal added Type: Internal Cleanup Refactors, etc and removed Type: Feature New features or improvements in behavior labels Dec 26, 2024
balancer/pickfirst/pickfirstleaf/pickfirstleaf.go Outdated Show resolved Hide resolved

func init() {
balancer.Register(newBuilder())
var err error
endpointShardingLBConfig, err = endpointsharding.ParseConfig(json.RawMessage(endpointsharding.PickFirstConfig))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in the other PR, too.

Probably we should have endpointsharding produce a parsed PF config if it's going to be used frequently?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a nice idea. I've raised a PR with this change: #8007

PTAL, I'll rebase both the PRs once #8007 is merged.

balancer/roundrobin/roundrobin.go Outdated Show resolved Hide resolved
@@ -173,7 +175,9 @@ func (s) TestRoundRobin_NewAddressWhileBlocking(t *testing.T) {

// Send a resolver update with a valid backend to push the channel to Ready
// and unblock the above RPC.
r.UpdateState(resolver.State{Addresses: []resolver.Address{{Addr: backends[0].Address}}})
r.UpdateState(resolver.State{Endpoints: []resolver.Endpoint{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needed to be done eventually. But maybe for similar things in the future...it could have been easier for this PR (and may still be easier when dealing with future migration PRs) to make the manual resolver's UpdateState and InitialState methods set Endpoints according to the default behavior if it's unset.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change was not required since the code to populate endpoints based using addresses is present in resolver_wrapper.go:

grpc-go/resolver_wrapper.go

Lines 125 to 132 in f35fb34

if s.Endpoints == nil {
s.Endpoints = make([]resolver.Endpoint, 0, len(s.Addresses))
for _, a := range s.Addresses {
ep := resolver.Endpoint{Addresses: []resolver.Address{a}, Attributes: a.BalancerAttributes}
ep.Addresses[0].BalancerAttributes = nil
s.Endpoints = append(s.Endpoints, ep)
}
}

I've reverted changes to tests that use the manual resolver. In tests that call balancer.UpdateClientConnState directly, without creating a real channel, conversion from addresses to endpoints is required.

balancer/weightedtarget/weightedtarget_test.go Outdated Show resolved Hide resolved
xds/internal/balancer/priority/balancer_test.go Outdated Show resolved Hide resolved
xds/internal/balancer/priority/balancer_test.go Outdated Show resolved Hide resolved
xds/internal/balancer/priority/balancer_test.go Outdated Show resolved Hide resolved
@dfawley dfawley assigned arjan-bal and unassigned dfawley Jan 9, 2025
@arjan-bal arjan-bal assigned dfawley and unassigned arjan-bal Jan 13, 2025
@arjan-bal arjan-bal requested a review from dfawley January 13, 2025 12:27
Comment on lines 80 to 85
func (b *rrBalancer) ExitIdle() {
// Should always be ok, as child is endpoint sharding.
if ei, ok := b.child.(balancer.ExitIdler); ok {
ei.ExitIdle()
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still necessary, unfortunately, as only balancer.Balancer functions are transferred when embedding.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted.

balancer/weightedtarget/weightedtarget_test.go Outdated Show resolved Hide resolved
@dfawley dfawley assigned arjan-bal and unassigned dfawley Jan 13, 2025
@arjan-bal
Copy link
Contributor Author

Closing for now and prioritizing the changes to make Ringhash a petiole policy. Will also make the changes to have the metrics recorder retrieved from a method on the ClientConn as per the offline discussion. Will re-open when I'm back on this.

@arjan-bal arjan-bal closed this Jan 17, 2025
@arjan-bal arjan-bal reopened this Feb 6, 2025
@arjan-bal arjan-bal assigned dfawley and unassigned arjan-bal Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Resolvers/Balancers Includes LB policy & NR APIs, resolver/balancer/picker wrappers, LB policy impls and utilities. Type: Internal Cleanup Refactors, etc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants