internal/xds: move the LDS and RDS watchers to dependency manager #8651

eshitachandwani · 2025-10-14T16:23:20Z

This PR moves the LDS and RDS watchers to dependency manager without chaning the current functionality or behaviour. This is a part of implementation of gRFC A74.

RELEASE NOTES: None

codecov · 2025-10-15T06:28:38Z

Codecov Report

❌ Patch coverage is 69.82249% with 51 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.97%. Comparing base (ae62635) to head (c279455).
⚠️ Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/xds/xdsdependencymanager/watch_service.go	65.00%	14 Missing and 7 partials ⚠️
...xds/xdsdependencymanager/xds_dependency_manager.go	78.94%	10 Missing and 6 partials ⚠️
internal/xds/xdsclient/xdsresource/xdsconfig.go	0.00%	8 Missing ⚠️
internal/grpctest/tlogger.go	73.91%	4 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8651      +/-   ##
==========================================
+ Coverage   81.21%   82.97%   +1.76%     
==========================================
  Files         416      421       +5     
  Lines       41002    32464    -8538     
==========================================
- Hits        33298    26936    -6362     
+ Misses       6226     4129    -2097     
+ Partials     1478     1399      -79

Files with missing lines	Coverage Δ
internal/xds/xdsdependencymanager/logging.go	`100.00% <100.00%> (ø)`
internal/grpctest/tlogger.go	`74.82% <73.91%> (+3.90%)`	⬆️
internal/xds/xdsclient/xdsresource/xdsconfig.go	`0.00% <0.00%> (ø)`
...xds/xdsdependencymanager/xds_dependency_manager.go	`78.94% <78.94%> (ø)`
internal/xds/xdsdependencymanager/watch_service.go	`65.00% <65.00%> (ø)`

... and 365 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

easwars · 2025-10-15T22:09:37Z

The tests are failing. Is this ready for review?

easwars · 2025-10-17T18:12:19Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+// XDSConfig holds the complete and resolved xDS resource configuration
+// including LDS, RDS, CDS and endpoints.


Suggested change

// XDSConfig holds the complete and resolved xDS resource configuration

// including LDS, RDS, CDS and endpoints.

// XDSConfig holds the complete gRPC client-side xDS configuration

// containing all necessary resources.

easwars · 2025-10-17T18:14:51Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+// including LDS, RDS, CDS and endpoints.
+type XDSConfig struct {
+	// Listener is the listener resource update
+	Listener ListenerUpdate


We are moving the ResourceChanged methods on the resource watchers to accept a pointer to the update struct (instead of accepting the update by value). So, I think it would make sense for us to store them as pointers here as well.

See: #8652

easwars · 2025-10-17T18:17:34Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+// XDSConfig holds the complete and resolved xDS resource configuration
+// including LDS, RDS, CDS and endpoints.
+type XDSConfig struct {
+	// Listener is the listener resource update


Nit: Let's try to consistently use the word configuration or config instead of update in these docstrings.

So, maybe something like:
// Listener holds the listener configuration.

easwars · 2025-10-17T18:19:02Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	// RouteConfig is the route configuration resource update. It will be
+	// populated even if RouteConfig is inlined into the Listener resource.


Suggested change

// RouteConfig is the route configuration resource update. It will be

// populated even if RouteConfig is inlined into the Listener resource.

// RouteConfig holds the route configuration. It will be

// populated even if the route configuration was inlined into the Listener resource.

easwars · 2025-10-17T18:24:34Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	// VirtualHost is the virtual host from the route configuration matched with
+	// dataplane authority .


Maybe?

Suggested change

// VirtualHost is the virtual host from the route configuration matched with

// dataplane authority .

// VirtualHost selected from the route configuration whose domain field

// offers the best match against the provided dataplane authority.

easwars · 2025-10-17T18:26:06Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	// Clusters maps the cluster name with the ClusterResult which will have
+	// either the cluster configuration or error. It will have an error status
+	// if either
+	//
+	// (a) there was an error and we did not already have a valid resource or
+	//
+	// (b) the resource does not exist.


How about making it much simpler and leaving more of the documentation to the individual structs.

// Clusters is a map from cluster name to its configuration.

easwars · 2025-10-17T23:25:38Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	Clusters map[string]*ClusterResult
+}
+
+// ClusterResult contains either a cluster's configuration or an error.


Something like?

// ClusterResult contains a cluster's configuration when we receive a // valid resource from the management server. It contains an error when: // - we receive an invalid resource from the management server and // we did not already have a valid resource or // - the cluster resource does not exist on the management server

easwars · 2025-10-17T23:25:57Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	Err    error
+}
+
+// ClusterConfig contains cluster configuration for a single cluster.


Nit: ClusterConfig contains configuration for a single cluster.

easwars · 2025-10-17T23:26:13Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+// ClusterConfig contains cluster configuration for a single cluster.
+type ClusterConfig struct {
+	Cluster         ClusterUpdate   // Cluster configuration. Always present.
+	EndpointConfig  EndpointConfig  // Endpoint configuration for leaf clusters which will of type EDS or DNS.


I think just "Endpoint configuration for leaf clusters" should suffice.

easwars · 2025-10-17T23:26:24Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	AggregateConfig AggregateConfig // List of children for aggregate clusters.
+}
+
+// AggregateConfig contains a list of leaf cluster names.


This need not technically be all leaf clusters. Aggregate clusters can have children that are aggregate clusters as well.

easwars · 2025-10-17T23:26:39Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	LeafClusters []string
+}
+
+// EndpointConfig contains resolved endpoints for a leaf cluster either from DNS


Technically, this contains more than just resolved endpoints, at least for the EDS case. So, maybe the comment can be more generic.

// EndpointConfig contains configuration corresponding to the endpoints in a cluster.

And we should also clarify that only one of three fields can be populated at any given point in time.

Or is it the case that ResolutionNote can have a non-nil error even when one of EDSUpdate or DNSEndpoints is set? If so, we need to clarify that.

Yes , the Resolution note will also be set when we have an ambient error along with the old endpoints.

easwars · 2025-10-17T23:29:19Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// including LDS, RDS, CDS and EDS and sends update once we have all the
+// resources and sends an error when we get error in listener or route
+// resources.
+func New(listenername, dataplaneAuthority string, xdsClient xdsclient.XDSClient, watcher ConfigWatcher) *DependencyManager {


s/listenername/listenerName

easwars · 2025-10-17T23:29:30Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// resources and sends an error when we get error in listener or route
+// resources.
+func New(listenername, dataplaneAuthority string, xdsClient xdsclient.XDSClient, watcher ConfigWatcher) *DependencyManager {
+	// Builds the dependency manager and starts the listener watch.


Nit: nix this comment as this is very obvious that you are creating the struct here. And the listener watch is not started here though.

easwars · 2025-10-17T23:29:48Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// ConfigWatcher is notified of the XDSConfig resource updates and errors that
+// are received by the xDS client from the management server. It only receives a
+// XDSConfig update after all the xds resources have been received.


How about:

// ConfigWatcher is the interface for consumers of aggregated xDS configuration
// from the DependencyManager. The only consumer of this configuration is
// currently the xDS resolver.

easwars · 2025-10-17T23:29:59Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// ConfigWatcher is notified of the XDSConfig resource updates and errors that
+// are received by the xDS client from the management server. It only receives a
+// XDSConfig update after all the xds resources have been received.
+type ConfigWatcher interface {


Please consider moving this to the top of the file so that the methods of the DependencyManager stay together and are not mixed in with another type's definition.

easwars · 2025-10-17T23:30:33Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+
+func (m *DependencyManager) maybeSendUpdate() {
+	if m.logger.V(2) {
+		m.logger.Infof("Sending update to watcher: Listener: %v, RouteConfig: %v", pretty.ToJSON(m.currentListenerUpdate), pretty.ToJSON(m.currentRouteConfig))


We've had many performance problems with using pretty.JSON for printing structs. I would recommend using %+v or some other native formatting directive instead.

Another thing to consider is also whether the xDS resolver also outputs this log. If so, we don't want the same information being repeated twice.

Yeah I think adding this is resolver looks better.

easwars

I haven't looked at the tests yet. But I guess these comments will give you enough to make progress.

easwars · 2025-10-17T23:53:50Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+type ConfigWatcher interface {
+	// OnUpdate is invoked by the dependency manager to provide a new,
+	// validated xDS configuration to the watcher.
+	OnUpdate(xdsresource.XDSConfig)


Given that we are changing the resource watcher APIs to accept pointers to resource update structs, it would make more sense to store pointers to them in the XDSConfig struct as well.

And continuing in that same vein, we could return a pointer to the XDSConfig struct from here. Also, it would make sense to document that the watcher must not modify the returned XDSConfig and that it should read-only for the watcher.

easwars · 2025-10-17T23:57:27Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	// OnError is invoked when an error is received in listener or route
+	// resource. This includes cases where:
+	//  - The listener or route resource watcher reports a resource error.
+	//  - The received listener resource is a socket listener, not an API listener - TODO :  This is not yet implemented, tracked here #8114


You could generalize this and specify that any resource validations performed at the DependencyManager that fail, also lead to OnError being invoked on the watcher.

That is not necessarily true, since cluster error are stored separately in a struct and endpoint error are stored in resolution note. Only errors in Listener and route resource are sent using OnError function.

easwars · 2025-10-17T23:58:05Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	OnError(error)
+}
+
+func (m *DependencyManager) maybeSendUpdate() {


Why is this called maybeSendUpdate? Under what conditions will it not send an update? Can this be captured in its docstring.

We want to check if the whole cluster tree is resolved , even if we have one leaf endpoint missing, we might not send the update and the checks for that will go in this function, and also c++ and java both have the same name so that got stuck in my head... Let me know if we should change it?

easwars · 2025-10-17T23:59:34Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// Only executed in the context of a serializer callback.
+func (m *DependencyManager) onListenerResourceUpdate(update *xdsresource.ListenerUpdate) {
+	if m.logger.V(2) {
+		m.logger.Infof("Received update for Listener resource %q: %v", m.ldsResourceName, pretty.ToJSON(update))


For all these usages of pretty.JSON, please consider switching them to native formatting directives. Experiment with a few of them like %v, %+v, %#v, %+V, %#V and see which one provides the best output and use that.

easwars · 2025-10-18T00:04:12Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+}
+
+func (m *DependencyManager) applyRouteConfigUpdate(update xdsresource.RouteConfigUpdate) {
+	matchVh := xdsresource.FindBestMatchingVirtualHost(m.dataplaneAuthority, update.VirtualHosts)


s/matchVh/matchVH to comply with Go initialisms.

easwars · 2025-10-18T00:13:33Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+// Only executed in the context of a serializer callback.
+func (m *DependencyManager) onListenerResourceError(err error) {
+	if m.logger.V(2) {
+		m.logger.Infof("Received resource error for Listener resource %q: %v", m.ldsResourceName, err)


I believe we have some code in the xDS client to ensure that the returned errors contain the xDS node ID. Could you please ensure that that property still holds. Thanks.

easwars · 2025-10-18T00:15:33Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+		m.rdsResourceName = ""
+		if m.routeConfigWatcher != nil {
+			m.routeConfigWatcher.stop()
+			m.routeConfigWatcher = nil
+		}


Don't we have to set the matching virtual host to nil here?

It would be nice if we have a method to do all the cleanup when a listener resource error or a listener resource update invalidates the previously received route config. I see similar code in onListenerResourceError, but that one sets the matching virtual host to nil as well.

We dont set to virtual host to nil because we are going to update the virtual host in the function below using the inline route resource that we get. And we set it to nil in `OnListenerResourceError because we want to invalidate the route resource. Here we are just updating it and cancelling just the watchers since we get the resource inline.

easwars · 2025-10-18T00:17:19Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	m.rdsResourceName = ""
+	m.currentVirtualHost = nil
+	m.routeConfigWatcher = nil
+	m.watcher.OnError(status.Errorf(codes.Unavailable, "Listener resource error : %v", err))


Again, if the watcher is going to be given status errors, we need to document that clearly along with what status codes are returned when. And why?

Removed. We do not need status code. I looked at C++ code and got confused between absl status codes and gRPC status codes.

easwars · 2025-10-18T00:18:25Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	if m.rdsResourceName != resourceName {
+		// Drop updates from canceled watchers.
+		return
+	}


I'm guessing we are going need code like this for cluster and endpoint watchers as well. Can we make this part of the watcher instead?

Sure! I have changed the listener and route watchers let me know if it looks good.

easwars · 2025-10-18T00:18:36Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	//If update is not for the current watcher
+	if m.rdsResourceName != resourceName {
+		return
+	}


eshitachandwani · 2025-10-22T09:04:44Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	AggregateConfig AggregateConfig // List of children for aggregate clusters.
+}
+
+// AggregateConfig contains a list of leaf cluster names.


eshitachandwani · 2025-10-22T12:59:27Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+	LeafClusters []string
+}
+
+// EndpointConfig contains resolved endpoints for a leaf cluster either from DNS


Yes , the Resolution note will also be set when we have an ambient error along with the old endpoints.

eshitachandwani · 2025-10-23T06:13:59Z

internal/xds/xdsclient/xdsresource/xdsconfig.go

+
+	// RouteConfig is the route configuration resource update. It will be
+	// populated even if RouteConfig is inlined into the Listener resource.
+	RouteConfig RouteConfigUpdate


Will change other resources to pointer after #8652 is merged

eshitachandwani · 2025-10-23T06:23:56Z

internal/xds/xdsdependencymanager/watch_service.go

+
+func (l *listenerWatcher) stop() {
+	l.cancel()
+	l.parent.logger.Infof("Canceling watch on Listener resource %q", l.resourceName)


Changed but I wanted to iknow where the log should be printed unconditionally where we should put a V(2) check, because I thought shutdown and cancel should be default becuase its useful information.

eshitachandwani · 2025-10-23T06:31:55Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	serializer       *grpcsync.CallbackSerializer
+	serializerCancel context.CancelFunc


From what I understand, the ResourceChanged and error methods call Parent's serializer (like here and here) which is going to be dependency manager now.

eshitachandwani · 2025-10-23T07:50:55Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	OnError(error)
+}
+
+func (m *DependencyManager) maybeSendUpdate() {


We want to check if the whole cluster tree is resolved , even if we have one leaf endpoint missing, we might not send the update and the checks for that will go in this function, and also c++ and java both have the same name so that got stuck in my head... Let me know if we should change it?

eshitachandwani · 2025-10-23T08:00:30Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+
+func (m *DependencyManager) maybeSendUpdate() {
+	if m.logger.V(2) {
+		m.logger.Infof("Sending update to watcher: Listener: %v, RouteConfig: %v", pretty.ToJSON(m.currentListenerUpdate), pretty.ToJSON(m.currentRouteConfig))


Yeah I think adding this is resolver looks better.

eshitachandwani · 2025-10-23T09:18:47Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+		m.rdsResourceName = ""
+		if m.routeConfigWatcher != nil {
+			m.routeConfigWatcher.stop()
+			m.routeConfigWatcher = nil
+		}


We dont set to virtual host to nil because we are going to update the virtual host in the function below using the inline route resource that we get. And we set it to nil in `OnListenerResourceError because we want to invalidate the route resource. Here we are just updating it and cancelling just the watchers since we get the resource inline.

eshitachandwani · 2025-10-23T12:16:52Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	m.rdsResourceName = ""
+	m.currentVirtualHost = nil
+	m.routeConfigWatcher = nil
+	m.watcher.OnError(status.Errorf(codes.Unavailable, "Listener resource error : %v", err))


Removed. We do not need status code. I looked at C++ code and got confused between absl status codes and gRPC status codes.

eshitachandwani · 2025-10-23T12:30:57Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	if m.rdsResourceName != resourceName {
+		// Drop updates from canceled watchers.
+		return
+	}


Sure! I have changed the listener and route watchers let me know if it looks good.

eshitachandwani · 2025-10-24T08:52:06Z

internal/xds/xdsdependencymanager/xds_dependency_manager.go

+	m.rdsResourceName = ""
+	m.currentVirtualHost = nil
+	m.routeConfigWatcher = nil
+	m.watcher.OnError(fmt.Errorf("listener resource error : %v", err))


Should we annotate the errors passed to other components too, or just make sure the are annotated with node id when they are actually printed?

LDS/RDS to deps manager

fbfb4b1

eshitachandwani added this to the 1.77 Release milestone Oct 14, 2025

eshitachandwani added Type: Internal Cleanup Refactors, etc Area: xDS Includes everything xDS related, including LB policies used with xDS. labels Oct 14, 2025

eshitachandwani and others added 4 commits October 14, 2025 22:12

remove const

66270ee

Merge branch 'grpc:master' into A74_LDS_RDS

d28cb9e

Change log

d80ee7a

resolve conflicts

3502dbf

easwars assigned eshitachandwani Oct 15, 2025

remove test

b696279

eshitachandwani requested a review from easwars October 16, 2025 10:51

change test

ed60964

eshitachandwani requested a review from arjan-bal October 17, 2025 05:19

eshitachandwani assigned easwars and arjan-bal and unassigned eshitachandwani Oct 17, 2025

easwars reviewed Oct 17, 2025

View reviewed changes

easwars reviewed Oct 18, 2025

View reviewed changes

eshitachandwani added 2 commits October 23, 2025 20:00

review comments

fff6013

cancelled watcher updates

58478e6

eshitachandwani commented Oct 24, 2025

View reviewed changes

fine changes

c279455

eshitachandwani commented Oct 24, 2025

View reviewed changes

eshitachandwani requested a review from easwars October 24, 2025 09:35

		// XDSConfig holds the complete and resolved xDS resource configuration
		// including LDS, RDS, CDS and endpoints.

		// RouteConfig is the route configuration resource update. It will be
		// populated even if RouteConfig is inlined into the Listener resource.

		// VirtualHost is the virtual host from the route configuration matched with
		// dataplane authority .

		serializer *grpcsync.CallbackSerializer
		serializerCancel context.CancelFunc

internal/xds: move the LDS and RDS watchers to dependency manager #8651

Are you sure you want to change the base?

internal/xds: move the LDS and RDS watchers to dependency manager #8651

Uh oh!

Conversation

eshitachandwani commented Oct 14, 2025

Uh oh!

codecov bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

easwars commented Oct 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eshitachandwani Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

easwars left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Oct 15, 2025 •

edited

Loading

eshitachandwani Oct 22, 2025 •

edited

Loading

eshitachandwani Oct 22, 2025 •

edited

Loading