HDDS-15417. Handle mixed datanode versions on the replication path

[dombizita]

What changes were proposed in this pull request?

Datanodes act in a client server relationship between themselves for replication. In general the replication process is quite simple - commands are sent to a datanode hosting a container replica (the replication client), and it is told to push/upload the data to another node (the replication server). Incompatible changes in this area can be handled in a similar way to write requests.

SCM knows the apparent version of all Datanodes since their last heartbeat. The last known apparent version of the target/server Datanode can be included in the replicate command sent by SCM to the source/client Datanode. It is always the job of the newer component to handle compatibility, whether it involves software or apparent version. If the source/client Datanode is newer in software or apparent version, it must "downgrade" its request to the lowest common apparent version supported by the server indicated by the SCM command. A newer target/server Datanode will not make incompatible changes to existing APIs, so old client Datanodes will continue to work.

Changes in this patch:

• Added peerApparentVersion field to ReplicateContainerCommandProto.
• SCM (ReplicationManager): When sending a replicate command, SCM looks up the peer datanode's last known apparent version (from heartbeat reports) and sets it on the command. For push replication the peer is the target DN; for pull replication it is the minimum apparent version across all source DNs.
• Datanode (ReplicateContainerCommandHandler): Upon receiving the command, the datanode computes min(own apparent version, peer apparent version) and stores it as lowestCommonApparentVersion on the ReplicationTask.
• ReplicationTask: Exposes getLowestCommonApparentVersion() for replicators to use when gating version-dependent protocol features in the future.

Used Claude Opus 4.6.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15417

How was this patch tested?

Green CI on my fork: https://github.com/dombizita/ozone/actions/runs/27760621062

[errose28]

Thanks for adding this @dombizita. Left some comments, mostly around a few high level ideas:

• IMO we can assume that the pull replicator will be removed from master by the time this lands, so we should not reference it here.
• SCM should decide the version to use in the command. The command itself should be a vessel for whatever version is provided and datanodes should use the version they are instructed without needing to know why SCM has made that decision.
• We should use the typed HDDSVersion instead of a serialized int up until it needs to be converted to a proto.
• We should fail fast on the replication path instead of trying to absorb errors for cases like missing node information or unexpected versions which may lead to incorrect behavior.

[dombizita]

In the last commit I moved lowestApparentVersion() and lookupApparentVersion() from ReplicationManager to a common place, so it can be re-used in HDDS-15641.

[errose28]

Thanks for the updates. The PR description needs to be updated since the min computation is now on SCM.

[errose28]

Ideally nothing outside the version class would be comparing serialized values. Maybe we add a static ComponentVersion.min(v1, v2) method to handle this.

[dombizita]

Added ComponentVersion.min(ComponentVersion...) method.

[errose28]

Suggested change

	* Returns the lowest apparent (finalized) version among the given datanodes,
	* Returns the lowest apparent version among the given datanodes,

[errose28]

The version returned by the datanodes may be an HDDSVersion or an HDDSLayoutFeature. This in-place serde is just erasing the type. Really this whole method should be deleted. We should add ComponentVersion.min(ComponentVersion...) and pass it getLastKnownApparentVersion for all Datanodes we are considering. If you check the constructor of this class you'll see that we already assume the version info is always present, otherwise we'd get an NPE. That part could possibly be improved, even though DNs are expected to send version info on every heartbeat.