HDDS-15084. Core Runtime: SCM + OM

[henrybear327]

What changes were proposed in this pull request?

This PR adds the core SCM and OM runtime support for ozone local.

ozone local currently has the local filesystem and configuration lifecycle from HDDS-15083. This change builds on that base and updates LocalOzoneCluster so it can initialize SCM/OM storage, start StorageContainerManager and OzoneManager in process, wait for scoped runtime readiness, and shut the services down cleanly.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15084

How was this patch tested?

https://github.com/apache/ozone/actions/runs/27076487958

mvn test -pl hadoop-ozone/tools -Dtest=TestLocalOzoneCluster
mvn test -pl hadoop-ozone/integration-test -Dtest=TestLocalOzoneClusterRuntime

[henrybear327]

cc: @peterxcli

[rich7420]

@henrybear327 thanks for the patch, this reads really cleanly!

but I got one thought: a lot of the bring-up here (storage init, port allocation, readiness, shutdown) overlaps with what MiniOzoneClusterImpl already does. Did you consider building on it or extracting a shared piece, so the two don't drift apart as SCM/OM init changes? Not a blocker — just curious if it was ruled out for a reason. Thanks!

[henrybear327]

@henrybear327 thanks for the patch, this reads really cleanly!

but I got one thought: a lot of the bring-up here (storage init, port allocation, readiness, shutdown) overlaps with what MiniOzoneClusterImpl already does. Did you consider building on it or extracting a shared piece, so the two don't drift apart as SCM/OM init changes? Not a blocker — just curious if it was ruled out for a reason. Thanks!

Hi @rich7420 , thanks for the question.

I think this stream of work is quite different from the MiniOzoneClusterImpl implementation.

MiniOzoneClusterImpl lives in ozone-mini-cluster, a test-only module (it depends on hdds-test-utils) that isn't packaged into the distribution.

ozone local ships from ozone-tools and is meant to be run directly via the CLI, so the use case is quite different and the implementations will diverge in the long run — configuration, node setup, persistent ports (e.g. ports.properties for devcontainer/compose reuse vs. MiniOzoneCluster's random ephemeral ports), etc.

Not sure if this answers your question!

[henrybear327]

cc: @chungen0126 @peterxcli for a review! Thank you!

[henrybear327]

Rebased. PTAL @peterxcli @chungen0126 :) Thanks

This should unblock #10159 so we can move forward.

[chungen0126]

Thanks @henrybear327 for the patch. Left some comments.

[chungen0126]

+1 LGTM

[peterxcli]

@henrybear327 thanks for the further update! LGTM +1!
pending ci

[peterxcli]

Thanks @henrybear327 for the patch, @henrybear327 for the review!

[henrybear327]

Thanks @henrybear327 for the patch, @henrybear327 for the review!

I guess it's thanks @peterxcli and @chungen0126 for the review!

Get some rest :D