Skip to content

HDDS-10237. Dynamic reconfiguration of replication supervisor thread pool - #10403

Merged
adoroszlai merged 2 commits into
apache:masterfrom
Russole:HDDS-10237
Jun 11, 2026
Merged

HDDS-10237. Dynamic reconfiguration of replication supervisor thread pool#10403
adoroszlai merged 2 commits into
apache:masterfrom
Russole:HDDS-10237

Conversation

@Russole

@Russole Russole commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  • Update ReplicationSupervisor when hdds.datanode.replication.streams.limit is reconfigured dynamically.
  • Extract the supervisor resize logic so both node state changes and stream limit reconfiguration use the same scaling behavior.
  • Preserve out-of-service scaling for decommissioning and maintenance datanodes when the replication stream limit changes.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10237

How was this patch tested?

  • Added unit test coverage for ReplicationSupervisor resize on stream limit reconfiguration.
  • mvn -pl hadoop-hdds/container-service -Dtest=TestReplicationSupervisor test
  • All CI Test passed

@Russole
Russole requested a review from adoroszlai June 1, 2026 15:24

@adoroszlai adoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Russole for the patch, LGTM.

rs.setReplicationMaxStreams(7);
assertEquals(7, threadPoolSize.get());

rs.nodeStateUpdated(DECOMMISSIONING);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test assumes OUTOFSERVICE_FACTOR_DEFAULT_VALUE=2, but since its configurable its better use config value in calculating expected value in assert.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, updated to use the configured factor.

@Russole
Russole requested a review from yandrey321 June 2, 2026 13:55
@Russole

Russole commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Hi @adoroszlai, I made a small update to address the latest comment. Would you like to take another look?

@Russole
Russole requested a review from adoroszlai June 6, 2026 12:42

@yandrey321 yandrey321 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@adoroszlai
adoroszlai merged commit 74f54d8 into apache:master Jun 11, 2026
47 checks passed
@adoroszlai

Copy link
Copy Markdown
Contributor

Thanks @Russole for the patch, @yandrey321 for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants