Skip to content

enhance: [2.5] Remove balance constraints between channel and segment tasks #42410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

weiliu1031
Copy link
Contributor

@weiliu1031 weiliu1031 commented May 30, 2025

issue: #42176
pr: #42177

Remove the mutual exclusion constraints between channel and segment balance tasks to allow them to run concurrently.

Changes include:

  • Remove permitBalanceChannel() and permitBalanceSegment() methods from RoundRobinBalancer
  • Update ChannelLevelScoreBalancer, MultiTargetBalancer, RowCountBasedBalancer, and ScoreBasedBalancer to remove constraint checks
  • Allow segment balance tasks to proceed even when channel balance tasks are running
  • Update test cases to reflect new behavior where balance tasks no longer block each other
  • Improve error handling in task executor by preferring serviceable shard leaders for segment release operations
  • Add fallback logic to find latest shard leader when serviceable leader is not available

This change improves the efficiency of load balancing by removing unnecessary coordination overhead between different types of balance operations.

Signed-off-by: Wei Liu [email protected]

Remove the mutual exclusion constraints between channel and segment
balance tasks to allow them to run concurrently.

Changes include:
- Remove permitBalanceChannel() and permitBalanceSegment() methods
  from RoundRobinBalancer
- Update ChannelLevelScoreBalancer, MultiTargetBalancer,
  RowCountBasedBalancer, and ScoreBasedBalancer to remove constraint
  checks
- Allow segment balance tasks to proceed even when channel balance
  tasks are running
- Update test cases to reflect new behavior where balance tasks no
  longer block each other
- Improve error handling in task executor by preferring serviceable
  shard leaders for segment release operations
- Add fallback logic to find latest shard leader when serviceable
  leader is not available

This change improves the efficiency of load balancing by removing
unnecessary coordination overhead between different types of balance
operations.

Signed-off-by: Wei Liu <[email protected]>
@sre-ci-robot sre-ci-robot added the size/M Denotes a PR that changes 30-99 lines. label May 30, 2025
@mergify mergify bot added the dco-passed DCO check passed. label May 30, 2025
Copy link
Contributor

mergify bot commented May 30, 2025

@weiliu1031

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly 

Please review and update your PR to comply with these guidelines.

Copy link

codecov bot commented May 30, 2025

Codecov Report

Attention: Patch coverage is 54.16667% with 11 lines in your changes missing coverage. Please review.

Project coverage is 80.64%. Comparing base (b7fc382) to head (14fa621).
Report is 7 commits behind head on 2.5.

Files with missing lines Patch % Lines
internal/querycoordv2/task/executor.go 12.50% 7 Missing ⚠️
...ernal/querycoordv2/balance/multi_target_balance.go 0.00% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              2.5   #42410      +/-   ##
==========================================
- Coverage   80.68%   80.64%   -0.04%     
==========================================
  Files        1428     1428              
  Lines      205836   205829       -7     
==========================================
- Hits       166069   165991      -78     
- Misses      33927    33999      +72     
+ Partials     5840     5839       -1     
Components Coverage Δ
Client 79.42% <ø> (ø)
Core 70.63% <ø> (ø)
Go 82.61% <54.16%> (-0.05%) ⬇️
Files with missing lines Coverage Δ
internal/querycoordv2/balance/balance.go 93.82% <ø> (-0.30%) ⬇️
...erycoordv2/balance/channel_level_score_balancer.go 89.77% <100.00%> (-4.02%) ⬇️
...al/querycoordv2/balance/rowcount_based_balancer.go 93.63% <100.00%> (-0.03%) ⬇️
...ernal/querycoordv2/balance/score_based_balancer.go 96.43% <100.00%> (-0.65%) ⬇️
...ernal/querycoordv2/balance/multi_target_balance.go 81.77% <0.00%> (+0.40%) ⬆️
internal/querycoordv2/task/executor.go 76.71% <12.50%> (-1.48%) ⬇️

... and 21 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify mergify bot added ci-passed and removed ci-passed labels May 30, 2025
@weiliu1031 weiliu1031 changed the title refactor: [2.5] Remove balance constraints between channel and segment tasks enhance: [2.5] Remove balance constraints between channel and segment tasks May 30, 2025
@mergify mergify bot added kind/enhancement Issues or changes related to enhancement ci-passed and removed do-not-merge/invalid-pr-format labels May 30, 2025
@xiaofan-luan
Copy link
Collaborator

/lgtm
/approve

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: weiliu1031, xiaofan-luan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit b298218 into milvus-io:2.5 Jun 3, 2025
18 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/M Denotes a PR that changes 30-99 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants