Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE#12994]Optimize config operation and capacity management. #13002

Merged
merged 19 commits into from
Mar 5, 2025

Conversation

KiteSoar
Copy link
Contributor

What is the purpose of the change

its close #12994

Brief changelog

XX

Verifying this change

XXXX

Follow this checklist to help us incorporate your contribution quickly and easily:

  • Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
  • Run mvn -B clean package apache-rat:check findbugs:findbugs -Dmaven.test.skip=true to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.

Copy link

Thanks for your this PR. 🙏
Please check again for your PR changes whether contains any usage/api/configuration change such as Add new API , Add new configuration, Change default value of configuration.
If so, please add or update documents(markdown type) in docs/next/ for repository nacos-group/nacos-group.github.io


感谢您提交的PR。 🙏
请再次查看您的PR内容,确认是否包含任何使用方式/API/配置参数的变更,如:新增API新增配置参数修改默认配置等操作。
如果是,请确保在提交之前,在仓库nacos-group/nacos-group.github.io中的docs/next/目录下添加或更新文档(markdown格式)。

@wuyfee
Copy link

wuyfee commented Dec 30, 2024

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar marked this pull request as draft December 31, 2024 01:21
@codecov-commenter
Copy link

codecov-commenter commented Jan 1, 2025

Codecov Report

Attention: Patch coverage is 79.16667% with 95 lines in your changes missing coverage. Please review.

Project coverage is 72.30%. Comparing base (1e889ea) to head (d2afe08).
Report is 6 commits behind head on develop.

Files with missing lines Patch % Lines
...cos/config/server/controller/ConfigController.java 60.75% 27 Missing and 4 partials ⚠️
...config/server/aspect/CapacityManagementAspect.java 77.14% 11 Missing and 5 partials ⚠️
...a/nacos/config/server/aspect/RequestLogAspect.java 77.08% 7 Missing and 4 partials ⚠️
...nacos/config/server/aspect/ConfigChangeAspect.java 80.39% 6 Missing and 4 partials ⚠️
...server/exception/ConfigAlreadyExistsException.java 20.00% 8 Missing ⚠️
...java/com/alibaba/nacos/api/model/v2/ErrorCode.java 50.00% 4 Missing ⚠️
...a/nacos/config/server/model/ConfigRequestInfo.java 60.00% 4 Missing ⚠️
...fig/server/remote/ConfigPublishRequestHandler.java 33.33% 2 Missing ⚠️
...asource/impl/derby/GroupCapacityMapperByDerby.java 95.55% 2 Missing ⚠️
...source/impl/derby/TenantCapacityMapperByDerby.java 93.10% 2 Missing ⚠️
... and 3 more
Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##             develop   #13002      +/-   ##
=============================================
+ Coverage      72.27%   72.30%   +0.03%     
- Complexity     10032    10045      +13     
=============================================
  Files           1310     1312       +2     
  Lines          42016    42171     +155     
  Branches        4416     4425       +9     
=============================================
+ Hits           30367    30493     +126     
- Misses          9555     9577      +22     
- Partials        2094     2101       +7     
Files with missing lines Coverage Δ
...om/alibaba/nacos/api/exception/NacosException.java 100.00% <ø> (ø)
...libaba/nacos/config/server/constant/Constants.java 76.92% <ø> (ø)
...onfig/server/controller/v2/ConfigControllerV2.java 79.62% <100.00%> (+0.38%) ⬆️
...ba/nacos/config/server/monitor/MetricsMonitor.java 87.17% <ø> (+11.56%) ⬆️
...nfig/server/remote/ConfigRemoveRequestHandler.java 100.00% <100.00%> (+24.13%) ⬆️
.../config/server/service/ConfigOperationService.java 80.80% <100.00%> (+1.66%) ⬆️
.../service/capacity/GroupCapacityPersistService.java 81.93% <100.00%> (+0.11%) ⬆️
.../plugin/datasource/mapper/GroupCapacityMapper.java 100.00% <ø> (ø)
...plugin/datasource/mapper/TenantCapacityMapper.java 100.00% <ø> (ø)
...service/capacity/TenantCapacityPersistService.java 82.01% <85.71%> (+0.13%) ⬆️
... and 12 more

... and 48 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e889ea...d2afe08. Read the comment docs.

@wuyfee
Copy link

wuyfee commented Jan 1, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar marked this pull request as ready for review January 1, 2025 12:54
@wuyfee
Copy link

wuyfee commented Jan 1, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar marked this pull request as draft January 2, 2025 23:49
@Dario-s-j
Copy link

Dario-s-j commented Jan 3, 2025 via email

@shiyiyue1102
Copy link
Collaborator

shiyiyue1102 commented Jan 6, 2025

change capacity aspect cutpoint to ConfigOperationService

@Dario-s-j
Copy link

Dario-s-j commented Jan 6, 2025 via email

@KiteSoar KiteSoar marked this pull request as ready for review January 10, 2025 02:34
@wuyfee
Copy link

wuyfee commented Jan 10, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
❌ - e2e-java-test (standalone & cluster & standalone_auth): failure
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
❌ - e2e-csharp-test (standalone & cluster): failure
❌ - e2e-nodejs-test (standalone & cluster): failure
❌ - e2e-python-test (standalone & cluster): failure
✅ - clean (standalone & cluster & standalone_auth): success

@wuyfee
Copy link

wuyfee commented Jan 10, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
❌ - e2e-java-test (standalone & cluster & standalone_auth): failure
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
❌ - e2e-csharp-test (standalone & cluster): failure
❌ - e2e-nodejs-test (standalone & cluster): failure
❌ - e2e-python-test (standalone & cluster): failure
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar marked this pull request as draft January 10, 2025 05:48
@KiteSoar KiteSoar marked this pull request as ready for review January 10, 2025 06:22
@wuyfee
Copy link

wuyfee commented Jan 10, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
❌ - e2e-java-test (standalone & cluster & standalone_auth): failure
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
❌ - e2e-csharp-test (standalone & cluster): failure
❌ - e2e-nodejs-test (standalone & cluster): failure
❌ - e2e-python-test (standalone & cluster): failure
✅ - clean (standalone & cluster & standalone_auth): success

/**
* Deletes configuration information based on the IDs list.
*/
public Boolean deleteConfigs(List<Long> ids, String srcIp, String srcUser) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleteConfigs should check capacity , or can we split deleteConfigs operation to serval single deleteConfig operation in ConfigController?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have already split batch operations on controllers, method deteConfigs should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

/**
* Batch insert or update configuration information.
*/
public Map<String, Object> batchInsertOrUpdate(List<ConfigAllInfo> configInfoList, String srcUser, String srcIp,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all batch operation invoked ConfigOperationService single operation interface, all aspects works at single config publish & delete interface of ConfigOperationService

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have already split batch operations on controllers, method batchInsertOrUpdate should be removed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@KiteSoar KiteSoar force-pushed the optimize-config-operation branch from 8a6e38c to 2fab31d Compare February 15, 2025 09:55
@wuyfee
Copy link

wuyfee commented Feb 15, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@@ -70,6 +70,8 @@ public class ConfigForm implements Serializable {

private String schema;

private Boolean updateForExist = Boolean.TRUE;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems that moving this param to ConfigRequestInfo may be more proper

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is updateForExist means whether to update when the configuration already exists. It should be part of business logic. The ConfigRequestInfo class belong to request metadata. Irrelevant to business operations,so it seems appropriate to put it in Conform?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you said, "updateForExist means whether to update when the configuration already exists", it is not a part of config itself. Conform contains things about config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, it has been moved to the ConfigRequestInfo class.

@wuyfee
Copy link

wuyfee commented Feb 20, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar force-pushed the optimize-config-operation branch from 3c22682 to 1abdaa8 Compare February 24, 2025 02:10
@wuyfee
Copy link

wuyfee commented Feb 24, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

if (limitType != null) {
return response4Limit(request, response, limitType);
throw new NacosException(ErrorCode.SERVER_ERROR.getCode(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should return different error code for different limit type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@wuyfee
Copy link

wuyfee commented Feb 24, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar force-pushed the optimize-config-operation branch from 572a3eb to 1b9d0be Compare February 24, 2025 03:49
@wuyfee
Copy link

wuyfee commented Feb 24, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar force-pushed the optimize-config-operation branch from 1b9d0be to eb7cfcc Compare February 24, 2025 05:46
@wuyfee
Copy link

wuyfee commented Feb 24, 2025

$\color{green}{SUCCESS}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
✅ - e2e-go-test (standalone & cluster): success
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@wuyfee
Copy link

wuyfee commented Mar 3, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@wuyfee
Copy link

wuyfee commented Mar 4, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@KiteSoar KiteSoar changed the title [ISSUE#12994]Optimize config operation. [ISSUE#12994]Optimize config operation and capacity management. Mar 5, 2025
@wuyfee
Copy link

wuyfee commented Mar 5, 2025

$\color{red}{FAILURE}$
DETAILS
✅ - docker: success
✅ - deploy (standalone & cluster & standalone_auth): success
✅ - e2e-java-test (standalone & cluster & standalone_auth): success
❌ - e2e-go-test (standalone & cluster): failure
✅ - e2e-cpp-test (standalone & cluster): success
✅ - e2e-csharp-test (standalone & cluster): success
✅ - e2e-nodejs-test (standalone & cluster): success
✅ - e2e-python-test (standalone & cluster): success
✅ - clean (standalone & cluster & standalone_auth): success

@shiyiyue1102 shiyiyue1102 merged commit c2f1579 into alibaba:develop Mar 5, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Server Config Operation Interface Refactor(Optimize)
6 participants