enhance: add cache shard disk usage metric

[sunby]

Summary

• Thread the insert-channel shard through field, index, JSON stats, text index, storage v1, and storage v2 load metadata into cachinglayer translator metadata.
• Add cgo shard setters so Go-side load requests can pass shard into C++ load info.
• Keep the actual cachinglayer metric definition and CacheSlot accounting in companion milvus-common PR: enhance: add cache shard disk usage metric zilliztech/milvus-common#98

issue: #50941

[mergify]

@sunby Please associate the related issue to the body of your Pull Request. (eg. "issue: #")

[mergify]

@sunby Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[mergify]

@sunby Please associate the related issue to the body of your Pull Request. (eg. "issue: #")

[sre-ci-robot]

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

• /ci-rerun-code-check // for ci-v2/code-check
• /ci-rerun-code-check-macos // for Code Checker MacOS (GitHub Actions)
• /ci-rerun-build // for ci-v2/build
• /ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
• /ci-rerun-buildenv // for ci-v2/build-env (build milvus-env builder images; update .env after the new tag is ready)
• /ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
• /ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
• /ci-rerun-ut-cpp // for ci-v2/ut-cpp
• /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
• /ci-rerun-e2e-default // for ci-v2/e2e-default
• /ci-rerun-e2e-amd // for ci-v2/e2e-amd (e2e pool dispatcher)
• /ci-rerun-build-ut-cov // for ci-v2/build-ut-cov (build + unit tests in one pipeline)
• /ci-rerun-gosdk // for ci-v2/go-sdk (Go SDK E2E tests, ARM)

If you have any questions or requests, please contact @zhikunyao.

[sre-ci-robot]

❌ CI Loop Results 62decaf

Stage	Result	Duration	Tests
❌ Build	FAILURE	11.2min	-
❌ Code-Check	UNKNOWN	-	-
❌ UT-GO	UNKNOWN	-	-
❌ UT-Integration	UNKNOWN	-	-
❌ UT-CPP-Cov	UNKNOWN	-	-

Total: 15min | Pipeline | Artifacts

Failed Test Logs:

• Build: view log

[sre-ci-robot]

✅ CI Loop Results 9d1c90a

Stage	Result	Duration	Tests
✅ Build	SUCCESS	12.4min	-
✅ Code-Check	SUCCESS	9.8min	-
✅ UT-GO	SUCCESS	22.5min	1071 total, 1071 passed, 0 failed
✅ UT-Integration	SUCCESS	24.5min	46 total, 46 passed, 0 failed
✅ UT-CPP-Cov	SUCCESS	62.5min	8019 total, 8019 passed, 0 failed

Total: 84min | Pipeline | Artifacts

Overall Coverage: 72.1%
Diff Coverage: CPP 60.6% (40 hit, 26 miss, 66 measurable lines, 108 unmeasured) | Go 86.7% (13 hit, 2 miss, 15 measurable lines, 4 unmeasured)
Diff Coverage HTML: view changed lines
Total Patch Coverage: 65.4% (53/81 measurable lines, 112 unmeasured)

[codecov]

Codecov Report

❌ Patch coverage is 63.23529% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.31%. Comparing base (4018b29) to head (51e5a82).
⚠️ Report is 22 commits behind head on master.

Files with missing lines	Patch %	Lines
...re/storagev1translator/V1SealedIndexTranslator.cpp	0.00%	6 Missing ⚠️
...rnal/core/src/segcore/ChunkedSegmentSealedImpl.cpp	54.54%	5 Missing ⚠️
internal/core/src/segcore/load_field_data_c.cpp	0.00%	4 Missing ⚠️
internal/core/src/segcore/load_index_c.cpp	0.00%	4 Missing ⚠️
internal/querynodev2/segments/load_index_info.go	57.14%	2 Missing and 1 partial ⚠️
internal/core/src/segcore/CacheMetricAttribution.h	66.66%	1 Missing ⚠️
...core/storagev1translator/SealedIndexTranslator.cpp	50.00%	1 Missing ⚠️
...e/storagev1translator/TextMatchIndexTranslator.cpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #50733      +/-   ##
==========================================
- Coverage   79.37%   79.31%   -0.07%     
==========================================
  Files        2279     2280       +1     
  Lines      414553   414603      +50     
==========================================
- Hits       329051   328838     -213     
- Misses      75416    75630     +214     
- Partials    10086    10135      +49     

Components	Coverage Δ
Client	80.65% <ø> (-0.03%)	⬇️
Core	86.43% <58.49%> (-0.01%)	⬇️
Go	76.97% <86.36%> (-0.09%)	⬇️

Files with missing lines	Coverage Δ
internal/core/src/common/LoadInfo.h	100.00% <ø> (ø)
...nternal/core/src/index/json_stats/JsonKeyStats.cpp	90.53% <100.00%> (+0.04%)	⬆️
internal/core/src/index/json_stats/JsonKeyStats.h	69.23% <ø> (ø)
internal/core/src/mmap/Types.h	100.00% <100.00%> (ø)
internal/core/src/monitor/monitor_c.cpp	100.00% <100.00%> (ø)
internal/core/src/segcore/SegmentGrowingImpl.cpp	63.51% <100.00%> (ø)
internal/core/src/segcore/SegmentLoadInfo.cpp	88.96% <100.00%> (+0.02%)	⬆️
internal/core/src/segcore/Types.h	100.00% <100.00%> (ø)
...toragev1translator/BsonInvertedIndexTranslator.cpp	94.73% <100.00%> (ø)
.../storagev1translator/BsonInvertedIndexTranslator.h	100.00% <ø> (ø)
... and 18 more

... and 46 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sre-ci-robot]

✅ CI Loop Results 51e5a82

Stage	Result	Duration	Tests
✅ Build	SUCCESS	12.7min	-
✅ Code-Check	SUCCESS	7.7min	-
✅ UT-GO	SUCCESS	22.2min	1071 total, 1071 passed, 0 failed
✅ UT-Integration	SUCCESS	24.9min	46 total, 46 passed, 0 failed
✅ UT-CPP-Cov	SUCCESS	57.2min	8028 total, 8028 passed, 0 failed

Total: 77min | Pipeline | Artifacts

Overall Coverage: 72.1%
Diff Coverage: CPP 60.6% (40 hit, 26 miss, 66 measurable lines, 108 unmeasured) | Go 86.7% (13 hit, 2 miss, 15 measurable lines, 4 unmeasured)
Diff Coverage HTML: view changed lines
Total Patch Coverage: 65.4% (53/81 measurable lines, 112 unmeasured)

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sunby, zhengbuqian

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [zhengbuqian]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[aoiasd]

/lgtm