Skip to content

Conversation

@BohuTANG
Copy link
Member

@BohuTANG BohuTANG commented Nov 15, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • add dedicated profile counters for remote/local/memory scan bytes so EXPLAIN ANALYZE can report realistic IO splits
  • surface percentages for each scan counter in the formatted plan output, e.g.
bytes scanned from remote: 1.23 MiB (72.4%)
bytes scanned from local cache: 363.00 KiB (21.4%)
bytes scanned from memory cache: 105.00 KiB (6.2%)
  • hook disk/memory caches into the new counters so cache hits update the right stats

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why: profiling/metrics only

Type of change

  • Performance Improvement
  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Other (please describe):

This change is Reviewable

@BohuTANG BohuTANG changed the title Improve scan IO metrics and add replace-into auto increment helper Improve scan IO profile metrics Nov 15, 2025
@BohuTANG BohuTANG changed the title Improve scan IO profile metrics feat: improve scan IO profile metrics Nov 15, 2025
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Nov 15, 2025
@BohuTANG BohuTANG added the ci-cloud Build docker image for cloud test label Nov 15, 2025
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-18975-31804d8-1763177210

note: this image tag is only available for internal use.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 15, 2025

🤖 CI Job Analysis (Retry 1)

Workflow: 19391529825

📊 Summary

  • Total Jobs: 83
  • Failed Jobs: 7
  • Retryable: 0
  • Code Issues: 7

NO RETRY NEEDED

All failures appear to be code/test issues requiring manual fixes.

🔍 Job Details

  • linux / test_stateful_standalone: Not retryable (Code/Test)
  • linux / sqllogic / standalone_udf_server: Not retryable (Code/Test)
  • linux / sqllogic / standalone_cloud: Not retryable (Code/Test)
  • linux / sqllogic / standalone (standalone, 2c8g, http): Not retryable (Code/Test)
  • linux / sqllogic / standalone (standalone, 2c8g, hybrid): Not retryable (Code/Test)
  • linux / sqllogic / cluster (cluster, 2c8g, hybrid): Not retryable (Code/Test)
  • linux / sqllogic / cluster (cluster, 2c8g, http): Not retryable (Code/Test)

🤖 About

Automated analysis using job annotations to distinguish infrastructure issues (auto-retried) from code/test issues (manual fixes needed).

Add slt:ignore markers for the newly introduced scan IO metrics:
- bytes scanned from remote
- bytes scanned from local cache
- bytes scanned from memory cache

These metrics were added in the scan IO metrics feature but the test
baselines were not fully updated to ignore all three new metrics.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant