Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update In-progress CS Stats + Y_ASSERTs #16050

Merged
merged 2 commits into from
Mar 24, 2025
Merged

Update In-progress CS Stats + Y_ASSERTs #16050

merged 2 commits into from
Mar 24, 2025

Conversation

Hor911
Copy link
Collaborator

@Hor911 Hor911 commented Mar 21, 2025 •
edited
Loading

Update in-progress ColumnShard statistics (bytes + rows per shard) during query execution

There were 2 attempts in the past:

#15320
#15423

reverted due to cluster destabilization (likely to be invalid mem address access).

After this there were a bunch of fixes and improvements:

#15722
#15742
#15768

This PR seems to be pretty stable and was checked at perf cluster with TPC-H scale 10 000 load

@Hor911 Hor911 requested a review from a team as a code owner March 21, 2025 10:37
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 21, 2025 •
edited
Loading

🟢 2025-03-24 11:14:58 UTC The validation of the Pull Request description is successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 21, 2025 •
edited
Loading

⚪ 2025-03-21 10:43:31 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0649b3e has started.
⚪ 2025-03-21 10:43:47 UTC Artifacts will be uploaded here
⚪ 2025-03-21 10:46:41 UTC ya make is running...
🟡 2025-03-21 12:08:03 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19673 18316 0 13 1228 116

⚪ 2025-03-21 12:09:54 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 12:26:43 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
184 (only retried tests) 62 0 13 0 109

⚪ 2025-03-21 12:26:57 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-03-21 12:39:34 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
175 (only retried tests) 57 0 12 0 106

🟢 2025-03-21 12:39:46 UTC Build successful.
🟢 2025-03-21 12:40:07 UTC ydbd size 2.2 GiB changed* by +55.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 6dc2e21 merge: 0649b3e diff diff %
ydbd size 2 314 348 712 Bytes 2 314 405 792 Bytes +55.7 KiB +0.002%
ydbd stripped size 484 361 728 Bytes 484 372 352 Bytes +10.4 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 21, 2025 •
edited
Loading

⚪ 2025-03-21 10:43:36 UTC Pre-commit check linux-x86_64-release-asan for 0649b3e has started.
⚪ 2025-03-21 10:43:52 UTC Artifacts will be uploaded here
⚪ 2025-03-21 10:46:43 UTC ya make is running...
🟡 2025-03-21 12:17:54 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12165 12062 0 49 17 37

⚪ 2025-03-21 12:18:57 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 12:36:40 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
160 (only retried tests) 107 0 15 4 34

⚪ 2025-03-21 12:36:52 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-21 12:49:01 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
91 (only retried tests) 42 0 14 3 32

🟢 2025-03-21 12:49:12 UTC Build successful.
🟡 2025-03-21 12:49:44 UTC ydbd size 3.8 GiB changed* by +106.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6dc2e21 merge: 0649b3e diff diff %
ydbd size 4 073 892 992 Bytes 4 074 001 552 Bytes +106.0 KiB +0.003%
ydbd stripped size 1 409 061 352 Bytes 1 409 087 656 Bytes +25.7 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Hor911 Hor911 force-pushed the update_in_progress_cs_stat branch from 789e5fa to 17484d9 Compare March 21, 2025 16:09
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 21, 2025 •
edited
Loading

⚪ 2025-03-21 16:12:32 UTC Pre-commit check linux-x86_64-release-asan for dffd9cc has started.
⚪ 2025-03-21 16:12:53 UTC Artifacts will be uploaded here
⚪ 2025-03-21 16:15:34 UTC ya make is running...
🟡 2025-03-21 17:28:08 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12179 12097 0 39 8 35

⚪ 2025-03-21 17:29:42 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-21 17:46:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
128 (only retried tests) 86 0 4 4 34

⚪ 2025-03-21 17:47:08 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-03-21 18:00:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
71 (only retried tests) 33 0 3 1 34

🟢 2025-03-21 18:00:17 UTC Build successful.
🟡 2025-03-21 18:00:46 UTC ydbd size 3.8 GiB changed* by +110.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a95ce0d merge: dffd9cc diff diff %
ydbd size 4 075 721 248 Bytes 4 075 833 904 Bytes +110.0 KiB +0.003%
ydbd stripped size 1 409 668 456 Bytes 1 409 698 792 Bytes +29.6 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 21, 2025 •
edited
Loading

⚪ 2025-03-21 16:12:36 UTC Pre-commit check linux-x86_64-relwithdebinfo for dffd9cc has started.
⚪ 2025-03-21 16:12:50 UTC Artifacts will be uploaded here
⚪ 2025-03-21 16:15:38 UTC ya make is running...
🟢 2025-03-21 17:15:17 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19687 18341 0 0 1229 117

🟢 2025-03-21 17:16:57 UTC Build successful.
🟢 2025-03-21 17:17:14 UTC ydbd size 2.2 GiB changed* by +55.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: a95ce0d merge: dffd9cc diff diff %
ydbd size 2 315 431 248 Bytes 2 315 488 256 Bytes +55.7 KiB +0.002%
ydbd stripped size 484 584 992 Bytes 484 595 552 Bytes +10.3 KiB +0.002%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Hor911 Hor911 merged commit ce1388f into ydb-platform:main Mar 24, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lll-phill-lll lll-phill-lll lll-phill-lll approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Hor911 @lll-phill-lll