Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [improve](cloud-mow) Make mow stream load retry times more reasonable #47666 #48194

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #47666

…#47666)

pr #46365 adjust timeout about mow load, thers is no need so much retry
times, can reduce to 5.
@Thearas
Copy link
Contributor

Thearas commented Feb 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 21, 2025
@Thearas
Copy link
Contributor

Thearas commented Feb 21, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40724 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb6b7e4024f00f1f5bb34e587ee551999118a387, data reload: false

------ Round 1 ----------------------------------
q1	17622	7367	7211	7211
q2	2059	178	168	168
q3	10643	1087	1153	1087
q4	10561	739	698	698
q5	7752	2892	2856	2856
q6	239	146	150	146
q7	979	611	593	593
q8	9579	1949	2015	1949
q9	6617	6399	6379	6379
q10	6940	2287	2336	2287
q11	456	264	262	262
q12	401	214	205	205
q13	17779	2987	2992	2987
q14	232	209	212	209
q15	567	512	542	512
q16	646	605	582	582
q17	972	576	545	545
q18	7164	6740	6828	6740
q19	1404	1134	994	994
q20	462	196	199	196
q21	4192	3297	3140	3140
q22	1094	979	978	978
Total cold run time: 108360 ms
Total hot run time: 40724 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7218	7183	7166	7166
q2	331	235	236	235
q3	2896	2912	2912	2912
q4	2055	1849	1856	1849
q5	5693	5733	5716	5716
q6	220	142	142	142
q7	2260	1803	1827	1803
q8	3370	3530	3493	3493
q9	8852	8877	8849	8849
q10	3592	3546	3598	3546
q11	597	526	505	505
q12	803	615	586	586
q13	6905	3216	3125	3125
q14	299	269	271	269
q15	585	519	518	518
q16	714	665	660	660
q17	1830	1623	1702	1623
q18	8383	7726	7559	7559
q19	1685	1531	1456	1456
q20	2100	1838	1859	1838
q21	5588	5542	5348	5348
q22	1136	1064	1036	1036
Total cold run time: 67112 ms
Total hot run time: 60234 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198419 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb6b7e4024f00f1f5bb34e587ee551999118a387, data reload: false

query1	1338	939	923	923
query2	6237	1960	2023	1960
query3	10850	4370	4397	4370
query4	66063	29119	23884	23884
query5	5030	456	445	445
query6	416	168	178	168
query7	5639	318	328	318
query8	313	212	223	212
query9	9161	2722	2712	2712
query10	470	270	281	270
query11	17394	15203	15729	15203
query12	160	107	102	102
query13	1549	438	437	437
query14	9738	7677	7129	7129
query15	207	191	180	180
query16	7154	461	490	461
query17	1097	587	592	587
query18	1830	330	308	308
query19	231	162	160	160
query20	113	104	107	104
query21	216	104	102	102
query22	4987	4598	4369	4369
query23	34618	34175	34416	34175
query24	6191	2987	2909	2909
query25	540	442	440	440
query26	657	176	176	176
query27	1928	357	371	357
query28	4048	2451	2477	2451
query29	700	465	468	465
query30	268	167	170	167
query31	1009	839	805	805
query32	66	60	57	57
query33	400	295	285	285
query34	898	526	526	526
query35	836	738	720	720
query36	1111	968	972	968
query37	122	80	74	74
query38	4149	4057	4018	4018
query39	1507	1472	1508	1472
query40	200	102	110	102
query41	55	48	47	47
query42	115	103	109	103
query43	533	502	490	490
query44	1151	842	852	842
query45	181	170	171	170
query46	1146	730	721	721
query47	2059	1889	1916	1889
query48	493	379	386	379
query49	768	405	395	395
query50	854	422	433	422
query51	7418	7153	7376	7153
query52	103	92	88	88
query53	255	181	182	181
query54	565	450	457	450
query55	80	77	79	77
query56	275	240	251	240
query57	1244	1158	1118	1118
query58	221	212	211	211
query59	3305	2947	2969	2947
query60	280	264	256	256
query61	105	105	105	105
query62	818	735	716	716
query63	222	186	189	186
query64	1341	652	633	633
query65	3286	3214	3172	3172
query66	697	302	315	302
query67	16002	15730	15676	15676
query68	4017	603	590	590
query69	438	265	280	265
query70	1157	1143	1123	1123
query71	349	258	258	258
query72	6371	4143	3987	3987
query73	741	352	349	349
query74	10184	9039	9141	9039
query75	3312	2645	2647	2645
query76	1810	1073	1024	1024
query77	490	277	263	263
query78	10537	9693	9651	9651
query79	1252	609	613	609
query80	848	428	429	428
query81	495	248	270	248
query82	1296	126	119	119
query83	246	144	141	141
query84	281	80	83	80
query85	875	298	286	286
query86	337	305	297	297
query87	4392	4290	4253	4253
query88	3713	2397	2370	2370
query89	420	291	292	291
query90	1989	186	189	186
query91	184	153	154	153
query92	65	49	52	49
query93	1648	555	565	555
query94	804	298	264	264
query95	360	256	250	250
query96	606	279	288	279
query97	3300	3204	3216	3204
query98	217	203	199	199
query99	1647	1424	1470	1424
Total cold run time: 317394 ms
Total hot run time: 198419 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb6b7e4024f00f1f5bb34e587ee551999118a387, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.52	0.49	0.51
query6	1.13	0.74	0.73
query7	0.02	0.01	0.03
query8	0.03	0.03	0.03
query9	0.56	0.50	0.49
query10	0.54	0.54	0.56
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.59	0.60
query14	2.73	2.72	2.72
query15	0.89	0.82	0.83
query16	0.38	0.37	0.39
query17	1.03	0.97	1.07
query18	0.24	0.21	0.21
query19	1.97	1.81	1.97
query20	0.01	0.00	0.01
query21	15.36	0.57	0.56
query22	2.76	2.44	1.95
query23	16.94	1.19	0.71
query24	2.95	1.03	1.48
query25	0.17	0.23	0.16
query26	0.44	0.13	0.13
query27	0.04	0.05	0.04
query28	10.35	1.10	1.07
query29	12.59	3.27	3.23
query30	0.25	0.06	0.05
query31	2.85	0.38	0.38
query32	3.25	0.45	0.45
query33	2.98	3.00	3.04
query34	17.22	4.44	4.54
query35	4.50	4.51	4.50
query36	0.69	0.49	0.51
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.72 s
Total hot run time: 32.69 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants