Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [Bug](function) add index check on function like #47991 #48018

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from
Open

branch-3.0: [Bug](function) add index check on function like #47991 #48018

wants to merge 1 commit into from
+1 −1

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #47991

@BiteTheDDDDt
[Bug](function) add index check on function like (#47991)
1aad5c4 
### What problem does this PR solve?
Sometimes we encounter offset[last]<chars.size(), since we can't
completely avoid this situation at the moment, we will add protective
code here first.

related check&fix : #47964

```cpp
false, DefaultMemoryAllocator>, pad_right_ = 16, pad_left_ = 15]: 假设 ‘(n >= (static_cast<ssize_t>(pad_left_) ? -1 : 0)) && (n <= static_cast<ssize_t>(this->size()))’ 失败。
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/common/signal_handler.h:421
 1# 0x00007F756FE8BB50 in /lib64/libc.so.6
 2# gsignal in /lib64/libc.so.6
 3# __GI_abort in /lib64/libc.so.6
 4# _nl_load_domain.cold.0 in /lib64/libc.so.6
 5# 0x00007F756FE84426 in /lib64/libc.so.6
 6# doris::vectorized::PODArray<unsigned int, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>::operator[](long) const at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/vec/common/pod_array.h:365
 7# doris::vectorized::FunctionLikeBase::execute_substring(doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul> const&, doris::vectorized::PODArray<unsigned int, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul> const&, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 15ul>&, doris::vectorized::LikeSearchState*) const at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/vec/functions/like.cpp:580
 8# doris::vectorized::FunctionLikeBase::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/vec/functions/like.cpp:532
 9# doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long) const at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/vec/functions/function.h:461
10# doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, unsigned long, bool) const at /mnt/disk2/liyuanyuan/doris_test/3.0/doris/be/src/vec/functions/function.cpp:120
```

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [x] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@Thearas
Copy link
Contributor

Thearas commented Feb 18, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Feb 18, 2025
@Thearas
Copy link
Contributor

Thearas commented Feb 18, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41792 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1aad5c4f41bc20aa2b2e2391fb9255c93d4ec476, data reload: false

------ Round 1 ----------------------------------
q1	17631	7524	7395	7395
q2	2040	166	173	166
q3	10558	1153	1157	1153
q4	10236	774	728	728
q5	7985	3007	2960	2960
q6	239	151	146	146
q7	995	618	644	618
q8	9906	2130	2059	2059
q9	6874	6559	6504	6504
q10	7579	2383	2373	2373
q11	481	279	287	279
q12	418	225	214	214
q13	17834	3076	3091	3076
q14	251	214	228	214
q15	591	539	541	539
q16	684	608	606	606
q17	1047	635	604	604
q18	7735	6753	6773	6753
q19	1401	1089	1142	1089
q20	495	208	207	207
q21	4334	3195	3150	3150
q22	1118	959	989	959
Total cold run time: 110432 ms
Total hot run time: 41792 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7380	7258	7395	7258
q2	363	248	243	243
q3	3141	3035	2920	2920
q4	2066	1796	1798	1796
q5	5649	5812	5743	5743
q6	230	134	133	133
q7	2192	1801	1837	1801
q8	3411	3552	3479	3479
q9	8990	8975	9013	8975
q10	3666	3597	3563	3563
q11	598	489	498	489
q12	809	573	603	573
q13	7449	3108	3142	3108
q14	313	271	274	271
q15	577	525	526	525
q16	664	652	643	643
q17	1852	1634	1616	1616
q18	8293	7899	7590	7590
q19	1793	1637	1622	1622
q20	2094	1789	1776	1776
q21	5346	5125	5172	5125
q22	1111	988	993	988
Total cold run time: 67987 ms
Total hot run time: 60237 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192383 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1aad5c4f41bc20aa2b2e2391fb9255c93d4ec476, data reload: false

query1	990	373	366	366
query2	6516	2128	2045	2045
query3	6707	212	221	212
query4	34009	23700	23775	23700
query5	4312	467	437	437
query6	261	176	177	176
query7	4628	304	319	304
query8	281	220	225	220
query9	9489	2681	2695	2681
query10	485	262	254	254
query11	18500	15545	15442	15442
query12	155	103	100	100
query13	1636	429	412	412
query14	9754	7097	7533	7097
query15	278	172	177	172
query16	8184	470	469	469
query17	1666	590	570	570
query18	2147	308	322	308
query19	387	161	159	159
query20	114	109	111	109
query21	211	107	103	103
query22	4482	4296	4088	4088
query23	34669	33788	33803	33788
query24	11592	2852	2896	2852
query25	728	413	409	409
query26	1766	173	170	170
query27	2787	356	354	354
query28	7799	2416	2393	2393
query29	1040	456	444	444
query30	333	163	169	163
query31	1050	786	819	786
query32	103	60	60	60
query33	797	314	304	304
query34	933	503	500	500
query35	877	745	702	702
query36	1113	950	948	948
query37	180	80	76	76
query38	4066	3812	3886	3812
query39	1499	1437	1426	1426
query40	294	103	103	103
query41	57	50	53	50
query42	113	101	103	101
query43	520	514	488	488
query44	1210	804	812	804
query45	186	173	174	173
query46	1143	723	719	719
query47	1948	1810	1839	1810
query48	479	391	397	391
query49	1297	419	404	404
query50	805	418	425	418
query51	7250	7219	6988	6988
query52	105	95	92	92
query53	259	185	188	185
query54	1306	486	476	476
query55	78	81	82	81
query56	293	249	265	249
query57	1226	1121	1099	1099
query58	244	213	218	213
query59	3303	3124	3018	3018
query60	281	251	254	251
query61	118	109	115	109
query62	915	755	735	735
query63	220	193	185	185
query64	5275	672	669	669
query65	3302	3230	3180	3180
query66	1435	317	308	308
query67	15989	15501	15506	15501
query68	5030	574	551	551
query69	442	259	304	259
query70	1189	1035	1118	1035
query71	367	253	257	253
query72	6364	4048	4059	4048
query73	756	352	352	352
query74	10550	9246	9205	9205
query75	3364	2676	2632	2632
query76	2968	1084	1074	1074
query77	427	281	272	272
query78	10511	9668	9620	9620
query79	1501	599	594	594
query80	1152	442	419	419
query81	552	242	238	238
query82	962	126	118	118
query83	224	152	153	152
query84	235	77	81	77
query85	1289	313	302	302
query86	365	300	305	300
query87	4369	4319	4275	4275
query88	3650	2458	2400	2400
query89	410	292	292	292
query90	1952	188	192	188
query91	188	150	167	150
query92	60	50	50	50
query93	1158	555	535	535
query94	890	310	280	280
query95	351	260	268	260
query96	606	280	286	280
query97	3315	3165	3215	3165
query98	216	205	199	199
query99	1662	1447	1397	1397
Total cold run time: 303250 ms
Total hot run time: 192383 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.08 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1aad5c4f41bc20aa2b2e2391fb9255c93d4ec476, data reload: false

query1	0.03	0.02	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.10
query5	0.52	0.50	0.53
query6	1.14	0.74	0.73
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.56	0.49	0.50
query10	0.55	0.55	0.56
query11	0.15	0.10	0.10
query12	0.13	0.12	0.11
query13	0.62	0.61	0.60
query14	2.72	2.76	2.85
query15	0.87	0.82	0.82
query16	0.38	0.39	0.38
query17	0.99	1.04	1.06
query18	0.24	0.22	0.21
query19	1.96	1.85	1.98
query20	0.01	0.01	0.02
query21	15.36	0.57	0.57
query22	2.47	2.33	1.71
query23	17.03	0.86	0.74
query24	3.56	1.49	0.47
query25	0.37	0.14	0.07
query26	0.35	0.13	0.14
query27	0.06	0.04	0.04
query28	10.40	1.10	1.07
query29	12.59	3.22	3.22
query30	0.25	0.06	0.06
query31	2.85	0.39	0.38
query32	3.26	0.46	0.45
query33	3.00	2.99	3.01
query34	17.01	4.49	4.48
query35	4.50	4.53	4.54
query36	0.68	0.47	0.50
query37	0.09	0.06	0.06
query38	0.06	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.05 s
Total hot run time: 32.08 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Thearas @doris-robot @dataroaring @BiteTheDDDDt