UBSan SEGV in `_Py_LazyJitTrampoline` when building with Clang and `--with-undefined-behavior-sanitizer` + experimental JIT

[ashm-dev]

Bug report

Bug description:

Environment

• CPython: main branch (local clone at .../cpython/main)

• Platform: Linux x86_64 (glibc)

• Compiler:

  • clang-18, clang-19, clang-20, clang-21 (see “Additional information”)
  • System default compiler (no CC / CXX set) – used for comparison

• Configure flags:

./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes

Steps to reproduce

With Clang (example with clang-21):

CC=clang-21 CXX=clang++-21 ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make

Actual behavior

During the build, _bootstrap_python crashes under UndefinedBehaviorSanitizer while freezing the abc module:

Crash log

./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==1010144==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x786f1ca45ff8 (pc 0x5feecae08630 bp 0x7ffe824db540 sp 0x7ffe824d5720 T1010144)
==1010144==The signal is caused by a READ memory access.
    #0 0x5feecae08630 in _Py_LazyJitTrampoline /home/shamil/oss/cpython/main/Python/jit.c:693:12
    #1 0x5feecaba2c40 in _PyEval_EvalFrameDefault /home/shamil/oss/cpython/main/Python/generated_cases.c.h:5495:13
    #2 0x5feeca7d564f in _PyEval_EvalFrame /home/shamil/oss/cpython/main/./Include/internal/pycore_ceval.h:121:16
    #3 0x5feeca7d564f in gen_send_ex2 /home/shamil/oss/cpython/main/Objects/genobject.c:259:24
    #4 0x5feeca7ce905 in gen_iternext /home/shamil/oss/cpython/main/Objects/genobject.c:635:9
    #5 0x5feeca81a7ae in list_extend_iter_lock_held /home/shamil/oss/cpython/main/Objects/listobject.c:1263:26
    #6 0x5feeca8122c0 in list_extend_impl /home/shamil/oss/cpython/main/Objects/listobject.c:1471:9
    #7 0x5feeca8122c0 in list_extend /home/shamil/oss/cpython/main/Objects/clinic/listobject.c.h:145:20
    #8 0x5feeca8122c0 in _PyList_Extend /home/shamil/oss/cpython/main/Objects/listobject.c:1480:12
    #9 0x5feeca726ca5 in PySequence_List /home/shamil/oss/cpython/main/Objects/abstract.c:2086:10
    #10 0x5feeca726ca5 in PySequence_Fast /home/shamil/oss/cpython/main/Objects/abstract.c:2117:9
    #11 0x5feecaa2167d in PyUnicode_Join /home/shamil/oss/cpython/main/Objects/unicodeobject.c:9960:12
    #12 0x5feecab87a03 in _PyEval_EvalFrameDefault /home/shamil/oss/cpython/main/Python/generated_cases.c.h:3902:35
    #13 0x5feecab4a2ad in _PyEval_Vector /home/shamil/oss/cpython/main/Python/ceval.c:2104:12
    #14 0x5feecab4a2ad in PyEval_EvalCode /home/shamil/oss/cpython/main/Python/ceval.c:944:21
    #15 0x5feecaed18b7 in run_eval_code_obj /home/shamil/oss/cpython/main/Python/pythonrun.c:1372:12
    #16 0x5feecaed1256 in run_mod /home/shamil/oss/cpython/main/Python/pythonrun.c:1475:19
    #17 0x5feecaecd6eb in pyrun_file /home/shamil/oss/cpython/main/Python/pythonrun.c:1300:15
    #18 0x5feecaecc28e in _PyRun_SimpleFileObject /home/shamil/oss/cpython/main/Python/pythonrun.c:521:13
    #19 0x5feecaecbc06 in _PyRun_AnyFileObject /home/shamil/oss/cpython/main/Python/pythonrun.c:81:15
    #20 0x5feecaf486bf in pymain_run_file_obj /home/shamil/oss/cpython/main/Modules/main.c:410:15
    #21 0x5feecaf486bf in pymain_run_file /home/shamil/oss/cpython/main/Modules/main.c:429:15
    #22 0x5feecaf47a95 in pymain_run_python /home/shamil/oss/cpython/main/Modules/main.c:691:21
    #23 0x5feecaf47a95 in Py_RunMain /home/shamil/oss/cpython/main/Modules/main.c:772:5
    #24 0x5feecb12bcc3 in main /home/shamil/oss/cpython/main/Programs/_bootstrap_python.c:100:12
    #25 0x786f1c62a574 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #26 0x786f1c62a627 in __libc_start_main csu/../csu/libc-start.c:360:3
    #27 0x5feeca499954 in _start (/home/shamil/oss/cpython/main/_bootstrap_python+0x775954) (BuildId: 7bed80e84f1f8597cf26ffc2c98fb769c5537a67)

==1010144==Register values:
rax = 0x0000786f1ca46000  rbx = 0x00005feecb965378  rcx = 0x0000000000000000  rdx = 0x0000000000000005
rdi = 0x0000786f1ca46000  rsi = 0x0000000000001000  rbp = 0x00007ffe824db540  rsp = 0x00007ffe824d5720
 r8 = 0x00000000ffffffff   r9 = 0x0000000000000000  r10 = 0x0000000000000022  r11 = 0x0000000000000246
r12 = 0x00005fef0c1a3970  r13 = 0x00005feecb70e550  r14 = 0x0000786f1c5af308  r15 = 0x0000786f1c5af298
UndefinedBehaviorSanitizer can not provide additional info.
SUMMARY: UndefinedBehaviorSanitizer: SEGV /home/shamil/oss/cpython/main/Python/jit.c:693:12 in _Py_LazyJitTrampoline
==1010144==ABORTING
make: *** [Makefile:1952: Python/frozen_modules/abc.h] Error 1

Expected behavior

The build should complete successfully, without an UndefinedBehaviorSanitizer crash in _Py_LazyJitTrampoline during module freezing.

Additional information

• The same crash reproduces with:

  CC=clang-18 CXX=clang++-18 ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make
  
  CC=clang-19 CXX=clang++-19 ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make
  
  CC=clang-20 CXX=clang++-20 ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make
  
  CC=clang-21 CXX=clang++-21 ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make

• With the same configure flags but without forcing Clang (no CC / CXX in the environment), the build finishes successfully and _bootstrap_python does not crash under UBSan:

  ./configure --with-pydebug --with-undefined-behavior-sanitizer --enable-experimental-jit=yes && make

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[ashm-dev]

And these issues may be related, since the segmentation fault occurs due to the same _Py_LazyJitTrampoline. But in the current bug, it occurs during build, and in the past, only when running certain tests.

Issue - #139834

[efimov-mikhail]

I was able to reproduce this crash on main once:

Crash log

./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
UndefinedBehaviorSanitizer:DEADLYSIGNAL
==3259649==ERROR: UndefinedBehaviorSanitizer: SEGV on unknown address 0x7f342aac8ff8 (pc 0x56321f0d8fc0 bp 0x7ffcc20aea80 sp 0x7ffcc20a8c60 T3259649)
==3259649==The signal is caused by a READ memory access.
    #0 0x56321f0d8fc0 in _Py_LazyJitTrampoline /home/sikko/projects/cpython/Python/jit.c:693:12
    #1 0x56321ee73a90 in _PyEval_EvalFrameDefault /home/sikko/projects/cpython/Python/generated_cases.c.h:5495:13
    #2 0x56321eaa698f in _PyEval_EvalFrame /home/sikko/projects/cpython/./Include/internal/pycore_ceval.h:121:16
    #3 0x56321eaa698f in gen_send_ex2 /home/sikko/projects/cpython/Objects/genobject.c:259:24
    #4 0x56321ea9fc45 in gen_iternext /home/sikko/projects/cpython/Objects/genobject.c:635:9
    #5 0x56321eaeba5e in list_extend_iter_lock_held /home/sikko/projects/cpython/Objects/listobject.c:1263:26
    #6 0x56321eae3570 in list_extend_impl /home/sikko/projects/cpython/Objects/listobject.c:1471:9
    #7 0x56321eae3570 in list_extend /home/sikko/projects/cpython/Objects/clinic/listobject.c.h:145:20
    #8 0x56321eae3570 in _PyList_Extend /home/sikko/projects/cpython/Objects/listobject.c:1480:12
    #9 0x56321e9f7fe5 in PySequence_List /home/sikko/projects/cpython/Objects/abstract.c:2086:10
    #10 0x56321e9f7fe5 in PySequence_Fast /home/sikko/projects/cpython/Objects/abstract.c:2117:9
    #11 0x56321ecf254d in PyUnicode_Join /home/sikko/projects/cpython/Objects/unicodeobject.c:9960:12
    #12 0x56321ee58853 in _PyEval_EvalFrameDefault /home/sikko/projects/cpython/Python/generated_cases.c.h:3902:35
    #13 0x56321ee1b0fd in _PyEval_Vector /home/sikko/projects/cpython/Python/ceval.c:2104:12
    #14 0x56321ee1b0fd in PyEval_EvalCode /home/sikko/projects/cpython/Python/ceval.c:944:21
    #15 0x56321f18f2b7 in run_eval_code_obj /home/sikko/projects/cpython/Python/pythonrun.c:1372:12
    #16 0x56321f18ec56 in run_mod /home/sikko/projects/cpython/Python/pythonrun.c:1475:19
    #17 0x56321f18b144 in pyrun_file /home/sikko/projects/cpython/Python/pythonrun.c:1300:15
    #18 0x56321f189da0 in _PyRun_SimpleFileObject /home/sikko/projects/cpython/Python/pythonrun.c:521:13
    #19 0x56321f18973d in _PyRun_AnyFileObject /home/sikko/projects/cpython/Python/pythonrun.c:81:15
    #20 0x56321f205dbf in pymain_run_file_obj /home/sikko/projects/cpython/Modules/main.c:410:15
    #21 0x56321f205dbf in pymain_run_file /home/sikko/projects/cpython/Modules/main.c:429:15
    #22 0x56321f2051ca in pymain_run_python /home/sikko/projects/cpython/Modules/main.c:691:21
    #23 0x56321f2051ca in Py_RunMain /home/sikko/projects/cpython/Modules/main.c:772:5
    #24 0x56321f3e8b43 in main /home/sikko/projects/cpython/Programs/_bootstrap_python.c:100:12
    #25 0x7f342ae88249 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #26 0x7f342ae88304 in __libc_start_main csu/../csu/libc-start.c:360:3
    #27 0x56321e787440 in _start (/home/sikko/projects/cpython/_bootstrap_python+0x752440) (BuildId: b5e13572959c6381f55515e608e9131c4d7769c3)

==3259649==Register values:
rax = 0x00007f342aac9000  rbx = 0x000056321fc004d0  rcx = 0x0000000000000000  rdx = 0x0000000000000005  
rdi = 0x00007f342aac9000  rsi = 0x0000000000001000  rbp = 0x00007ffcc20aea80  rsp = 0x00007ffcc20a8c60  
 r8 = 0x00000000ffffffff   r9 = 0x0000000000000000  r10 = 0x0000000000000022  r11 = 0x0000000000000246  
r12 = 0x000056322a5088c0  r13 = 0x000056321f9ab070  r14 = 0x00007f342adf7208  r15 = 0x00007f342adf7198  
UndefinedBehaviorSanitizer can not provide additional info.
SUMMARY: UndefinedBehaviorSanitizer: SEGV /home/sikko/projects/cpython/Python/jit.c:693:12 in _Py_LazyJitTrampoline
==3259649==ABORTING
make: *** [Makefile:1952: Python/frozen_modules/abc.h] Error 1

This log looks exactly like crash log in the first post.
Even register values are the same (only differences are pointer values).

But it's kinda hard to reproduce, a lot of builds are provided successfully.

CC @Fidget-Spinner