Cannot run nova in VirtualBox VM: terminated by signal SIGILL (Illegal instruction)

[ahaoboy]

close: ahaoboy/js-engine-benchmark#10

#796

Ubuntu 2404:
nova eval ./dist/run.js
fish: Job 1, 'nova eval ./dist/run.js' terminated by signal SIGILL (Illegal instruction)

[aapoalas]

Which executable are you using? This should've been fixed in #850

[ahaoboy]

I remember nova used to be able to run, but it does not work on Virtual Box virtual machines.

download https://github.com/trynova/nova/releases/download/latest/nova-linux-amd64

I tested it on CI and it seems there is a problem with the regular expression.

https://github.com/ahaoboy/nova-test/actions

https://github.com/ahaoboy/nova-test/actions/runs/17666877218/job/50210052788

RegExp: SyntaxError: regex parse error:
    HF(?=;)
      ^^^
error: look-around, including look-ahead and look-behind, is not supported

[aapoalas]

Hmmmm.... That's really interesting... It's pretty odd that it SIGILL's immediately with just running nova instead of eg. nova eval foo.js because that means it's not even getting into the main function. The very first thing the main function does is parses the CLI arguments using Clap, and that would exit with the help when you haven't given any arguments. Since that doesn't happen, we don't get that far.

Can you check what readelf -n /root/.ei/nova gives you?

[aapoalas]

Oh, and regarding the regex error: that is unlikely to get fixed any time soon. This PR would open up look-behind to Nova's RegExp but it seemed to me like the RegExp benchmark will test a lot of nooks and crannies of JS RegExp, including lookaheads and backreferences. Lookaheads are still possibly coming after the linked PR gets merged (if it ever does), but backreferences cannot be computed in linear time. The regex crate we use is linear-time only and thus will never support backreferences: I may be forking the crate at some point (to better align with the ECMAScript spec) but supporting backreferences is not going to happen any time soon.

[ahaoboy]

 u24~ 
 ll /root/.ei/nova
-rwxr-xr-x 1 root root 11M Sep 12 14:51 /root/.ei/nova*

 u24~ 
 md5sum /root/.ei/nova
bfc25835d64314d2e3b918c06e894f57  /root/.ei/nova

 u24~ 
 readelf -n /root/.ei/nova

Displaying notes found in: .note.gnu.property
  Owner                Data size        Description
  GNU                  0x00000010       NT_GNU_PROPERTY_TYPE_0
      Properties: x86 ISA needed: x86-64-baseline

Displaying notes found in: .note.gnu.build-id
  Owner                Data size        Description
  GNU                  0x00000014       NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: c247b7238942f942aed3184b72295332f9214c5a

Displaying notes found in: .note.ABI-tag
  Owner                Data size        Description
  GNU                  0x00000010       NT_GNU_ABI_TAG (ABI version tag)
    OS: Linux, ABI: 3.2.0

Displaying notes found in: .note.stapsdt
  Owner                Data size        Description
  stapsdt              0x0000003c       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: job_evaluation_start
    Location: 0x00000000000a2f91, Base: 0x000000000086a690, Semaphore: 0x000000000096aa60
    Arguments: 8@%rdi
  stapsdt              0x0000003b       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: job_evaluation_done
    Location: 0x00000000000a30f4, Base: 0x000000000086a690, Semaphore: 0x000000000096aa62
    Arguments: 8@%rdi
  stapsdt              0x0000003d       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: eval_evaluation_start
    Location: 0x00000000002d9f56, Base: 0x000000000086a690, Semaphore: 0x000000000096aa64
    Arguments: 8@%rdi
  stapsdt              0x0000003d       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: eval_evaluation_start
    Location: 0x00000000002d9f7f, Base: 0x000000000086a690, Semaphore: 0x000000000096aa64
    Arguments: 8@%rdi
  stapsdt              0x0000003c       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: eval_evaluation_done
    Location: 0x00000000002dc4fa, Base: 0x000000000086a690, Semaphore: 0x000000000096aa66
    Arguments: 8@%rdi
  stapsdt              0x0000003c       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: eval_evaluation_done
    Location: 0x00000000002dc652, Base: 0x000000000086a690, Semaphore: 0x000000000096aa66
    Arguments: 8@%rdi
  stapsdt              0x0000003c       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: eval_evaluation_done
    Location: 0x00000000002dd2e5, Base: 0x000000000086a690, Semaphore: 0x000000000096aa66
    Arguments: 8@%rdi
  stapsdt              0x0000003f       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: module_evaluation_start
    Location: 0x00000000002f2359, Base: 0x000000000086a690, Semaphore: 0x000000000096aa68
    Arguments: 8@%rdi
  stapsdt              0x0000003e       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: module_evaluation_done
    Location: 0x00000000002f3736, Base: 0x000000000086a690, Semaphore: 0x000000000096aa6a
    Arguments: 8@%rdi
  stapsdt              0x0000003f       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: script_evaluation_start
    Location: 0x00000000002f7a62, Base: 0x000000000086a690, Semaphore: 0x000000000096aa6c
    Arguments: 8@%rdi
  stapsdt              0x0000003e       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: script_evaluation_done
    Location: 0x00000000002fa32c, Base: 0x000000000086a690, Semaphore: 0x000000000096aa6e
    Arguments: 8@%rdi
  stapsdt              0x0000002a       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: gc_start
    Location: 0x0000000000319712, Base: 0x000000000086a690, Semaphore: 0x000000000096aa70
    Arguments: 
  stapsdt              0x00000029       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: gc_done
    Location: 0x0000000000331dd0, Base: 0x000000000086a690, Semaphore: 0x000000000096aa72
    Arguments: 
  stapsdt              0x00000044       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: javascript_call_start
    Location: 0x00000000005e7df6, Base: 0x000000000086a690, Semaphore: 0x000000000096aa74
    Arguments: 8@%rdi 8@%rsi
  stapsdt              0x00000041       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_call_start
    Location: 0x00000000005e7f3c, Base: 0x000000000086a690, Semaphore: 0x000000000096aa76
    Arguments: 8@%rdi 8@%rsi
  stapsdt              0x0000003c       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: javascript_call_done
    Location: 0x00000000005e8393, Base: 0x000000000086a690, Semaphore: 0x000000000096aa78
    Arguments: 8@%rdi
  stapsdt              0x00000039       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_call_done
    Location: 0x00000000005e857b, Base: 0x000000000086a690, Semaphore: 0x000000000096aa7a
    Arguments: 8@%rdi
  stapsdt              0x0000004b       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: javascript_constructor_start
    Location: 0x00000000005e90bd, Base: 0x000000000086a690, Semaphore: 0x000000000096aa7c
    Arguments: 8@%rdi 8@%rsi
  stapsdt              0x00000048       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_constructor_start
    Location: 0x00000000005e919e, Base: 0x000000000086a690, Semaphore: 0x000000000096aa7e
    Arguments: 8@%rdi 8@%rsi
  stapsdt              0x00000048       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_constructor_start
    Location: 0x00000000005e932b, Base: 0x000000000086a690, Semaphore: 0x000000000096aa7e
    Arguments: 8@%rdi 8@%rsi
  stapsdt              0x00000040       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_constructor_done
    Location: 0x00000000005e947e, Base: 0x000000000086a690, Semaphore: 0x000000000096aa80
    Arguments: 8@%rdi
  stapsdt              0x00000043       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: javascript_constructor_done
    Location: 0x00000000005ea724, Base: 0x000000000086a690, Semaphore: 0x000000000096aa82
    Arguments: 8@%rdi
  stapsdt              0x00000040       NT_STAPSDT (SystemTap probe descriptors)
    Provider: nova_vm
    Name: builtin_constructor_done
    Location: 0x00000000005ea7b3, Base: 0x000000000086a690, Semaphore: 0x000000000096aa80
    Arguments: 8@%rdi

[aapoalas]

Thanks; I have a test build running here: https://github.com/trynova/nova/actions/runs/17667587294/job/50212049458?pr=854 (oh I think it finished just now).

Artifact URL: https://github.com/trynova/nova/actions/runs/17667587294/artifacts/3993701847

Can you test if that works on VirtualBox?

[ahaoboy]

I recreated the VM and it still doesn't work. However, it works fine on CI, which suggests there might be a problem with VirtualBox or some VM settings.

Ignore this for now and see if anyone else has encountered the same issue.

[aapoalas]

I'll produce one more test executable; I want to see if it's the USDTs that are causing the issues or not.

[aapoalas]

Aight, if you could try this https://github.com/trynova/nova/actions/runs/17669046333/artifacts/3994159695 to see if it works now then that'd be swell.

If it works then it... well, then I'm quite confused honestly :D If it doesn't, then this has nothing to do with the USDTs that were causing similar issues on ARM because I'd made a booboo. In that case this is more likely about SSE4/AVX instructions being present in the executable and VirtualBox not supporting them.

[ahaoboy]

Most likely does not support SSE4/AVX

 u24~ 
 cpuid | grep -E 'sse4|avx'

 u24~ 
 lscpu | grep -E 'sse4|avx'

Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt aes rdrand hypervisor lahf_lm abm 3dnowprefetch ibrs_enhanced fsgsbase bmi1 bmi2 invpcid rdseed adx clflushopt sha_ni arat md_clear flush_l1d arch_capabilities

 u24~ 
 

[ahaoboy]

virtualbox
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 movbe popcnt aes rdrand hypervisor lahf_lm abm 3dnowprefetch ibrs_enhanced fsgsbase bmi1 bmi2 invpcid rdseed adx clflushopt sha_ni arat md_clear flush_l1d arch_capabilities

ci
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves user_shstk clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm

[aapoalas]

@ahaoboy Yup, but then it's a question of whether that's actually the problem. Can you try running this executable from above (https://github.com/trynova/nova/actions/runs/17669046333/artifacts/3994159695) in the VirtualBox?

[ahaoboy]

noop :(

Maybe you can add nightly.link to download the latest CI products

https://nightly.link/

[aapoalas]

noop :(

Okay, this is actually good news! It means that the USDTs were not to blame for this and thus it is indeed fairly likely that this is something to do with eg. AVX or SSE4 instructions.

And actually yeah, I just tested and indeed there are SSE4 instructions (specifically only lzcnt) and AVX instructions (many, vaddsd, vsubpd, ...) in the executable. So it may well be that VirtualBox is the blocking factor here.