Threading error for vmedian

[Beforerr]

MWE. However change the number to smaller (10000) or larger (30000) do not error.

julia> vmedian(diff(Millisecond.(0:20000)))
ERROR: StackOverflowError:
Stacktrace:
  [1] _exec
    @ ./pcre.jl:205 [inlined]
  [2] exec
    @ ./pcre.jl:199 [inlined]
  [3] exec_r_data(re::Ptr{Nothing}, subject::String, offset::Int64, options::UInt32)
    @ Base.PCRE ./pcre.jl:222
  [4] _findnext_re(re::Regex, str::String, idx::Int64, match_data::Ptr{Nothing})
    @ Base ./regex.jl:479
  [5] findnext
    @ ./regex.jl:467 [inlined]
  [6] iterate
    @ ./strings/util.jl:572 [inlined]
  [7] iterate
    @ ./strings/util.jl:571 [inlined]
  [8] _collect(cont::UnitRange{Int64}, itr::Base.SplitIterator{String, Regex}, ::Base.HasEltype, isz::Base.SizeUnknown)
    @ Base ./array.jl:727
  [9] collect
    @ ./array.jl:716 [inlined]
 [10] #split#582
    @ ./strings/util.jl:739 [inlined]
 [11] split
    @ ./strings/util.jl:737 [inlined]
 [12] normpath(path::String)
    @ Base.Filesystem ./path.jl:394
 [13] fixup_stdlib_path
    @ ./methodshow.jl:138 [inlined]
 [14] macro expansion
    @ ./logging/logging.jl:410 [inlined]
 [15] macro expansion
    @ ~/.julia/packages/LoopVectorization/ImqiY/src/condense_loopset.jl:1179 [inlined]
 [16] quickselect!(A::Vector{Millisecond}, iₗ::Int64, iᵤ::Int64, k::Int64)
    @ VectorizedStatistics ~/.julia/packages/VectorizedStatistics/lBP0N/src/quicksort.jl:88
 [17] quickselect!(A::Vector{Millisecond}, iₗ::Int64, iᵤ::Int64, k::Int64) (repeats 7418 times)
    @ VectorizedStatistics ~/.julia/packages/VectorizedStatistics/lBP0N/src/quicksort.jl:118
 [18] _vmedian!(A::Vector{Millisecond}, ::Colon)
    @ VectorizedStatistics ~/.julia/packages/VectorizedStatistics/lBP0N/src/vmedian.jl:87
 [19] _vmedian!
    @ ~/.julia/packages/VectorizedStatistics/lBP0N/src/vmedian.jl:49 [inlined]
 [20] vmedian!
    @ ~/.julia/packages/VectorizedStatistics/lBP0N/src/vmedian.jl:48 [inlined]
 [21] vmedian(A::Vector{Millisecond})
    @ VectorizedStatistics ~/.julia/packages/VectorizedStatistics/lBP0N/src/vmedian.jl:54

[brenhinkeller]

Ach, I'm not sure if this is a heisenbug or just a consequence of the recursive design we used here plus a particularly unfortunate / hard to sort input sequence..

How reproducible is this particular this for you? I haven't gotten it to reproduce yet, but may be a difference in hardware or Julia version

[Beforerr]

I attached the versioninfo below, quite reproducible when n is between between 15000 and 20000 for Millisecond.(0:n). Smaller and larger n do not have problems...

julia> versioninfo()
Julia Version 1.11.6
Commit 9615af0f269 (2025-07-09 12:58 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 12 × Apple M2 Max
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 8 default, 0 interactive, 4 GC (on 8 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_VSCODE_REPL = 1
  JULIA_NUM_THREADS = auto

[brenhinkeller]

Huh, I get

julia> using VectorizedStatistics, Dates

julia> vmedian(diff(Millisecond.(0:20000)))
1 millisecond

julia> versioninfo()
Julia Version 1.11.6
Commit 9615af0f269 (2025-07-09 12:58 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 10 × Apple M1 Max
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

[brenhinkeller]

(@v1.11) pkg> st VectorizedStatistics
Status `~/.julia/environments/v1.11/Project.toml`
  [3b853605] VectorizedStatistics v0.5.10

in case that's relevant

[Beforerr]

I am also using 0.5.10. Now I think this is a threading problem. With Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores) the problem goes away.

[Beforerr]

Confirmed this fixed. Just wondering why change the number would fix it? Like a Magical number

[brenhinkeller]

Since the implementation for quickselect! is recursive, sometimes this seems to have hit Julia's recursion depth limit (hence the stack overflow), but there is less chance of that happening the smaller the input array is. For large enough arrays Base.partialsort is more efficient anyways I believe (though haven't benchmarked this in a while).