I think we would need to reformulate the binding as a C or Rust based extension rather than using ctypes

It seems that using CDLL rather than PyDLL will release the Gil.

We do use CDLL for unix like systems:

If I understand correctly, it’s the one for libjulia that matters.

yeah, I think because you're calling Julia via the libjulia, so whether or not GIL is dropped during call depends on how you loaded libjulia?

maybe it's worth getting perspective from @stevengj and @MilesCranmer ?

1. Can we by default drop GIL (i.e. use CDLL)?
2. Does it actually drop GIL and speed things up?
3. Does it create corretness issues?
4. Given 3, how can we drop GIL call by call

I would love to drop GIL. This would be great so that I can do other things in Python (like printing updates from another thread) while some large Julia processing is happening in the background.

Regarding 3, I'm not sure what correctness issues it would cause, if any. It's unclear to me if you'd be able to mutate any Julia variables while processing is already taking place, so maybe it's safe as-is...? Maybe we should just try it out and see.

yeah I'd like to try it, but I would prefer if we can present it as an option. In this case, it would be better if it can be a per-call site option, but a per-session option would do.

One case we know definitely won't work is if Julia needs to call Python stuff as call back, or mutate Python variable, but I feel like that's minor usage

I assume just having two libjulia handle should allow having this as a per call option with the ctypes api.

I can't quite figure out how to have a local testable pyjulia, clone and pip install -e . didn't quite work, or maybe I'm looking at the wrong place to print debug

I guess for one I can't figure out which handle is used when I do

import julia
#vs.
from julia import Base

I can't get any print statement to work -- these two must share the same Julia handle right? but which one / where I can't quite figure out

See also JuliaPy/PythonCall.jl#343

I think the main use case is when calling Julia code from Python, releasing the GIL would be useful.