New asyncWrap helper to execute blocking calls on the native Qt event loop

[kristjanvalur]

This pr introduces a helper, call_sync, to call a synchronous method via the Qt event loop.
This is useful to use from asynchronous methods, for example to invoke modal dialogue boxes via the .exec() method.

If this is invoked directly from an async method, it can cause task re-entrancy in the asyncio scheduler.

By using call_sync, the re-entrant event loop call is done from the event loop itself with no actually running task in the scheduler.

the name call_sync is just a suggestion, happy to change it.

If this is not a good addition, then at least we could have a similar example to show how to achieve this.

[johnzhou721]

My 2 cents: Can't you already wrap the syncronous function into an async one and call it from the asyncio event loop?

[kristjanvalur]

My 2 cents: Can't you already wrap the syncronous function into an async one and call it from the asyncio event loop?

This is a wrapping. What other wrapping are you suggesting? The trick here is to run the blocking method on the main thread, while the async caller awaits it, so that the event loop is not re-entered from the async method.

Maybe I wasn't clear, but this is mostly to help with calling methods, like dialogue boxes, that re-enter the QT event loop. If called synchronously from an async slot, e.g. when popping up a modal, then your async loop can break, because you re-enter the task scheduling loop.

This is a way to wrap the function to run on the same thread, effectively blocking the current task, but awaiting int in the async method, so that the called function can also call QT stuff.

I think this is the only missing bit in converting a classic pyqt app into an async one. I was hit by so much trouble with modal dialogue boxes calling .exec() to run them, which could cause re-entrancy and task errors. This is the correct way to call such methods from a task. I had to google a lot and roll my own to solve that problem. I think it ought to be part of qasync, as necessary plumbing to get back into synchronous qt from an async slot.

[kormax]

I'd like to ask a question about processEvents() invocation used in the provided "failing" example.

Is there any reason why making that function async and adding an asyncio.sleep(0) invocation instead, which would return control back to the loop, allowing it to start the task, wouldn't fit your case?

UPD. If there's something preventing one from doing that with real use cases, like with modals, it would be great to look at a more direct example.

[johnzhou721]

My 2 cents: Can't you already wrap the syncronous function into an async one and call it from the asyncio event loop?

This is a wrapping. What other wrapping are you suggesting? The trick here is to run the blocking method on the main thread, while the async caller awaits it, so that the event loop is not re-entered from the async method.

Maybe I wasn't clear, but this is mostly to help with calling methods, like dialogue boxes, that re-enter the QT event loop. If called synchronously from an async slot, e.g. when popping up a modal, then your async loop can break, because you re-enter the task scheduling loop.

This is a way to wrap the function to run on the same thread, effectively blocking the current task, but awaiting int in the async method, so that the called function can also call QT stuff.

I think this is the only missing bit in converting a classic pyqt app into an async one. I was hit by so much trouble with modal dialogue boxes calling .exec() to run them, which could cause re-entrancy and task errors. This is the correct way to call such methods from a task. I had to google a lot and roll my own to solve that problem. I think it ought to be part of qasync, as necessary plumbing to get back into synchronous qt from an async slot.

Sorry; just saw how the trick applies. Thanks!

[kormax]

UPD. If there's something preventing one from doing that with real use cases, like with modals, it would be great to look at a more direct example.

Looking at QMessageBox.information example, it started to make sense to me.

So, what is done here is lifting the continuation all the way up to the native Qt Event loop via QTimer, which prevents blocking of the python QEventLoop, as the continuation operates as another callback on native loop, which also allows you to continue native loop execution using some of the compatible Qt native methods directly, like processEvents, without keeping the python loop blocked.

This is useful, but I wonder if another name would be more expressive of what it actually does? Maybe something like run_on_native_q_event_loop?

[kristjanvalur]

Yeah, the examples/modal_example.py is intended to illustrate this.
The key here is to allow event loop re-entranacy

I agree that call_sync() is not very descriptive and can be improved, which is why I left this in draft mode.

The key here is that it allows async tasks to call the event loop again, re-enter it, such as when running and waiting for dialogue boxes. For me, it is sort of the opposite of asyncSlot, in that it serves as an interface to safely calling synchronous handlers from asynchronous handlers. It can be used to call (and wait for) synchronous slots, that might put up message boxes, or it can be used directly for the message box.

The thing is, once you start using async slots in your app, soon the whole app migrates to one. Writing async logic, particularly more complex, longer running jobs, is the whole reason for doing so. But occasionally, those longer running slots will want to do something like displaying a confirmation message. This is where that comes in.

Some suggestions:

• call_sync()
• from_async()
• run_in_event_loop()
• yield_to()
• fromAsyncSlot()

Whatever you think best.

Also, btw, I guess the Readme could do with a short description of the available methods/functions. (And not all of the api is exported from __init__.__all__)

[kristjanvalur]

hm, I guess a simpler version, one that doesn't await, could be useful too, if one simply wants to fire off a sync slot and not wait for it to end. but this await mechanism is essential for confirmation boxes, etc. and the non-awaiting version can be done by creating a Task...

[kormax]

But occasionally, those longer running slots will want to do something like displaying a confirmation message.

I think this would be great to list native methods that this could be useful with inside of the function docstring.

As a dev who had worked with QML exclusively, I was unaware of native methods like QMessageBox.*** that could crank the native event loop forward while "blocking" sync functions. Is that common for Qt Widgets?

I was aware about using processEvents/exec directly, as that's something used by libraries like pytest-qt for non-blocking sync wait, but not the other stuff.

[hosaka]

I can see the benefit adding this to help handling modals, confirmation dialogues etc. As for the name (keeping the camel case because we're stuck with asyncSlot and asyncClose) something along the lines of:

• syncCall
• syncWrap
• fromAsyncSlot - but is it always meant to be ran from an async slot?

The docstring can be a bit more descriptive about in which situation a user might want to reach for this function, maybe listing a couple of common cases/native functions that you have experienced before.

[kristjanvalur]

As a dev who had worked with QML exclusively, I was unaware of native methods like QMessageBox.*** that could crank the native event loop forward while "blocking" sync functions. Is that common for Qt Widgets?

Interesting. I am new to qt, but inherited an application in pyqt6 that I needed to convert to async. It had lots of those little confirmation modals run directly. I just assumed that this was how it was commonly done.

[johnzhou721]

Also another cosmetic question for @hosaka: test_qeventloop.py is getting huge, should we split it? Almost everything is related to qeventloop. If so, this should probably be in a separate PR.

[hosaka]

@johnzhou721 I think time is better spent elsewhere than splitting test_qeventloop.py. If all the tests are related to QEventLoop, let's not fix what's not broken, a 1k LOC is not what i'd call "huge" yet.

[hosaka]

Any objections to calling the function asyncWrap?

[kormax]

I think that the new docstring is great and properly explains the purpose of the method - at this point, the name does not matter that much.

If this question refers to my previous comment, my concern regarding using "generic" names was that, some users, without consulting the documentation, might mistakenly infer from the name alone that this method automatically converts all synchronous or blocking functions into fully asynchronous-compatible ones, causing them to misuse it. But technically, such misuse is also possible if users put asyncSlot() everywhere thinking it would increase performance or magically unblock the UI... So better not to overthink it.

[hosaka]

Same, I'm not too bothered about the name. I wasn't referring to your comment specifically, just had to post something to get everyones attention :) @kristjanvalur if you're okay with my changes, we can merge this

[kristjanvalur]

Same, I'm not too bothered about the name. I wasn't referring to your comment specifically, just had to post something to get everyones attention :) @kristjanvalur if you're okay with my changes, we can merge this

Sure. Sorry, I've been on vacation and thus a bit late to respond. Great to see you appreciate this, thanks for the changes. Cheers!

[kristjanvalur]

💯