Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented native inputhooks for MacOSX backend #156

Closed
wants to merge 4 commits into from
Closed

Implemented native inputhooks for MacOSX backend #156

wants to merge 4 commits into from

Conversation

skrisna
Copy link

@skrisna skrisna commented Dec 14, 2019

Pulled in IPython (7.10.1) code for MacOSX backend. This code contains
MacOSX-native inputhooks. It cannot directly be incorporated IPython uses
prompt_tooklit for AsyncIO. In order to avoid dependency on prompt_toolkit,
wrote a custom event-loop trigger. See details below.

Benefits:

  • The MacOSX event loop is interrupted only on user input. This greatly
    improves responsiveness of the backend. The backend now feels more fluid and
    mac-like.
  • In PyCharm, the figure does not remain in the forefront even after Alt-Tab.

Impelementation Details:
IPython uses prompt_toolkit trigger mechanism to initiate callback on user
input. To avoid the prompt_toolkit dependency, a similar trigger mechanism
using pipes is impelemented along with dedicated thread to check for user
input (by checking stdin_read()). When user input is available, this thread
terminates the event loop using the pipe functionality similar to IPython.

Updates

Pulled in IPython (7.10.1) code for MacOSX backend. This code contains
MacOSX-native inputhooks. It cannot directly be incorporated IPython uses
prompt_tooklit for AsyncIO. In order to avoid dependency on prompt_toolkit,
wrote a custom event-loop trigger. See details below.

Benefits:
- The MacOSX event loop is interrupted only on user input. This greatly
  improves responsiveness of the backend. The backend now feels more fluid and
  mac-like.
- In PyCharm, the figure does not remain in the forefront even after Alt-Tab.

Impelementation Details:
IPython uses prompt_toolkit trigger mechanism to initiate callback on user
input. To avoid the prompt_toolkit dependency, a similar trigger mechanism
using pipes is impelemented along with dedicated thread to check for user
input (by checking stdin_read()). When user input is available, this thread
terminates the event loop using the pipe functionality similar to IPython.

Updates
@skrisna
Copy link
Author

skrisna commented Dec 14, 2019

For reference, commented on a long-pending request on PyCharm forum asking for a fix to this problem: https://youtrack.jetbrains.com/issue/PY-28781#focus=streamItem-27-3868844.0-0

Krishna Subramanian added 2 commits December 17, 2019 12:04
Implemented a singleton resource manager to avoid creating pipe
resources for every event loop. This also plays nicely with the native
hooks. In a previous implementation, closing the pipes lead the native
callbacks getting triggered immediately.
import os
from pydev_ipython.inputhook import stdin_ready
import time
from threading import Thread, Event
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change this import for: from _pydev_imps._pydev_saved_modules.threading import Thread, Event ?

The main issue is that gevent monkey-patches threading and that import will make sure that the original (non monkey-patched version) of the module is used.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the commit 7f755ca [1].

[1] 7f755ca


def inputhook_mac():

import datetime
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this import top-level? The reason for this is that Python 2 has an import lock, so, if a thread is stopped in a breakpoint inside of some import no other import will run (so, in the debugger imports must be top-level).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore.

gevent monkey-patches threading and this import will make sure that the
original (non monkey-patched version) of the module is used.
@skrisna
Copy link
Author

skrisna commented Dec 23, 2019

@fabioz I did some more testing of the commit and the commit itself seems to work well.

But I noticed that on a MacOSX/Pycharm and using either the MacOSX or WXAgg backends, the responsiveness on new code entry starts out well but after a time, there is a 10second delay for any new input in the pydev_code_executor/add_exec loop. This happens only when the matplotlib window is not in focus. When it is in focus, the responsiveness on new code entry is immediate.

This problem is not in IPython for either of these backends. In IPython, even after a long delay, the 10 second delay does not kick-in with or without the matplotlib window being in focus and the responsiveness is consistently good.

Do you know the source of this difference? Once difference could be because IPython uses prompt_toolkit/asyncio and PyDev uses XML-RPC. But don't know enough to understand this further.

@skrisna
Copy link
Author

skrisna commented Dec 24, 2019

I figured out the issue. PyCharm was under App Nap and the terminal for ipython was not. So the differences between the two.

I was not able to disable App Nap only for PyCharm so I disabled App Nap across all applications. After that, I am no longer facing any issues.

@fabioz I did some more testing of the commit and the commit itself seems to work well.

But I noticed that on a MacOSX/Pycharm and using either the MacOSX or WXAgg backends, the responsiveness on new code entry starts out well but after a time, there is a 10second delay for any new input in the pydev_code_executor/add_exec loop. This happens only when the matplotlib window is not in focus. When it is in focus, the responsiveness on new code entry is immediate.

This problem is not in IPython for either of these backends. In IPython, even after a long delay, the 10 second delay does not kick-in with or without the matplotlib window being in focus and the responsiveness is consistently good.

Do you know the source of this difference? Once difference could be because IPython uses prompt_toolkit/asyncio and PyDev uses XML-RPC. But don't know enough to understand this further.

@skrisna
Copy link
Author

skrisna commented Mar 28, 2020

I was not able to disable App Nap only for PyCharm so I disabled App Nap across all applications. After that, I am no longer facing any issues.

For those interested, I disabled Nap App on macos using the following -

defaults write NSGlobalDomain NSAppSleepDisabled -bool YES

@liamtoney
Copy link

liamtoney commented Oct 3, 2021

@skrisna, it looks like this hack doesn't work in PyCharm with cffi=1.14.6. I get a RuntimeError: ffi_prep_cif_var failed when importing interactive Matplotlib in the python console. If I downgrade to cffi=1.14.4 it works fine... any ideas?

@liamtoney
Copy link

@skrisna, it looks like this hack doesn't work in PyCharm with cffi=1.14.6. I get a RuntimeError: ffi_prep_cif_var failed when importing interactive Matplotlib in the python console. If I downgrade to cffi=1.14.4 it works fine... any ideas?

Things also work with cffi=1.14.5.

@Jofkos
Copy link

Jofkos commented Jan 10, 2022

@skrisna, it looks like this hack doesn't work in PyCharm with cffi=1.14.6. I get a RuntimeError: ffi_prep_cif_var failed when importing interactive Matplotlib in the python console. If I downgrade to cffi=1.14.4 it works fine... any ideas?

I had the same issue and so did people over at ipython, so I applied their fixes and this seems to have done the trick (see #209).

@liamtoney
Copy link

@skrisna, it looks like this hack doesn't work in PyCharm with cffi=1.14.6. I get a RuntimeError: ffi_prep_cif_var failed when importing interactive Matplotlib in the python console. If I downgrade to cffi=1.14.4 it works fine... any ideas?

I had the same issue and so did people over at ipython, so I applied their fixes and this seems to have done the trick (see #209).

Awesome, thanks @Jofkos. I'll give the code over in #209 a shot.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants