Add libei backend support (and thus ability to run on Wayland) #1594
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
p12tic
force-pushed
the
libei
branch
2 times, most recently
from
8860f59
to
32d03eb
Compare
Modified-by: Povilas Kanapickas <[email protected]>
Unless run with input-leapc --disable-portal, use the org.freedesktop.portal.RemoteDesktop portal (via libportal) to get a socket to the compositor EIS implementation. libportal requires a glib mainloop [1]. It's hooked in via a thread in EiEventQueueBuffer. This is the lowest-level we can add a glib mainloop without having to modify everything else in InputLeap (particularly EventQueue). In the glib thread we connect to the portal, request a file descriptor and pass that via a custom connectedToEis event to the screen. The screen can take this and set up the ei backend. [1] the alternative would be implementing the portal interaction ourselves but that's surprisingly complicated and the only good C DBus bindings are Glib (requires a mainloop, goto 10) or sd-bus (requires an sd-bus mainloop, goto 10). Modified-by: Povilas Kanapickas <[email protected]>
For capturing input events, purely libei is not really sufficient - that is just the transport layer for the events. Instead, we have a new XDG portal for InputCapture which provides us with the (display-system independent) meta-information we need to set up pointer barriers. Modified-by: Povilas Kanapickas <[email protected]>
Modified-by: Povilas Kanapickas <[email protected]>
Provide a shell wrapper "input-leap-flatpak" that checks for one of two configuration files. If the $XDG_CONFIG_HOME/InputLeap/server.name file exists, it is used as server location for input-leapc. (Note that flatpak doesn't allow mDNS so you can't use foo.local) Otherwise, if InputLeap.conf exists in that directory we start the server with that config file. Note that input-leaps is untested because it requires the xdg-desktop-portal and various other bits in place which I don't have all ready yet. The script overrides the LIBEI_SOCKET environment variable to be in XDG_RUNTIME_DIR/eis-0, so placing a socket there will enable the existing hacks to connect directly. Modified-by: Povilas Kanapickas <[email protected]>
Release is required for transitioning between displays, to restore normal event processing on the server.
Modified-by: Povilas Kanapickas <[email protected]>
No functional change, just makes the code easier to read (and write).
We do not actually warp the cursor, just update the current location. Modified-by: Povilas Kanapickas <[email protected]>
So we can move back from the secondary screen and use the devices.
Still quite crude, it does not support modifiers and does not take into account wheel axis (which are handled separately). Modified-by: Povilas Kanapickas <[email protected]>
Still quite crude, it does not handle modifiers. Modified-by: Povilas Kanapickas <[email protected]>
And keep it updated on key events. Modified-by: Povilas Kanapickas <[email protected]>
Modified-by: Povilas Kanapickas <[email protected]>
Modified-by: Povilas Kanapickas <[email protected]>
InputLeap does its best to replicate a key + modifier to achieve the desired symbol. To do, InputLeap considers the modifiers required to get the symbol, but also the modifiers the keyval is sensitive to. getKeymap() would however add only one key item entry for all the required modifiers, which later breaks the logic in the keymap code KeyMap::keysForKeyItem() as it would try to press all the modifiers to get the desired symbol. In the case of Shift, it also meant pressing CapsLock, which obviously breaks the logic, as pressing Shift while CapsLock is enabled would produce the lowercase symbol. Fix EiKeyState::getKeyMap() to add a different key entry for all the required modifiers. Modified-by: Povilas Kanapickas <[email protected]>
Modified-by: Povilas Kanapickas <[email protected]>
To avoid drifting into the large values when the pointer is on the secondary screen (which can lead to an abort of InputLeap eventually). Modified-by: Povilas Kanapickas <[email protected]>
The API of the activated and deactivated callbacks include the activation ID. Fix the code to match the API, so we actually get the GVariant options and can receive the pointer location at activation. Signed-off-by: Olivier Fourdan <[email protected]> Modified-by: Povilas Kanapickas <[email protected]>
Add active state query to keep track when the input capture is actually active. The goal is to avoid having the input capture sending events to a remote client when there is no mapping client, which eventually leads either to the wrong client or no motion at all. Signed-off-by: Olivier Fourdan <[email protected]> Modified-by: Povilas Kanapickas <[email protected]>
Our implementation of input capture sets barriers on the edges of all the zones, regardless of the actual machines layout. That means that input capture will activate every time the pointer reaches the edge of the screen. When there is no secondary screen found at the given edge, we might either jump to the wrong secondary screen or worse, stop pointer input entirely. To avoid this, simply check whether the input capture is active while we're still on the primary screen, and disable input capture when that's the case. Signed-off-by: Olivier Fourdan <[email protected]> Modified-by: Povilas Kanapickas <[email protected]>
We do not know where on screen the cursor is actually located, even less after a relayout. Simple assume the cursor is right in the center of the screen. Signed-off-by: Olivier Fourdan <[email protected]>
libxkbcommon fails to compile a keymap that contains a terminating null byte if passed as buffer. Ensure we always have a null-terminated string instead and pass it as such to the keymap compiler. This avoids issues when different EIS implementation give us the keymap with or without that trailing null. Modified-by: Povilas Kanapickas <[email protected]>
We keep the XKB state around, but the XKB state depends on the keymap. At startup, we create a default keymap, then later replaced by the actual keymap. However, we would not update the XKB state when doing so, and we would still be using the old (default US), keymap. To solve the problem, make sure to recreate both the keymap and the XKB state in sync. Modified-by: Povilas Kanapickas <[email protected]>
This was referenced Jan 21, 2023
|
is there any chance this would work on kde plasma? not just gnome? |
|
Sure, there is nothing binding us to gnome in the patches here and inputleap doesn't even know which compositor is running - the only communication here is with the portal via DBus and then the events via libei. In fact during development I had a python program faking the portal and the eis-demo-server tool from the libei git repo take the place of the compositor. |
|
Has anyone tried this with any wlroots based display manager such as Wayfire or Sway? Do you think there's any point in spending time trying to build for that, or do you think it more likely to be even further broken there? |
It needs support in the compositor, wlroots does not have support for libei yet, and also the bits in the wlroots portal. |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a rebased #1524 which contains support for libei and consequently is able to run on Wayland. Currently this only works on a custom branch of Mutter and thus is not friendly to end users, but at least something works.
libei support is currently disabled by default. It can be enabled by passing
INPUT_LEAP_BUILD_LIBEI=TRUEto cmake. E.g., if building viaclean-build.sh:env B_CMAKE_FLAGS="-DINPUTLEAP_BUILD_LIBEI=TRUE" ./clean_build.sh.Testing the functionality requires the following custom packages:
mutter https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2628 (tested with 1ffe5c4818)
libportal (Add support for the InputCapture portal flatpak/libportal#96 (tested with e1908e7a41)
xdg-desktop-portal Add a InputCapture portal flatpak/xdg-desktop-portal#714 (tested with c1aa05091e)
xdg-desktop-portal-gnome https://gitlab.gnome.org/GNOME/xdg-desktop-portal-gnome/-/merge_requests/61 (tested with 532889384)
libei https://gitlab.freedesktop.org/libinput/libei (tested 0.4.1 tag)
What currently works: InputLeap client and server, only simple mouse and keyboard actions on Mutter.
What currently does not work:
Clipboard sharing.
Encryption (even though libei backend is not involved with TCP connection between InputLeap server and client)
The integration with Mutter permission dialogs could be better.
Task bar icon support.
No release note have been added in this PR. As far as end users are concerned Wayland is still not supported until there is support in upstream mutter.
Thanks a lot to @whot and @ofourdan who made this happen!
I want to stress that this particular PR is only a very small part of the puzzle required to get InputLeap to work on Wayland. Going by lines of code this PR is less than 5% of the total amount of changes. @whot and @ofourdan were working in this area for the past several years, without their work this PR couldn't exist.
The original PR description by @whot is below.
libei is a library for Emulated Input and a replacement for what is XTEST under X11. Unlike XTEST it's windowing-system agnostic, it's just a transport layer for sending logical input events. See the libei README for details on the motivation etc.
libei clients like input-leap can work as receiver context (barriers, real input events are sent to us via libei) or as sender context (barrierc, emulated events are sent by us to the compositor). However, libei is only the transport layer, the logic of establishing a connection is punted to the portals, in particular the
RemoteDesktopportal for barrierc and the newInputCaptureportal for barriers.For the portal communication we're using libportal which requires a GLib mainloop (and thus a separate thread).
This PR adds the required bits to:
--use-eiin barrierc/barriers)barriercto talk to org.freedesktop.portal.RemoteDesktop.ConnectToEIS to set up libei and then use those devices to send input eventsbarriersto talk to org.freedesktop.portal.InputCapture to set up pointer barriers and get input events from the compositorTo actually run this, you will need to replace a bunch of components, see the tracking issue on libei. As of opening the PR, you will need updated mutter, libportal, xdg-desktop-portal, xdg-desktop-portal-gnome and of course libei itself.
The input-leap internal bits are unchanged, so you can use a libei-enabled barriers to talk to input-leap's barrierc on Win/MacOs/X11 and so on.
cc @ofourdan who deserves most of the credit for the hard parts