Skip to content

feat(libSceVideoOut): capture the host cursor for guest mice and scale their deltas - #4217

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
Mouad-Scriptz-0:feat/videoout-mouse-capture
Oct 13, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
Mouad-Scriptz-0:feat/videoout-mouse-capture

Conversation

@Mouad-Scriptz-0

@Mouad-Scriptz-0 Mouad-Scriptz-0 commented Oct 12, 2026 •

Copy link
Copy Markdown

What

The console's mouse reports relative motion only, so a host cursor can never coincide with the cursor a title draws, and on a scaled desktop or a letterboxed window the title's cursor moved a different distance than the pointer. Minecraft (PPSA17221) draws its own cursor from sceMouseRead deltas: the host pointer drifted away from it and left the window.

  • MouseInput scales xrel/yrel from the presented image's size in window units to the title's display resolution, with sub-pixel carry so slow motion is not lost (SetPresentation); the presentation loop in VideoOutDriver feeds it the window size and the letterboxed size it already computes.
  • UpdateCapture puts SDL in relative mouse mode while the window is focused, a title has a mouse open (sceMouseOpen, exported as MouseIsOpen_nid_postfix from libSceMouse) and the pad's mouse-to-stick toggle is off, and releases it otherwise; PadInput::MouseModeEnabled() exposes the toggle.
  • New test target video_out_mouse_tests (video_out_mouse): deltas sum unchanged at 1:1, double at half size, and an empty presentation is ignored.
  • docs/user/INPUT_MAPPING.md describes the capture; TechnicalDebt records that the console's deltas are counts, so a title applying its own sensitivity still moves faster than the host pointer would.

Tested

Fedora 44 (kernel 7.2.8, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, Intel Arc Meteor Lake, Mesa 26.2.3), worktree on main 2cf23fc (rebased 2026-10-13: the entry moved to docs/dev/debt/libSceVideoOut.md; video_out_mouse and mouse_input pass; the latter now links SDL, which the shared mouse source needs for the capture calls; video_out_mouse also covers the capture decision (CaptureWanted: focus lost and regained, another window's focus, mouse closed and reopened, window closed, the pad's mouse-to-stick toggle) and applies the capture under SDL's dummy video driver, where SDL_GetRelativeMouseMode follows it and is released again): video_out_mouse passes; tools/check_conventions.py --base origin/main and git diff --check pass. Minecraft (PPSA17221) on a 125 % scaled Wayland desktop (SDL on XWayland): the host cursor is hidden while the game has the mouse and returns when another window takes focus, and the in-game cursor now travels as far as the hand does; before, it fell short and the host pointer left the window (Linux). Not tested on Windows.

The delta scale takes SDL's relative motion in logical window units, the units of SDL_GetWindowSize; under Windows DPI scaling SDL may report physical pixels, which no Windows run has checked yet; the debt entry says so.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: PPSA17221 (no Game bring-up discussion yet)
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@Mouad-Scriptz-0

Copy link
Copy Markdown
Author

Request for a Windows check: this PR is only tested on Linux (SDL on XWayland, 125 % desktop scaling), and Windows DPI scaling is where the delta scaling can go wrong. SDL reports the window size in logical units there, while relative motion may arrive in physical pixels depending on the DPI-awareness SDL ends up with, so the scale factor the PR computes could be off by the display scale.

What to test, with a title that opens the mouse (Minecraft PPSA17221 does, drawing its own cursor from sceMouseRead deltas):

  1. Display scaling at 100 %, then 125 % or 150 %: move the mouse across the full window width and check the in-game cursor travels the same distance (before this PR it fell short or overshot and the Windows pointer left the window).
  2. The Windows pointer should disappear while the title has the mouse and the window is focused, and come back when another window takes focus or the mouse-to-stick toggle (ToggleMouse, middle button by default) is on.
  3. A letterboxed window (resize to a non-16:9 shape) and fullscreen (F11).

The log prints one line at startup that shows what the PR computed, for example:

[mouse] video driver x11: guest 1920x1080 presented at 1844x1037, delta scale 1.0412 x 1.0415

On Windows the driver should read windows, and the presented size should match the window's client area in the units SDL uses for xrel/yrel. If the cursor travel is off by the display scale, that line plus the Windows scaling percentage is all I need to fix it. A [mouse] relative mouse mode unavailable line would mean the capture itself failed.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Blocking points:

  • The existing mouse_input test (core/libs/prx/libSceMouse/CMakeLists.txt:12) compiles libSceVideoOut/src/MouseInput.cpp into mouse_tests directly. UpdateCapture now calls SDL_SetRelativeMouseMode, SDL_GetRelativeMouseMode, SDL_GetRelativeMouseState and SDL_GetError, and that target doesn't link SDL, so it fails with undefined references on the PR head. Link SDL there (or split the capture code out of the file that test compiles) and run the existing libSceMouse tests as well as the new one.
  • The PR conflicts with main in VideoOutDriver.cpp and docs/dev/TechnicalDebt.md; please rebase.
  • Windows: the scale uses the SDL_GetWindowSize ratio (logical units), while relative-mode motion can be reported in physical pixels under DPI scaling. This is unverified and the body says it isn't tested on Windows. Please attach the [mouse] log line and the cursor-travel check at 125 %/150 % from a Windows run, or a test that pins the scale for a window whose size and drawable size differ, or make the scale independent of that assumption.
  • The new test covers only the delta arithmetic; UpdateCapture (mouse open + focus + toggle) has no test.

@Mouad-Scriptz-0
Mouad-Scriptz-0 force-pushed the feat/videoout-mouse-capture branch 3 times, most recently from 289d2ec to 13a3239 Compare October 13, 2026 13:46
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 13, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase and the SDL link fix; mouse_input and video_out_mouse pass on the merged head, and video_out_mouse fails to build without the change. Two points from the previous review are still open:

  • Windows: SetPresentation is fed SDL_GetWindowSize (logical units) while the relative-mode xrel/yrel units under DPI scaling are unverified, and nobody has run it on Windows. Either attach the [mouse] log line and the cursor-travel check at 125 %/150 % from a Windows run, or add a test that pins the scale when window size and drawable size differ.
  • MouseInput::UpdateCapture (mouse open + focus + MouseModeEnabled()) has no test; the new test only covers the delta arithmetic. A test for the capture decision (e.g. SDL_GetRelativeMouseMode() with the dummy/offscreen video driver) would cover it.

…e their deltas

The console's mouse reports relative motion, so the host cursor can never line
up with the cursor a title draws. While a title has a mouse open and the
window is focused the host cursor is hidden and captured, released on focus
loss or while the pad's mouse-to-stick toggle is on. Motion deltas are scaled
from the presented image's size in window units to the title's display
resolution with sub-pixel carry, so the title's cursor travels as far as the
host pointer would, on scaled desktops and letterboxed windows alike.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Mouad-Scriptz-0
Mouad-Scriptz-0 force-pushed the feat/videoout-mouse-capture branch from 13a3239 to 94087d0 Compare October 13, 2026 14:05
@Mouad-Scriptz-0

Copy link
Copy Markdown
Author

Both open points addressed in 94087d0:

  • MouseInput::CaptureWanted(allowed) now holds the decision (allowed && focused && mouse open) and UpdateCapture applies it. video_out_mouse checks the decision through synthetic window events (focus lost and regained, another window's focus loss ignored, the mouse closed and reopened, the window closed, the pad's mouse-to-stick toggle), then applies the capture under SDL's dummy video driver: SDL_GetRelativeMouseMode() follows UpdateCapture(true) and is released by UpdateCapture(false). On a driver that refuses relative mode the test says so and only checks the release.
  • Windows DPI: I can't run Windows here. The scale takes relative motion in logical window units (the units of SDL_GetWindowSize); the debt entry now states that and that Windows DPI scaling is unverified. @xXPBSXx, if your Windows batch can include this head: the [mouse] line at launch and whether a 125 % or 150 % desktop makes the title's cursor travel farther or shorter than the host cursor would settle it, and I'd switch the scale to the drawable size if it does.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked on the PR merged onto current main: build and the libSceMouse/libSceVideoOut ctest scope pass (mouse_input, video_out_mouse), and video_out_mouse fails to build without the change. The Windows DPI behaviour of the delta scale stays unverified and is recorded in the technical debt entry; merging.

@adriadam10
adriadam10 merged commit 5d7f1ff into boykopovar:main Oct 13, 2026
8 of 9 checks passed
@Tamalero

Copy link
Copy Markdown
Contributor

A follow-up from High on Life (PPSA15388), on Linux, after this PR: while the title reads the mouse, PadInput still turns mouse motion into the right stick, and mouse buttons still fire their pad bindings. So the camera gets the mouse twice, once through libSceMouse and once through the pad. In-game it feels like the mouse turned into a stick again.

Locally I added two guards in PadInput.cpp, keyed on MouseIsOpen_nid_postfix(), which this PR already provides:

// PadInput::Update: no right-stick deltas while the title reads the mouse
if ((deltaX != 0 || deltaY != 0) && !MouseIsOpen_nid_postfix()) { ... }

// PadInput::HandleEvent: mouse buttons bound to pad buttons are skipped while the title reads the mouse
if (mouse && binding.control == Pad::InputControl::Button && MouseIsOpen_nid_postfix()) {
    pressed[index] = false;
    continue;
}

With them, the mouse behaves like the PC version, and the pad/mouse tests pass. I can open this as a PR with a test if that's the intended behaviour, @Mouad-Scriptz-0.

Tested and written by Claude (AI assistant) on a contributor's machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants