Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) #32838

mikeclayton · 2024-05-12T19:35:21Z

Summary of the Pull Request

This is a re-visit of cancelled PR #29292 / issue #27511 to add customisable styles (2d / 3d effect borders, colours, padding, margins etc) around the Mouse Jump preview screen and the individual screenshots on it.

In the previous PR I hit a dead-end with the logic to upgrade existing Mouse Jump config files and with exposing all the style settings in the Settings UI, so in this PR I've just synched the latest rendering code from my FancyMouse utility but made no attempt to add any new settings to Mouse Jump right now.

As a result, there's no new Settings UI properties or application functionality in this PR - I've left the default Mouse Jump preview style as-is (i.e. flat preview border and no margins or bezels around screens), but this PR will make it easier to expose the style settings in follow-up work if all the rendering code is already there.

(See #27511 for a description of the eventual "customisable preview styles" feature proposal this PR is a step toward)

PR Checklist

Detailed Description of the Pull Request / Additional comments

Mostly a maintenance release to sync "customisable style rendering" code from FancyMouse to Mouse Jump::

moved all code shared with FancyMouse into a "Common" folder to make it easier to keep in sync
synced customisable style-based preview rendering from FancyMouse to Mouse Jump
fixed a bug where number pad keys didn't jump the mouse to the numbered screen
unit tests can now test real rendering of the preview image by "capturing" areas of a static test bitmap instead of the live desktop screen and comparing it to an embedded resource containing the expected image, which is a last line of defence against regression bugs

Validation Steps Performed

The latest commit seems to work fine locally, but I've not performed a full regression test yet...

Workflow tests
- Automated tests passing locally
- Minimal actions workflow (spelling check) passing for PR
- Full actions workflow (msbuild) passing for PR
UI tests
- Happy path
  - preview image appears when activated
  - clicking the preview image moves the mouse cursor to the correct location
  - right-click dismisses the preview image without moving the mouse
  - pressing escape dismisses the preview image without moving the mouse
- Works on multiple monitors with different dpi scaling settings (e.g. 100% vs 150%)
  - [Mouse Jump] - new feature - #23216 #23566 (comment)
  - [Mouse Jump] - new feature - #23216 #23566 (comment)
- Handling negative coordinates on non-primary monitors if higher or "lefter" than primary monitor
  - [Mouse Jump] - new feature - #23216 #23566 (comment)
- Mouse crosshair moves when Mouse Jump moves the cursor (mouse clicks and keyboard shortcuts)
  - Mouse pointer Crosshair & Mouse Jump Issue #24523
  - [MouseJump]Also simulate mouse input event #24527
- Activating when the preview window is already visible moves the form to the new mouse position
- Number and key shortcuts (1-9, Home / End, Left / Right Arrow) jump to the appropriate monitor
- Number-pad shortcuts (1-9 jump to the appropriate monitor
Lifecycle tests
- Starting PowerToys Runner launches MouseJumpUI.exe when enabled, and not when disabled
- Enabling / disabling Mouse Jump in settings starts / stops MouseJumpUI.exe
- Exiting PowerToys Runner stops MouseJumpUI.exe
- Killing runner exe via Task Manager stops MouseJumpUI.exe
- Stopping Visual Studio local debug run stops MouseJumpUI.exe
  - note - runner needs to be in non-admin mode otherwise Visual Studio debugger disconnects at launch
- Hotkey and size settings are automatically reloaded when config file is modified from Settings UI
- ~~Hotkey and size settings are automatically reloaded when config file is modified manually (e.g. in notepad) while runner and MouseJumpUI.exe are running~~
Internal Test Suite
- Enable Mouse Jump. Then:
- Press the activation shortcut and verify the screens preview appears.
- Change activation shortcut and verify that new shortcut triggers Mouse Jump.
- Click around the screen preview and ensure that mouse cursor jumped to clicked location.
- Reorder screens in Display settings and confirm that Mouse Jump reflects the change and still works correctly.
- Change scaling of screens and confirm that Mouse Jump still works correctly.
- Unplug additional monitors and confirm that Mouse Jump still works correctly.
- Disable Mouse Jump and verify that the module is not activated when you press the activation shortcut.

…icrosoft#25482)

…25482)

…al style unchanged) (microsoft#25482)

jaimecbernardo

LGTM!
Thank you for the contribution!
Just had a quick question about a code change.
@mikeclayton , so the idea is to merge this one so another PR can come later with some Settings changes?

jaimecbernardo · 2024-06-12T09:57:08Z

...useUtils/MouseJumpUI/Common/NativeMethods/User32/UI/Input/KeyboardAndMouse/User32.LPINPUT.cs

@@ -52,7 +52,7 @@ private static IntPtr ToPtr(INPUT[] values)
            var size = INPUT.Size;
            foreach (var value in values)
            {
-                Marshal.StructureToPtr(value, ptr, true);
+                Marshal.StructureToPtr(value, ptr, false);


Why aren't we deleting the old one now? Not sure why this change was made.

The memory is only allocated a few lines earlier so I figured there was actually nothing to delete and it should have been false all along.

Happy to revert it if preferred though?

(Nice spot as well, in what was a pretty sprawling PR :-))

That's what you're doing in other spots you use this as well 🤔 So I assume it must be correct, we didn't have reports of leaks. Although, I wonder. It it was incorrect, why didn't the free cause a crash? It might be correct to clean all those uses instead, right?

I can take a look at the other instances - I must have just spotted this one while making other changes.

Looking at the documentation it says false is the correct value for this scenario:

https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.marshal.structuretoptr?view=net-8.0

On the first call to the StructureToPtr method after a memory block has been allocated, fDeleteOld must be false, because there are no contents to clear.

I'll take a look at the other uses of StructureToPtr and fix any others that need it, if you want to hold off on merging this PR for now...

That's the only place where you used true, I've checked. Good to merge, then. Thank you!

awesome. thanks for the thorough review :-)

reading the small print in the documentation i think false might only cause a problem if the structures being marshalled have fields that contain object references, because the runtime keeps a reference count on thosee objects until the fDeleteOld flag is set to true or DestroyStructure is called

If you use the StructureToPtr method to copy a different instance to the memory block at a later time, specify true for fDeleteOld to remove reference counts for reference types in the previous instance. Otherwise, the managed reference types and unmanaged copies are effectively leaked.

We (I) possibly got lucky because all the fields on the marshalled structure are (eventually) value types. That's just speculation though and it's definitely better to comply with the documentation :-)

mikeclayton · 2024-06-12T11:50:56Z

so the idea is to merge this one so another PR can come later with some Settings changes?

@jaimecbernardo - That's correct. I got a bit overwhelmed in the previous (cancelled) PR with the amount of changes needed to pull in the new style rendering code and expose the config options in the Settings UI so I figured it would be more manageable to break it into two steps.

mikeclayton added 6 commits May 6, 2024 11:21

[Mouse Jump] - move code shared with FancyMouse into "Common" folder (m…

d22e07e

…icrosoft#25482)

[Mouse Jump] - updates to NativeMethods (microsoft#25482)

61a59ce

[Mouse Jump] - added new drawing / layout / style classes (microsoft#…

9854c33

…25482)

[Mouse Jump] - new style-based preview rendering (actual preview visu…

a451b15

…al style unchanged) (microsoft#25482)

[Mouse Jump] - add words to spell checker (microsoft#25482)

6a5a3a2

[Mouse Jump] - small tweak to error handling (microsoft#25482)

9747899

mikeclayton changed the title ~~Dev/mikeclayton/mousejump styles~~ [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) May 12, 2024

[Mouse Jump] - fixed failing test (microsoft#25482)

c032b0b

mikeclayton changed the title ~~[Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2)~~ Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) Jun 8, 2024

Merge branch 'main' into pr32838

0c1beee

jaimecbernardo approved these changes Jun 12, 2024

View reviewed changes

jaimecbernardo merged commit 651f2e4 into microsoft:main Jun 12, 2024
10 checks passed

mikeclayton mentioned this pull request Jun 23, 2024

WIP - [Mouse Jump] - Customisable appearance - borders, margins, colours, etc #33486

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) #32838

Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) #32838

mikeclayton commented May 12, 2024 •

edited

Loading

jaimecbernardo left a comment

jaimecbernardo Jun 12, 2024

mikeclayton Jun 12, 2024 •

edited

Loading

jaimecbernardo Jun 12, 2024

mikeclayton Jun 12, 2024

jaimecbernardo Jun 12, 2024

mikeclayton Jun 12, 2024

mikeclayton commented Jun 12, 2024 •

edited

Loading

Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) #32838

Ready for review - [Mouse Jump] Customisable appearance - borders, margins, colours, etc (🎬 Take 2) #32838

Conversation

mikeclayton commented May 12, 2024 • edited Loading

Summary of the Pull Request

PR Checklist

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

jaimecbernardo left a comment

Choose a reason for hiding this comment

jaimecbernardo Jun 12, 2024

Choose a reason for hiding this comment

mikeclayton Jun 12, 2024 • edited Loading

Choose a reason for hiding this comment

jaimecbernardo Jun 12, 2024

Choose a reason for hiding this comment

mikeclayton Jun 12, 2024

Choose a reason for hiding this comment

jaimecbernardo Jun 12, 2024

Choose a reason for hiding this comment

mikeclayton Jun 12, 2024

Choose a reason for hiding this comment

mikeclayton commented Jun 12, 2024 • edited Loading

mikeclayton commented May 12, 2024 •

edited

Loading

mikeclayton Jun 12, 2024 •

edited

Loading

mikeclayton commented Jun 12, 2024 •

edited

Loading