Skip to content

Conversation

@HKTITAN
Copy link

@HKTITAN HKTITAN commented Dec 16, 2025

Summary

This PR adds an auto-zoom feature that follows cursor movements during recording and automatically generates zoom regions based on click events.

Features Added:

1. Native Cross-Platform Mouse Tracking

  • Uses \uiohook-napi\ for native mouse/keyboard hooks (Windows, macOS, Linux)
  • Captures mouse movements, clicks, and scroll events during recording at 120fps
  • Normalizes coordinates to 0-1 range for resolution independence

2. Auto-Zoom Generation

  • Automatically creates zoom regions centered on click locations
  • Configurable zoom depth, duration, and minimum interval between zooms
  • Generates focus keyframes from cursor movements during zoom periods

3. Smooth Cursor Following

  • Uses Catmull-Rom spline interpolation for smooth cursor following
  • Ken Perlin's \smootherStep\ easing function for natural motion
  • Keyframe sampling at 50ms intervals with 1% movement threshold

4. Cursor Overlay (WIP)

  • Added customizable cursor styles: Windows, Mac (black/white variants), Circle, Ring, Dot
  • Settings panel with size, opacity, style, and click animation options
  • Video bounds-aware positioning that respects padding

New Files:

  • \electron/mouse-tracker.ts\ - Native mouse event tracking service
  • \src/utils/cursorUtils.ts\ - Cursor event to zoom region conversion
  • \src/components/video-editor/CursorOverlay.tsx\ - Cursor visualization component

Known Issues:

The mouse overlay during video editing is not working as intended. The cursor appears but the positioning/visibility seems off. Would appreciate if you could look into this - might be related to how video bounds are calculated or the coordinate transformation.

How to Test:

  1. Enable 'Auto-Zoom' toggle in the recording launch window
  2. Record a screen capture with some mouse clicks
  3. Open the video editor - zoom regions should be auto-generated at click points
  4. The cursor should follow smoothly during zoom periods

Let me know if you have any questions!

- Added native cross-platform mouse tracking using uiohook-napi (Windows, macOS, Linux)
- New electron/mouse-tracker.ts: Captures mouse movements, clicks, and scroll events during recording
- New src/utils/cursorUtils.ts: Converts cursor events into zoom regions with focus keyframes
- Auto-generates zoom regions centered on click locations
- Smooth cursor following during zoom using Catmull-Rom spline interpolation
- Added auto-zoom toggle in LaunchWindow
- Added cursor overlay with customizable styles (Windows/Mac cursors, circle, ring, dot)
- Cursor settings panel with size, opacity, style, and click animation options
- Video bounds-aware cursor positioning (respects padding)

Note: Mouse overlay during video editing is not working as intended and needs debugging
@siddharthvaddem
Copy link
Owner

Dammit! You beat me to it haha. I was literally working on this. Thanks a lot for working on this. I'm going to take some time to review it since it is a pretty advanced feature and want to ensure it works as intended. But great job on this!

Some things you might want to look into in the meanwhile...

It would be nice to add like a demo of the feature walkthrough as requested in the template. Makes it easier to have a quick glance.

The other thing I wanted to mention was during the first version of this app (when I was experimenting), there were issues across diff OS for uiohook-napi which might need some sort of recompilation. Basically dev != build version (lot of issues)
wilix-team/iohook#406 (comment)

Did you try building it and checking if the app works?

@HKTITAN
Copy link
Author

HKTITAN commented Dec 17, 2025

haha no way! perfect timing.

i’ve run this locally and tested a build via 'npm run dev' on windows end-to-end (recording → editor → auto-zoom playback) and it’s working as expected.

i’ve also recorded a quick demo walkthrough (recorded via canvid, which was actually the inspiration for this update) to make the review easier.

i haven’t yet validated a full build on linux or macOS though, so would love to know if it’s working there.

Entire.screen.12-17-2025.10-16-30.mp4

as you can see here, the overlay cursor is not replacing the recording's cursor as done in canvid. trying to figure it out, but could you please look into it once?

@siddharthvaddem
Copy link
Owner

siddharthvaddem commented Dec 18, 2025

I tested it out. Some running thoughts:

The cursor following works accurately and the camera view is solid. BUT...here is where it gets confusing (for a user). After the automatic following, if I'm unhappy with a zoom portion, the zoom has a depth as well as position associated with it. I randomly move it around. It plays no role. Zoom level works, but the overlay in preview is pointless and is bad UX.

Here is my suggestion:

  • Let each zoom item on the timeline have two properties (zoom depth and position).
  • If the position for that item is "follow cursor", we will not show the zoom overlay on the screen when a zoom item is clicked. - If the position for that item is "fixed" then the zoom overlay shows up and in the preview playback and it should NOT follow cursor and stick to default fixed positioning.
  • So something like a dropdown near zoom settings to choose between 1 of them.
  • This should be dynamically handled. So users have full flexibility and customization between changing each item to a static vs follow cursor option as per their wish.
  • Let the screen recording add automatic zooms by default & the default option in the dropdown be follow cursor.

There some other bugs like, if I customize the length of the zoom item let's say from default 2 seconds to let's say 4 seconds. the extra 2 seconds that are new, in that period ( it does not follow the mouse. just stuck there). There could be more, but I would like you to do some more testing.

As for the cursor stuff, I would suggest you to remove all of it. We might be moving to FFMPEG and there might be a way to capture separate layers (one just the screen with no cursor and another layer of one with just the cursor and nothing else), this way we don't have to overlay the custom cursors (more issues with overlapping, size constraints, etc). Not sure if this will work but that is kind of the base idea. This is also a separate feature and I would like to keep this out of scope for now.

I'll do a code review, once you address all of the above and do a thorough testing for all scenarios. Please make sure to list out all the cases you have tested (as it would be too much effort to record each case and share).

But good stuff!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants