Skip to content

Conversation

@gselzer
Copy link
Collaborator

@gselzer gselzer commented Jul 17, 2025

This PR adds a hierarchical event system for responding to user interaction. This is the main barrier to usage in e.g. NDV. It is blocked by #27, it relies upon the camera projection matrix to map each event to a ray traveling through the world.

Event Objects

This PR introduces an Event class hierarchy designed to abstract the various event objects of each widget toolkit that could display a scenex scene. The Event dataclass itself is minimal, with only a type field indicating the category of event. Subdataclasses MouseEvent, WheelEvent, etc. provide the additional fields necessary for containing the information needed to process a mouse press, wheel scroll, etc. respectively.

Event generation:

Each CanvasAdaptor is responsible for passing its native widget to the relevant AppWrap object (QtAppWrap, GlfwAppWrap, etc.). The exact AppWrap object involved depends on the active widget toolkit and is selected similarly to how NDV selects its toolkit. The AppWrap object creates an EventFilter for the native widget, responsible for converting the qt/glfw/... events into scenex events.

Event propagation:

When an event occurs over a particular position on the canvas, it is converted into a ray emanating from the camera into the world. Each object in the scene is checked for an intersection with that ray. All objects intersected by the ray are sorted by the depth at which they intersect.

For objects [o1, o2, ..., on] in this list, events are passed to:

  1. o1
  2. o1's parent, grandparent, ... recursively
  3. o2
  4. o2's parent, grandparent, ... recursively
    ...
  5. on
  6. on's parent, grandparent, ... recursively
  7. The view's camera.

Event filtering

The user is able to install an event filter on any node in the scene (including cameras) using

class Node(EventedBase):
    def set_event_filter(self, callable: Callable[[Event, Node], bool] | None) -> Callable[[Event, Node], bool] | None:

The Callable provided will receive the scenex Event as well as the Node oi that was intersected by the world ray. Within this function, they can edit the scene as needed and then return True if they wish to prevent further objects from processing this event.

Note that with the camera receiving the event as well, this seems the best way to enable controlling the camera with pan/zoom, orbit, etc.

TODOs

The main thing is to ensure this pattern is sufficient for our needs with (but not limited to!) NDV. We can certainly add:

  • More event subdataclasses (Key events, Mouse enter/leave)
  • More widget backends (wx, jupyter are the main ones)
  • More camera filters/controllers

Of course, beyond that, testing/removing FIXMEs/etc is necessary.

gselzer added 25 commits July 10, 2025 16:57
Defines how 2D view NDC are mapped to vectors in 3D space
In the pursuit of a single source of truth, let's resort to the
projection matrix (when/if possible?) for this.
For some reason, if I don't do this, the Matrix3D constructor RESURRECTS
some previous matrix I used in a previous test
Thanks to @tlambert03 for the suggestion. Could probably be cleaned up
further, but at least this is a step in the right direction!
Eventually, we'll want to compute grids ourselves on the model side, I
think, but for now this works
...man, this feels so good
Notably, this limits the type of pygfx-specific interaction available.
But we need a scenex version of this anyways - planning to implement the
beginnings of this with events
@gselzer gselzer self-assigned this Jul 17, 2025
@gselzer gselzer added the enhancement New feature or request label Jul 17, 2025
Some airport fixes coming around from me using my laptop screen for once
:)
Let's just keep wx out of the CI linting...
@gselzer gselzer changed the title WIP: feat: events feat: events Sep 17, 2025
This is a clean solution for implementing "on-leave" behavior. In NDV's
case, it's needed to clear the hover info when the cursor leaves the
image.
scenex image/volume nodes have pixel centers at integer coordinates. Our
math needs to reflect that
Tried my best to explain via comments why I think it's the right choice
Also needed for ndv :)
This was referenced Oct 3, 2025
This was referenced Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant