Autogenerate images of parts of the viewer

[TimMonko]

References and Motivation

With 0.6.0 on the way, there have been many changes to the UI from 0.5.6, especially when considering the years of napari development. Much of these changes have been done by me, and in recognition of that, I would like to update the current static images used throughout the docs with dynamically generated ones, so that any future UI changes do not require thorough updates of the docs. This PR was originally conceived slightly before, and then discussed at 2025-03-13/14 docs meeting; the docs meeting concluded that this PR should be made quickly, merged, and then checked for any flakiness (since it requires a merge) or appearance issues in the auto-generation. Then, we can make specific updates to the script (or new scripts) in order to improve visualization for various places in the docs. These are presently designed as proof-of-principle and change things as needed. Conceivably, this can also replace images where we hide building up a napari viewer instance, only to take an nbscreenshot. I believe this script method is considerably faster, but does require a bit less intuitive of a setup compared to the screenshot method. Overall, this takes less than 3 seconds to generate over 20 images. I'm aware that many of these are not currently needed in the docs, but may provide inspiration and design principles, even for the autogeneration, moving forward. We'll clean up what exactly is autogenerated in follow-up PRs. In follow ups, we can also consider adding unique stylings to Qt elements on the fly, such as making a certain button of interest have a yellow border.

Useful current places:

1. viewer tutorial buttons, popups (e.g. we just added grid popup and the buttons below are already out of date),, anything with console, etc. quite a few static images in here
2. layers tutorial
3. quick start. console related images
4. various buttons and images in how to each layer

And in the future, because we can more quickly generate images, we can conceivably use more focused images throughout the docs.

This will require doing something with doc merges as force-orphan to prevent build up of repo size with changing binaries, though nicely, these images are so far quite small in size (comparatively)

Description

In one single file, our goal is to autogenerate various elements of the UI; these are either initially gotten with viewer.screenshot of the current viewer state, or in follow-ups as specific pixmaps of the widgets (or a defined group of widgets). These pixmaps are defined by dictionaries at the top of the file, making it easier to find and contribute to an image of interest. It also serves as models for 'set up' of different element types (popups, menus, other states)

These include:

1. single widget elements always visible in the UI
2. menus, and specific submenus
3. triggered popup widgets, like grid, roll, ndisplay
4. groups of widgets. these are my favorite, but will take the most thought to define because it allows us to dynamically 'crop' the viewer to areas of interest. Lovely!

The current implementation prioritizes speed, but may need to add some QTimer / sleep events for CI, not sure.

Autogenerated image examples

Viewer and folder structure

Single widgets

Menus

Popups

Combined groups of widgets

Original PR Description

This is really a draft of trying to autogenerate images that we can use throughout the docs. I think this would also speed up doc generation in certain areas with reusable doc images, so that nbscreenshot would not need to be as frequent. Everything is getting saved into images/_autogenerated and could be used elsewhere.

1. Tries to autogenerate grabbing and saving many parts of the viewer using the attributes of the viewer

Needs:

1. Popups are not working right, sometimes they trigger, often its the dims slider popup. I probably need to just directly call the button, but I'm having a tough time figuring out where the buttons live.
2. I think the timing of things has me really confused. I think QTimer is necessary to wait for things to popup, but I can't figure out why these popups aren't working
3. Also sometimes the console popups and saves as a screenshot, other times its just the thin dock area prior to the console popup.

Examples of what should appear in autogenerated if CI does it the same was as locally

[TimMonko]

Adding a few links from the community meeting today:
https://simonwillison.net/2022/Oct/14/automating-screenshots/
https://simonwillison.net/2022/Mar/10/shot-scraper/
napari/napari#7678
and look into qt_bot wait

[melissawm]

Hi @TimMonko - this is the PR I mentioned on zulip: #308

That one is more focused on videos, so might not be helpful but just in case here it is. Cheers!

[TimMonko]

@Czaki
Everything is organized in one file now, and the script is not flaky anymore.
QTimer actually seems to create more problems than fewer problems for popups, compared to menus. Perhaps I was implementing incorrectly --though I was using same logic as menus, it would be delayed asynchronously. Strangely, lowering Qtimer for capturing menus to 50ms resulting in incomplete generation of popups. The async part of this is so confusing.

1. Creates images of many widgets
2. Creates images of many menu items
3. Creates images of popups with prep triggers.

We can add / change more and modify callbacks in various ways, but as @willingc we can start with a merged script and go from there.

I hope it's not flaky on anyone else's machine, or CI.

[willingc]

Let's merge this as is. I would like to do a pass over the script and change some of the print statements to logging (it will benefit us when running in CI).

[TimMonko]

Sounds good, the prints were really just for debugging lol. I take them out as things start to work. Definitely anything with prints from me is a 'draft'. Can still merge though if you like

We should be able to see if CI autogenerates correctly

[willingc]

I would like us to merge and iterate on it.

[melissawm]

Can we add a comment in the script to indicate which images it is generating?

[TimMonko]

@melissawm I added much more commenting to try to clarify what is being saved and I hope the code organization is sufficient to be self explanatory. However, if what you mean is to specifically say which images for the docs this is generating, I'm not sure that's a good idea (will be hard to maintain and keep up to date). Please let me know if I haven't accomplished the goal.

I've also added various interactions with the canvas to change the display of various widgets -- we can always be very specific if we have things in mind to demonstrate. At this point, I'm very happy with this implementation, and want us to merge this to check it works on CI, and then we can look to specifically define images of interest that can be used in the docs.

I also added a preliminary, rough implementation of capturing regions of the viewer as defined by the widgets. In the long run, this will be more maintainable than defining the region with our own coordinates (since inevitably these coordinates could move around). Currently, it just crops to the bounds of the defined widgets, for example:

Viewer buttons + Console:

Layer list and layer controls

[melissawm]

Ok - Maybe I just need more context on what we're accomplishing. Reading this is seems like we are generating one image for the empty viewer, one for the cells3d sample and another for cells3d with the console. My question is - are we supposed to use this script in the future to include all other images? If not, why are we only using those 3?

I don't want to block this though, feel free to merge and iterate on it later!

[TimMonko]

@melissawm updated the PR description to fully encapsulate the goals of this PR and the description of every image that is taken.