Skip to content

perf(desktop-gpui): convert screenshot editor frames in one pass and skip stale ones - #2836

Merged
richiemcilroy merged 1 commit into
mainfrom
perf/gpui-screenshot-editor-copies
Oct 22, 2026
Merged

richiemcilroy merged 1 commit into
mainfrom
perf/gpui-screenshot-editor-copies

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

What

Renderer-input edits in the screenshot editor (padding, rounding, shadow, border, background, crop, aspect) re-render the full-resolution still, and every frame that came back was converted three times on the CPU before it reached the atlas:

  1. unpad_rows into a tight RGBA copy (kept for the mask overlays),
  2. rgba.clone() into a second full-size buffer,
  3. swap_red_blue over that clone for gpui's BGRA atlas.

Frames were also converted strictly in order, so a frame already superseded by a newer one in frame_rx was still copied, swizzled and uploaded.

This PR:

  • Converts each frame with the fused cap_gpui_kernels::frame::unpad_rgba_to_bgra, through the video editor's existing editor_window::frame_image. One read of the padded GPU buffer, one write, one allocation.
  • Drops the separate RGBA copy. The mask overlays resample the atlas image's own BGRA bytes (RenderImage::as_bytes(0)), and the per-overlay swap_red_blue goes away because the output is already in atlas order. mask_filter (both the pixelate copy and the two Triangle resizes in image 0.25.10) treats every channel identically and independently, so filtering BGRA and skipping the swap is byte-identical to filtering RGBA and swapping.
  • Drains frame_rx to the newest frame before converting.

Not changed: preview resolution, texture allocation, annotation/mask coordinate spaces, export, copy to clipboard and save (those still go through render_export_rgba and RGBA masked_region_image).

Byte-identical output

New unit tests:

  • the_displayed_frame_matches_unpadding_then_swapping_a_copy: frame_image equals the old unpad_rows + clone + swap_red_blue path for odd widths (1, 3, 7, 33, 1001) and tight, +4 and 256-aligned strides.
  • mask_overlays_from_bgra_match_filtering_rgba_then_swapping: blur (levels 4/9/16/33) and pixelate (2/7/20) overlays on a 37x23 frame with a 256-byte stride and varying alpha are byte-identical between the old path (RGBA filter, then rgba_to_render_image) and the new one.
  • only_the_newest_queued_frame_is_converted.

Measured

Scratch branch ci/screenshot-copies-ab (runs 39609307696, 39611730829), on Blacksmith runners.

Conversion alone (release cargo bench, 256-byte stride, 40 samples, A/B alternating in one binary)

Base is unpad_rows + clone + swap_red_blue. Head is unpad_rgba_to_bgra. Median per frame:

frame macOS (M5 Max VM) base / head Windows base / head Linux base / head
1080p 2.61 / 0.88 ms 3.18 / 1.60 ms 6.03 / 2.57 ms
4K 10.46 / 3.79 ms 13.06 / 6.86 ms 36.35 / 13.58 ms
5K 17.29 / 6.97 ms 21.13 / 10.85 ms 59.39 / 29.86 ms
5K at padding 40 (6760x3802) 29.41 / 12.31 ms 37.17 / 19.29 ms 114.06 / 61.27 ms

CPU bytes written per converted frame go from 3x to 1x the tight frame size (5K: 176.9 MB to 59.0 MB; 5K at padding 40: 308.4 MB to 102.8 MB). Allocations per frame go from two full frames to one, and the window keeps one full-size copy of the frame resident instead of two (59 MB less at 5K).

Padding slider drag, end to end (real run_still_renderer + frame pump, 120 ticks at 60 Hz, padding 0 to 40 and back, median of 3 alternating rounds)

Dev-profile test build, same harness as benchmark_annotation_slider_drag. The Windows runner has no GPU adapter, so this part is macOS (Metal) and Linux (lavapipe) only.

still runner CPU / tick base / head convert median base / head MB written / edit base / head median edit-to-frame base / head
1920x1080 macOS 13.97 / 9.75 ms 12.54 / 3.10 ms 19.0 / 7.0 52.6 / 41.6 ms
3840x2160 macOS 22.65 / 13.37 ms 46.13 / 12.52 ms 33.5 / 11.6 135.1 / 92.2 ms
5120x2880 macOS 25.27 / 15.01 ms 87.21 / 22.94 ms 36.5 / 12.1 233.0 / 163.3 ms
1920x1080 Linux 80.76 / 78.06 ms 8.72 / 3.00 ms 26.2 / 9.7 38.6 / 28.9 ms
3840x2160 Linux 90.22 / 85.89 ms 49.02 / 24.75 ms 27.4 / 9.5 153.0 / 123.9 ms
5120x2880 Linux 94.46 / 92.42 ms 77.11 / 37.86 ms 33.1 / 11.1 244.6 / 193.1 ms

On real hardware (Apple M4 Max, 128 GB, on-device comment) the conversion speedup is the same 3x (5K: 11.93 to 3.93 ms). The drag benefits more than on the VM because the real GPU renders faster and conversion becomes base's bottleneck: at 5K base converts only 78 of 120 edits with a 113.6 ms median edit-to-frame, while head converts 119 at 18.6 ms. At 4K edit-to-frame goes from 21.0 to 11.5 ms.

On Linux, process CPU is dominated by lavapipe rasterizing on the CPU, so the conversion saving is a small share of it there.

The drain did not drop any frames in these runs: the renderer was always slower than the conversion, so a second frame never queued. The drain is a guard for when conversion or main-thread handoff falls behind (busy machine, slow atlas upload), and it costs one uncontended channel lock per frame.

The main-thread atlas upload (replace_region of the full frame) is unchanged and not covered by these numbers.

Interaction with #2703

#2703 (screenshot editor startup) touches the same files but none of the same lines:

  • It calls view.frame_arrived(image, rgba, image_size, ...) for its instant preview. This PR removes the rgba argument, so whichever lands second gets a compile error rather than a silent channel mismatch. The fix on that side is to drop the argument (and the source.rgba.clone() that feeds it): its preview image comes from rgba_to_render_image, which is BGRA, so masks read correct pixels from it.
  • Its revision >= view.revision check in the frame loop composes with the drain here: the drain makes the frame it checks the newest one, so fewer frames are converted only to be discarded.

RetriggerView in GreptileConfidence Score: 5/5

This PR appears safe to merge.

Summary

The screenshot editor now converts each frame once and shares its BGRA bytes with mask overlays.

  • Screenshot previews and mask overlays share one converted frame.
  • The screenshot editor skips queued frames that newer edits have replaced.

No actionable issues found. Tests and the native UI were not run.

Reviews (1) · Last reviewed commit: "perf(desktop-gpui): convert screenshot e..." · Reviewed by Greptile

@vercel

vercel Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 21, 2026 1:33pm UTC

@blacksmith-sh

This comment has been minimized.

@richiemcilroy

Copy link
Copy Markdown
Member Author

On-device macOS check (real hardware)

Machine: Apple M4 Max (Mac16,6, 40-core GPU, Metal 4), hw.memsize 137438953472 (128 GB), macOS 27.0, on AC power. Harness from ci/screenshot-copies-ab (6b88c75), synthetic still, no screen capture.

The three byte-identity tests (the_displayed_frame_matches..., mask_overlays_from_bgra..., only_the_newest_queued...) pass here.

Conversion alone (release cargo bench, screenshot_frame_convert, median of 3 runs, each A/B alternating in one binary)

frame base head speedup
1080p 1.54 ms 0.52 ms 2.96x
4K 6.50 ms 2.17 ms 3.00x
5K 11.93 ms 3.93 ms 3.04x
5K at padding 40 (6760x3802) 22.03 ms 7.08 ms 3.11x

Padding slider drag, end to end on Metal (dev-profile test build, benchmark_padding_slider_drag, 120 ticks at 60 Hz, median of 3 alternating rounds)

still converted of 120 base / head CPU / tick base / head convert median base / head MB written / edit base / head median edit-to-frame base / head
1920x1080 120 / 120 5.45 / 4.08 ms 3.27 / 0.92 ms 33.6 / 11.2 6.4 / 5.4 ms
3840x2160 120 / 120 18.14 / 8.83 ms 12.12 / 2.93 ms 134.5 / 44.8 21.0 / 11.5 ms
5120x2880 78 / 119 24.07 / 15.56 ms 24.57 / 5.20 ms 151.1 / 78.9 113.6 / 18.6 ms

Reading it:

  • The conversion speedup is the same as on the CI VMs (about 3x), with lower absolute times than the M5 Max VM.
  • The real GPU renders much faster than the VM, so conversion becomes the bottleneck on base. At 5K, base converts only 72 to 94 of the 120 edits per round and the median edit-to-frame latency is 114 ms. Head keeps up (119 of 120) at 18.6 ms. At 4K, latency drops from 21.0 to 11.5 ms and process CPU per tick halves.
  • The drain still dropped nothing on head (rendered equals converted in every round), so the latency win comes from the cheaper conversion, not from skipping frames.

@richiemcilroy
richiemcilroy marked this pull request as ready for review October 22, 2026 06:22
@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@blacksmith-sh

blacksmith-sh Bot commented Oct 22, 2026

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
crash/recorder-kill/inproc/static-screen/static-screen View Logs
crash/recorder-kill/oop/static-screen/static-screen View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@richiemcilroy
richiemcilroy merged commit f33be45 into main Oct 22, 2026
63 of 67 checks passed

This branch was successfully deployed

2 active deployments
Preview – cap-web — 9fd2d732 Deployed Oct 21, 2026 by vercel[bot]
Preview – cap-status — 9fd2d732 Deployed Oct 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant