Skip to content

perf(recording): speed up screenshot corner rounding and rgb packing - #2808

Merged
richiemcilroy merged 4 commits into
CapSoftware:mainfrom
kyzer1023:perf/screenshot-corner-rounding
Oct 24, 2026
Merged

richiemcilroy merged 4 commits into
CapSoftware:mainfrom
kyzer1023:perf/screenshot-corner-rounding

Conversation

@kyzer1023

@kyzer1023 kyzer1023 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Screenshot window corners are masked only in the four corner squares, and BGRA/RGBA to RGB packing no longer picks the channel order once per pixel. On x86_64 the packer uses SSSE3 pshufb when the CPU supports it, with a scalar tail and a scalar fallback. aarch64 keeps a tight scalar loop. Output matches the previous pixel loops byte for byte.

Why

apply_window_rounded_corners walked every pixel and ran the rounded-corner alpha test even though only the corner squares of radius r can be non-opaque. rgb_from_rgba matched RGBA vs BGRA inside the per-pixel loop.

Both helpers were compiled only for macOS and Windows, so their tests never ran on Linux CI.

What changed

  • Interior rows, and the middle of the top and bottom rows, are copied as RGB plus alpha 255. rounded_corner_alpha runs only inside the corner squares. If the radius is large enough that those squares overlap, each pixel is still evaluated once, with the same left-then-right branch order as before. Radius <= 0 still uses the existing opaque ConvertBuffer conversion.
  • rgb_from_rgba chooses the channel order once per image and, on x86_64, checks SSSE3 once per image. Groups of 4 pixels are shuffled with pshufb. The leftover 0 to 3 pixels, and every pixel when SSSE3 is absent, use the scalar loop. aarch64 compiles only that scalar loop.
  • The pure pixel helpers compile on macOS, Windows, and in tests. Platform capture stays behind its OS cfg. Linux non-test builds do not compile the packers, so they are not flagged as dead code.

Benchmarks

Apple Silicon / aarch64 was not measured.

Hardware: Intel(R) Xeon(R) Processor, family 6 model 173, 4 cores, 2700 MHz (/proc/cpuinfo), Linux 6.12.94+ x86_64. One core via taskset -c 1. Warmup 2, then the median of 9 runs. Corner radius 16 px. Every case checked byte-identical output before timing, and all of them matched. The timed call allocates the output buffer, same as the production functions.

The harness is one Rust file that contains the previous per-pixel loops and the new loops, so both sides share the compiler flags. It is not linked into cap-recording (that crate needs FFmpeg). The corner "before" loop is the full-frame alpha walk over raw RGB bytes.

RUSTC_VERSION="$(rustup run 1.88.0 rustc --version)" OPT_LABEL="opt-level=s,lto=thin,codegen-units=1,debuginfo=line-tables-only" \
  rustup run 1.88.0 rustc --edition 2024 -C opt-level=s -C lto=thin -C codegen-units=1 -C debuginfo=line-tables-only -C panic=unwind screenshot_bench.rs -o screenshot_bench_188
taskset -c 1 ./screenshot_bench_188

RUSTC_VERSION="$(rustup run stable rustc --version)" OPT_LABEL="opt-level=3,lto=thin,codegen-units=1" \
  rustup run stable rustc --edition 2024 -C opt-level=3 -C lto=thin -C codegen-units=1 -C panic=unwind screenshot_bench.rs -o screenshot_bench_stable
taskset -c 1 ./screenshot_bench_stable

rustc 1.88.0 (6b00bc388 2025-06-23). Flags match the workspace release profile: opt-level = "s", thin LTO, codegen-units = 1, debuginfo = line-tables-only, panic = unwind. Times are milliseconds.

Frame Corners before Corners after Speedup BGRA to RGB before BGRA to RGB after Speedup RGBA to RGB before RGBA to RGB after Speedup
1920x1080 5.058 1.644 3.08x 1.677 0.707 2.37x 1.674 0.711 2.36x
3840x2160 20.141 6.452 3.12x 7.207 2.781 2.59x 7.080 2.981 2.38x
5120x2880 46.642 21.501 2.17x 20.098 12.133 1.66x 20.083 12.201 1.65x

stable rustc 1.99.1 (58cf8f9c3 2026-10-13). opt-level = 3, thin LTO, codegen-units = 1, panic = unwind.

Frame Corners before Corners after Speedup BGRA to RGB before BGRA to RGB after Speedup RGBA to RGB before RGBA to RGB after Speedup
1920x1080 2.743 1.682 1.63x 1.344 0.707 1.90x 1.359 0.710 1.91x
3840x2160 10.856 6.460 1.68x 5.868 2.776 2.11x 5.551 3.030 1.83x
5120x2880 29.213 20.797 1.40x 17.947 12.135 1.48x 17.481 12.145 1.44x

Tests

FFmpeg 7 came from the repo native-deps bundle (target/native-deps, libavcodec.so.61), not Ubuntu libav.

FFMPEG_DIR=$PWD/target/native-deps \
LD_LIBRARY_PATH=$PWD/target/native-deps/lib \
PKG_CONFIG_PATH=$PWD/target/native-deps/lib/pkgconfig \
cargo test -p cap-recording --lib screenshot::screenshot_pixel_tests -- --test-threads=8

test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 832 filtered out; finished in 0.65s

The tests compare against the original per-pixel implementations. They cover odd widths, radius 0, radius larger than half the shorter side, tiny images, non-finite radii, BGRA and RGBA, the SSSE3 path including widths that leave a scalar tail, and the scalar fallback called directly.

FFMPEG_DIR=$PWD/target/native-deps \
LD_LIBRARY_PATH=$PWD/target/native-deps/lib \
PKG_CONFIG_PATH=$PWD/target/native-deps/lib/pkgconfig \
cargo check -p cap-recording --lib

Finished dev profile [unoptimized + debuginfo] target(s) with exit code 0.

cargo clippy -p cap-recording --no-deps --lib --tests -- -D warnings

Finished with exit code 0.

Merged main

Merged main at 2012f25c into the branch at 4e24e082 with no conflicts. Nothing on main touched screenshot.rs between the old base and that commit. After the merge, cargo fmt --all -- --check and cargo clippy -p cap-recording --no-deps --lib --tests -- -D warnings are clean. The same 7 screenshot pixel tests pass, with 837 filtered out because main added tests.

The same bench was rerun on a different VM (Xeon family 6 model 207, 2400 MHz). Every case was still byte-identical. With rustc 1.88.0 at the release profile, corners got 3.94x / 3.84x / 2.65x faster, BGRA to RGB 2.31x / 2.30x / 1.54x, and RGBA to RGB 2.02x / 2.32x / 1.50x at 1080p / 4K / 5K. The tables above are from the original 2700 MHz run.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues remain.

Summary

This PR speeds up screenshot corner rounding and RGB packing.

  • RGBA and BGRA screenshot frames turn into RGB with less per-pixel work.
  • Window screenshots check rounded alpha only near their corners.

No new actionable issues found. This review checked the code without running tests.

Reviews (3) · Last reviewed commit: "Merge main" · Reviewed by Greptile

@vercel

vercel Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

@kyzer1023 is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

@richiemcilroy

Copy link
Copy Markdown
Member

@greptileai review

Comment thread crates/recording/src/screenshot.rs Outdated
Comment on lines +2040 to +2043
assert!(
std::is_x86_feature_detected!("ssse3"),
"ssse3 is required to execute the shuffled path"
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Tests reject supported CPUs

If an x86_64 machine lacks SSSE3, or a virtual machine hides it, rgb_ssse3_matches_original_including_scalar_tail fails this assertion. Production supports those machines through its scalar fallback, so cargo test fails on an otherwise working configuration. Return early when SSSE3 is unavailable instead of asserting.

Suggested change
assert!(
std::is_x86_feature_detected!("ssse3"),
"ssse3 is required to execute the shuffled path"
);
if !std::is_x86_feature_detected!("ssse3") {
return;
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/recording/src/screenshot.rs
Line: 2040-2043

Comment:
**Tests reject supported CPUs**

If an x86_64 machine lacks SSSE3, or a virtual machine hides it, `rgb_ssse3_matches_original_including_scalar_tail` fails this assertion. Production supports those machines through its scalar fallback, so `cargo test` fails on an otherwise working configuration. Return early when SSSE3 is unavailable instead of asserting.

```suggestion
        if !std::is_x86_feature_detected!("ssse3") {
            return;
        }
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@richiemcilroy

Copy link
Copy Markdown
Member

Hey @kyzer1023, thanks for this, the corner-only alpha pass and the SSSE3 packing are nice wins and the scalar-tail coverage is thorough.

Greptile's one finding this round (crates/recording/src/screenshot.rs:2040-2043) is partly right. In practice virtually every x86_64 CPU from the last 15+ years has SSSE3, and the assert does guarantee the shuffled path actually runs in CI instead of silently passing. But it can fail on an emulated CPU without SSSE3 (for example QEMU's default qemu64 model), even though production handles that via the scalar fallback. The simplest fix is to take Greptile's suggestion and return early when SSSE3 isn't detected, ideally with an eprintln! so a skip is visible in the test output, which matches what crates/rendering/tests/cursor_texture_edges.rs does when no GPU is available.

Once pushed, could you comment @greptileai review so it re-scores the latest commit? Thanks!

Copy link
Copy Markdown
Contributor Author

@greptileai review

Co-authored-by: Kyzer  <kyzer1023@users.noreply.github.com>

Copy link
Copy Markdown
Contributor Author

@greptileai review

@kyzer1023

Copy link
Copy Markdown
Contributor Author

@richiemcilroy merged latest main into this one. CI is green apart from the known macOS playback flakes (backward_jumps / vfr_holds hw decode) and Engine verify, and Greptile is at 5/5 on the merge commit. Ready for another look whenever you have time, thanks!

@richiemcilroy
richiemcilroy merged commit c5b1871 into CapSoftware:main Oct 24, 2026
46 of 52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants