Skip to content

perf(agc): use a packed device-local LUT for half-float scanout - #5581

Draft
KROU4 wants to merge 1 commit into
boykopovar:mainfrom
KROU4:perf/gpu-color-transfer
Draft

KROU4 wants to merge 1 commit into
boykopovar:mainfrom
KROU4:perf/gpu-color-transfer

Conversation

@KROU4

@KROU4 KROU4 commented Oct 14, 2026 •

Copy link
Copy Markdown

Draft status and title evidence

This remains a draft. The available real-title run does not demonstrate a
half-float title gain: PS5 DOOM v1.1.1 uses an RGBA8_SRGB tiled scanout. Three
clean 240-present traces completed at 1920x1080 on the RTX 3060 Ti, but that
workload exercises the unchanged 8-bit path. A title/scene using half-float
scanout is still required to satisfy the repository's performance-evidence
requirement. The compute benchmark below is supporting kernel evidence only;
it is not an FPS or end-to-end frame-time claim.

Upstream records tiled RGBA16F scanout for PPSA11386 in
AGC technical debt,
but its bring-up stops before
stable presented frames. The public homebrew candidates inspected did not
provide an original, working half-float scanout workload.

What

Replace four eight-step binary searches per half-float pixel in
ColorTransfer.comp with one packed LUT lookup per channel. The 63,492-byte
table is generated from the existing thresholds and uploaded once into
device-local memory. RGB/alpha rounding, negative values, infinities, NaNs,
channel order, the 8-bit path, and the CPU reference retain their existing
results.

Add exhaustive coverage of all 65,536 half patterns in each RGBA channel, both
red/blue orders, and both guest-memory and image-copy paths. Existing random
cases and assertions remain. Five files change; no dependency or
vendor-specific branch is added.

RTX 3060 Ti compute benchmark

On an RTX 3060 Ti, the full shader's median compute-stage GPU interval was:

Half-float tiled scanout Original Packed LUT Reduction
1920x1080 0.200352 ms 0.066688 ms 66.71%
3840x2160 1.122912 ms 0.311008 ms 72.30%

Eight scenarios used 31 alternating pairs after warm-up (496 timestamp
samples). Readback and exhaustive per-channel comparisons passed. Both tables
were device-local. Timestamps exclude initialization, uploads, CPU work, waits,
and readback. The 4K baseline tails were variable: p95 was 3.636192 ms for
positive half data and 2.282512 ms for tiled positive half data, so the table
uses the predeclared median. The 8-bit controls were 0.050048 -> 0.050048 ms at
1080p and 0.193216 -> 0.192608 ms at 4K.

The measured prototype SPIR-V SHA-256 is
CEC55C0624653E42117E549715EF75472D59DE5D2E0046E31BBC296557252DF1.
The build-generated production SPIR-V is byte-identical to the measured file.
The benchmark was recorded at change 725224be9f1d83786e3749e4c135031d32dce127
on main 77b9f7270c25518d26052671311f43497328fdc7; the measured shader source is
unchanged at the prepared revision.

The trade-off is one initialization update/fence wait per transfer object and a
64 KiB device-local pool slot. A new device-local pool size class can reserve a
minimum 2 MiB slab. The LUT is recorded with vkCmdUpdateBuffer; compile-time
checks enforce its 64 KiB and four-byte size limits. This avoids a temporary host
staging buffer and its retained host-visible slab; command-buffer storage for
the update still has a driver-managed cost. Synchronization follows the
existing private CommandBatch initialization pattern and uses a
transfer-write to compute-read barrier.

Tested

The latest full run passed on a combined integration tree, not on the
GPU-only PR. Tested tree 9e65300c03a3ac4137c6b8f3550aa97dd6c063dd is main
ee9037d85154d0c840517c46c61087a736fe0641 plus this GPU change
1e187fb4d33791939b422b9c458760903ade8b16 and the separate
legacy CRT log-stream fix.
That seven-file relinker patch stays outside this five-file GPU diff.

The original windows_import_modules failure reproduces on clean main
ce5c8395333fbd4bfacdc35065f886333162caa5: MSVCRT stdout is absent from the GUI
log. The separate patch repairs missing FILE state and shares the original
native file handle/position, preserving inherited streams. Original and added
mixed UCRT/MSVCRT cases pass, including Unicode paths, changed CWD, exact
message counts and preservation of the native startup prefix. No assertion or
timeout was weakened. Independent source review found no remaining issue.

Windows 11 build 26200, Ryzen 5 5600, RTX 3060 Ti 8 GiB, NVIDIA 617.14,
Vulkan 1.4.351, WinLibs GCC 15.2 UCRT, Release.

  • Full integration CTest: 1117 passed, 2 expected skips, 0 failures out of 1119 cases in 147.66 s.
  • All 796 test executable targets are registered; author conventions and skip audits passed.
  • Expected skips: agc_driver_stencil_ref_export (missing VK_EXT_shader_stencil_export); guest_video_out_wqhd_detection_no_vulkan (intentional missing-Vulkan-ICD fixture).
  • agc_driver_graphics, including exhaustive half-float checks, passed in 7.75 s.
  • conventions_function_bodies passed in 51.04 s; no timeout was changed.
  • Earlier GPU-only full run at 88649c3a93abe685b715042ea4ed5b1597c4e0e6 on main feddf7f4c654112663b7576c8a1bc3b87657396d: 1114 passed, 2 expected skips, 1 failed out of 1117. That original failed result remains in its attachment.
  • Configured with BUILD_TESTING=ON, ANYPS5_ENABLE_SPIRV_TOOLS=ON, ANYPS5_REQUIRE_VULKAN=1, ANYPS5_GPU=NVIDIA GeForce RTX 3060 Ti, and ANYPS5_VIDEO_OUT_TEST_DRIVER=windows.
  • Production SPIR-V remains byte-identical to the measured prototype. The one-time vkCmdUpdateBuffer upload change is covered by this full integration run; earlier kernel timestamps exclude initialization.
  • Linux/macOS and maintainer-triggered upstream full CI were not run.

The standalone GPU PR still requires resolution of the upstream Windows
failure. The passing combined run does not replace the GPU-only result or
demonstrate a half-float title gain. The PR remains a draft. The relinker fix
is published as a separate fork branch; it is not merged upstream, and no
additional upstream PR was opened over the first-contributor limit.

Evidence attachments

Checklist

  • Based on main at the fixed tested revision; no duplicate open packed-LUT implementation found
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; no silent stub added
  • No notes, investigation files or images added to the repository
  • New third-party code is a submodule built from source (none added)
  • Generic behavior, independent of one title
  • Depends on: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 3060 Ti, NVIDIA 617.14, Windows 11 build 26200
  • AI-assisted: yes

The title-performance requirement remains open as described above: a real
half-float title/scene before/after profile is still required. This draft
does not claim that the kernel benchmark satisfies that requirement.

@KROU4
KROU4 force-pushed the perf/gpu-color-transfer branch from f7f8015 to 88649c3 Compare October 14, 2026 00:33
@github-actions github-actions Bot added the agc libSceAgc* label Oct 14, 2026
@KROU4
KROU4 force-pushed the perf/gpu-color-transfer branch from 88649c3 to 0ac5e41 Compare October 14, 2026 01:07
@KROU4
KROU4 force-pushed the perf/gpu-color-transfer branch from 0ac5e41 to 1e187fb Compare October 14, 2026 01:30
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 18, 2026
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Graphics/src/GpuColorTransfer.cpp

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* needs rebase Conflicts with main, rebase needed perf numbers missing perf pull request without a before and after summary line perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant