Repository navigation
feat(agc): scale the presentation with FSR 1 under ANYPS5_UPSCALE=fsr - #10043
EagleBearerYS wants to merge 4 commits into
Conversation
3rdparty/FidelityFX-FSR holds ffx-fsr/ffx_a.h, ffx-fsr/ffx_fsr1.h and license.txt, copied unmodified from GPUOpen's FidelityFX-FSR at tag v1.0.2 (a21ffb8f6c), the MIT-licensed reference of FidelityFX Super Resolution 1.0. The two headers are compiled into the presentation's EASU and RCAS shaders. README.md names the source, the tag and each file's SHA-256. The files are copied rather than added as a submodule because every release tag of that repository carries two submodules for AMD's sample (Cauldron and Cauldron-Media) that a recursive checkout would fetch: about 337 MB at depth 1 and 1.47 GB with full history, for 243 KB of headers.
With ANYPS5_UPSCALE=fsr the presentation draws or blits what it would show into a stage image of the swapchain's format and presents it through AMD's EASU and RCAS (ffx_fsr1.h, compiled by glslang like the other presentation shaders). When the fitted image is larger than the display buffer on either axis, the stage holds the display buffer at its own size and EASU upscales it into an image of the fitted size; otherwise the stage holds today's scaled image and only RCAS runs. RCAS writes the fitted rectangle and black letterbox bands. The stage holds today's decode of the display buffer (the presentation pass, or the blit of a resident or uploaded image), so FSR works on the values the display receives: sRGB-encoded for SDR and PQ code values for HDR10. RCAS output is clamped to [0, 1], so PQ never exceeds its range. ANYPS5_FSR_SHARPNESS sets the RCAS sharpness in stops, 0.0 to 2.0, default 0.2. Without ANYPS5_UPSCALE no upscaler is created and the presentation records the same commands as before. FSR runs only for a presentation it can upscale: a fitted image larger than the display buffer on either axis. Any other presentation is recorded exactly as without ANYPS5_UPSCALE, since RCAS alone would cost frame rate there without adding detail. The stage's render pass ends with a dependency that orders its writes and final layout transition before EASU and RCAS sample it.
agc_driver_presentation_fsr runs agc_driver_presentation_pass_tests --fsr on lavapipe. It checks the variables' parsing, the stage size decision, EASU from 960x540 to 1920x1080 (finite values within [0, 1], flat regions unchanged, edge transitions narrower than the bilinear scaler's), RCAS at 1:1 (sharpness 2.0 close to the input, 0.2 raising edge contrast, flat regions uniform, sharpness 0.0 within [0, 1]), that the default path's output stays bit-identical next to FSR and matches the FSR stage inside the image, and SDR and HDR10 presentations through the presentation pass with letterbox bands. It also checks the presentation render passes' exit dependencies: one that ends in SHADER_READ_ONLY_OPTIMAL orders its writes and final layout transition before a fragment-shader read, one that ends in TRANSFER_SRC_OPTIMAL before a transfer read, and the one that ends in PRESENT_SRC_KHR keeps its dependency. The test's drawn FSR presentations rely on that dependency alone, with no barrier after the render pass. agc_driver_present_upscale presents through a VK_EXT_headless_surface swapchain and checks VulkanDevice::UpscaledPresents, the count of presentations that ran FSR: - 3840x2160 and 1920x1080 into 1084x610 don't run FSR; - 1084x610 at 1:1 doesn't run it; - 960x540 into 1084x610 does; - with ANYPS5_UPSCALE empty, none do. The test skips without the extension. agc_driver_present_upscale also presents in all four combinations of FSR and the performance overlay, and agc_performance_overlay checks the overlay drawn over an FSR presentation (960x540 upscaled to 1280x720), with its FPS digits drawn bit for bit.
agc_driver_presentation_pass_tests --fsr-cost is a cost mode and is not registered with ctest. It records the production upscaler's passes into B8G8R8A8 images and brackets each part with Vulkan timestamps: - the stage write; - EASU and RCAS together, as RecordPresent records them; - EASU alone (RecordUpscale); - RCAS alone (a stage at the fitted size); - today's bilinear blit. It prints one line per case with the device name and the median and 95th percentile over 200 frames after 20 warm-up frames. Each line also gives the CPU time spent recording and releasing the FSR presentation next to the blit's. The cases: - 1920x1080 into 2560x1440 and into 3840x2160; - 1920x1080 and 3840x2160 at 1:1; - 1280x720 into 3840x2160; - 3840x2160 into 2560x1440. A queue without timestamps exits 77.
MarcZX10
left a comment
There was a problem hiding this comment.
Reviewed the current implementation and reported validation for head 672308b453c902d8434842e8bb8e9779eba33da3. The feature has substantial focused coverage and evidence across lavapipe and the RTX 3070, including default-off bit identity and presentation-path checks. One integration blocker remains: check_main_sync.py shows this head is 4 ahead/209 behind fetched upstream/main at 0db47b263, with four changed paths also modified after the merge base: core/libs/prx/libSceAgcDriver/CMakeLists.txt, Execution/src/VulkanDevice.cpp, and both usage/environment docs. The reported full Linux suite and tests are on base 7e69be679, so they do not establish the combined current-main tree. Please rebase/merge current main, resolve these shared paths, rerun overlap and the focused presentation/upscale/overlay/VideoOut tests on the resulting head, and report any changed failures. This request is for current-main integration evidence required by the repository contribution checklist; it is not about queue size. Local conventions pass; hosted checks currently cover policy only and skip product builds.
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
@adriadam10 a question about the merge queue, since you wrote on #9790 that MarcZX10's CHANGES_REQUESTED blocks it until it's addressed. On this PR, and on #10159, his latest review asks only for a refresh against |
What
An optional scaler for the presented image: AMD FidelityFX Super Resolution 1.0, EASU upscaling plus RCAS sharpening.
ANYPS5_UPSCALE=fsrturns it on. The default stays today's bilinear scaler.No title needs this to run, and nothing changes unless it's set. With
ANYPS5_UPSCALEunset, no upscaler object exists. The present path records exactly what it records today, and a test checks the output is bit-identical.Where it helps: a display buffer shown in a larger window, for example a 1920x1080 buffer in a 2560x1440 or 3840x2160 window, which today is a bilinear stretch. Here is Planet of Lana (PPSA19450) with
platform-ps5-video-out-width=1920in itsapp0/Media/boot.config, maximized on a 2560x1440 panel, where the 1920x1080 buffer is presented at 2372x1334. The main menu at 1:1, bilinear on the left and FSR at the default sharpness 0.2 on the right:The crops on their own: bilinear and FSR. The full frames: bilinear and FSR. That one-line edit of the title's own data file isn't part of this PR. megood's run of Bendy and the Ink Machine at 1920x1080 in a larger window, shared in the AnyPS5 Discord dev chat, is what pointed us at Unity's resolution settings. Titles that present 3840x2160 buffers in play, such as Resident Evil Village (PPSA01556) and Animal Well (PPSA22520), keep today's scaler in any smaller window, since FSR stays off where it can't upscale.
How to try it on a title: for Planet of Lana (PPSA19450), set
platform-ps5-video-out-width=1920in the dump'sapp0/Media/boot.config(it ships with 3840), run it in a window larger than 1920x1080, and compareANYPS5_UPSCALEunset withANYPS5_UPSCALE=fsr. Other Unity titles may have the same line, but none has been checked yet.The test's synthetic pattern shows the difference in numbers. It's a 960x540 pattern upscaled to 1920x1080, shown as a 320x180 crop at 2x, with bilinear on the left and FSR at 0.2 on the right:
The one-pixel grid lines peak at 179 under bilinear and at 219 with FSR (the source line is 230). A 51/204 edge goes 89 → 166 under bilinear and 65 → 190 with FSR.
Sharpness 0.2 (left) against 2.0, the weakest (right). EASU is the same in both; RCAS's halo is 5-9 levels at 0.2 and 1 level at 2.0:
The whole synthetic FSR frame at half size, for context: image. These three are rendered by this PR's own code on lavapipe.
How:
ANYPS5_UPSCALEandANYPS5_FSR_SHARPNESS(0.0 sharpest to 2.0 weakest, default 0.2) are read once when the window's device is created, likeANYPS5_PRESENT_MODE. An empty value is the same as unset; any other bad value stops the start with a message naming the variable.With
fsr, a presentation whose fitted image is larger than the display buffer on either axis goes through FSR:present()renders today's output at the buffer's own size into a per-slot stage image;FsrEasu.frag) scales it to the fitted size;FsrRcas.frag) sharpens it into the swapchain image, with the black letterbox around it.This covers the drawn, direct and shared present paths. Clear-only presents skip it.
With the performance overlay (feat(agc,videoout): show an optional performance overlay with F10 #6012) on: RCAS ends in
COLOR_ATTACHMENT_OPTIMAL, and the overlay records from there when the frame was drawn or upscaled. Otherwise the overlay records fromTRANSFER_DST_OPTIMALas it does today. The overlay's entry dependency orders it after RCAS's writes, so its text is neither scaled nor sharpened.Where it can't upscale (1:1 or a smaller window), FSR stays off for that frame, which is presented exactly as without
ANYPS5_UPSCALE. RCAS alone would cost frame rate there without adding detail.QueuePresentwaits for the presentation's rendering inside it.[present] ANYPS5_UPSCALE=fsr: FSR 1 EASU and RCAS when the window is larger than the display buffer, the normal scaler otherwise; RCAS sharpness 0.20.HDR10 (feat(videoout): present PQ display buffers through an HDR10 swapchain #5697) and SDR: FSR runs on the values the display receives, sRGB-encoded for SDR and PQ code values for HDR10, never on linear light. Both are perceptual encodings, which is what FSR 1 expects. EASU's deringing keeps each output inside its 2x2 neighbourhood. RCAS limits its negative lobe to stay inside [0, 1], and the shader clamps its result to [0, 1] after AMD's approximate reciprocal, so PQ output stays a valid code.
Cost when upscaling: two images per presentation in flight (7.9 + 31.6 MiB per slot for a 1080p buffer in a 4K window) and two extra passes per frame. The table gives FSR's GPU time per frame (the stage write, EASU and RCAS) against today's bilinear blit, as the median over 200 frames from
agc_driver_presentation_pass_tests --fsr-cost. The times are GPU timestamps of 200 back-to-back frames after 20 warm-up frames, offscreen with no other GPU work. The RTX 3070 column is from this head. Its clocks weren't held: during the measured frames nvidia-smi (every 100 ms) showed it in P3 and P0, at 780 to 945 MHz graphics and 5,501 to 6,001 MHz memory. So these are mid-clock figures, upper bounds for both passes at full clock. FSR and the blit were timed in the same frames at the same clocks, so the comparison is like for like at those clocks. lavapipe runs on the CPU, so only its ratios mean anything. Recording FSR costs 45-55 µs of CPU per frame on the RTX 3070 when it upscales (median; 89-101 at p95), against 3-4 µs for the blit.The cost mode measures RCAS alone too, which is what FSR would run where it can't upscale. At 1920x1080 1:1 it costs 0.12 ms against the blit's 0.05 on the RTX 3070 (20.6 against 5.0 on lavapipe), at 3840x2160 1:1 0.46 against 0.20, and for 3840x2160 into 2560x1440 0.29 against 0.15 (34.7 against 9.2 on lavapipe). That gives no detail for the cost, so FSR stays off there.
The shaders derive the EASU and RCAS constants per pixel from the sizes and the sharpness in their push constants, about 20 ALU per pixel; AMD's samples compute them once on the CPU, which would save that at the cost of a larger push-constant block. When off, the cost is two
getenvcalls at device creation and one bool per present.Things to know:
APrxMedRcpF1), so it darkens flat areas by a fixed fraction of their value that depends only on the sharpness: 0.17% at the default 0.2 (0.5 → 0.4992), 0.32% at 0.0 and 0.07% at 2.0. That's at most one 8-bit level in SDR and three 10-bit codes in HDR10. Fixing it would mean editing AMD's header, so it's left as is; USAGE says so, and the tests bound it.2.0is RCAS's weakest setting, not off. Withfsrselected, RCAS always runs.ANYPS5_FSR_SHARPNESSstops the start even withoutfsr, like any invalid setting.Third-party code: the FSR headers are vendored, not a submodule, which departs from AGENTS.md's submodule rule. A maintainer agreed in the dev chat that vendoring these two headers is fine.
3rdparty/FidelityFX-FSR/holdsffx-fsr/ffx_a.h,ffx-fsr/ffx_fsr1.handlicense.txt(MIT), copied unmodified from tag v1.0.2 (a21ffb8f6c). Its README gives the source, the commit and each file's SHA-256..gitmodulescan't stop recursion into a submodule's own submodules.Commits:
build(agc): the vendored FSR 1.0.2 headers and license, with the README.feat(agc):PresentationUpscaler,UpscaleMode, the EASU and RCAS shaders, and the hooks intoPresentationPass,PresentationScalerandVulkanDevice::present.PresentationPass::ExitDependency, shared with the upscaler's passes).Graphics::CreateShaderModule, asmain's presentation pass and overlay create theirs.docs/dev/ENVIRONMENT.mdgets rows forANYPS5_UPSCALEandANYPS5_FSR_SHARPNESS.test(agc):agc_driver_presentation_fsr, with the exit dependency's three layouts;agc_driver_present_upscale, which presents throughVulkanDeviceon a headless surface in all four combinations of FSR and the overlay and counts the presentations that ran FSR; andagc_performance_overlay's over-FSR case.test(agc): the--fsr-costmode ofagc_driver_presentation_pass_tests(30febcf), not registered with ctest.Tested
Planet of Lana (PPSA19450) with
platform-ps5-video-out-width=1920inapp0/Media/boot.config, built at this head, on the RTX 3070 Laptop GPU (driver 617.42, Windows 11), maximized on a 2560x1440 panel, with the performance overlay on. Six runs: four in the order bilinear, FSR, FSR, bilinear, then a later pair, FSR then bilinear (ANYPS5_UPSCALEunset orfsr). Clocks weren't held; nvidia-smi logged them every second.guest 1920x1080 presented at 2372x1334, and withfsrthe start line[present] ANYPS5_UPSCALE=fsr: FSR 1 EASU and RCAS when the window is larger than the display buffer, the normal scaler otherwise; RCAS sharpness 0.20. The performance overlay readsRES: 1920X1080 / 2560X1334.agc_driver_presentation_fsr(Vulkan; it runsagc_driver_presentation_pass_tests --fsr) covers:ANYPS5_UPSCALEunset the output is bit-identical, and the stage equals today's fitted image when FSR doesn't upscale;agc_driver_present_upscalepresents throughVulkanDeviceon aVK_EXT_headless_surfaceswapchain (1084x610) and counts the presentations that ran FSR. Withfsr, 3840x2160, 1084x610 and 1920x1080 buffers don't run it and a 960x540 buffer does; with the variable empty, nothing does. With the gate removed it fails: "FSR ran on a 3840x2160 display buffer in a 1084x610 window, where it cannot upscale". It skips where the extension is missing.Linux, CI's Linux job (Ubuntu 24.04, GCC 13.3, CI's configure flags, lavapipe, an unprivileged user), at the head, against
main7e69be6 built the same way:main.mainhas the same status here. The two new tests,agc_driver_presentation_fsrandagc_driver_present_upscale, pass.guest_net_buffer_validation, which fails onmainthere too.check_test_registration.pyfinds nothing unregistered on either.check_skips.pyflags onlyguest_sce_neton both, as on every branch on that machine.agc_driver_presentation_pass,agc_driver_presentation_fsr,agc_driver_present_upscale,agc_performance_overlayandagc_performance_overlay_statisticspass 10 of 10, headless and under Xvfb.Validation with
VK_LAYER_KHRONOS_validationand synchronization validation. The layer reports a planted write-after-write hazard on that machine, so validation was on.video_out_hdr_present's 259x137 buffer is presented at 1152x609.main, plus the 18 on X11.main, then this PR unset, withfsr, withfsrat sharpness 2.0, and withfsrand the performance overlay on (ANYPS5_OVERLAY=1).VUID-VkDeviceCreateInfo-pNext-pNext, once per device, from a device struct that layer version doesn't know. Every testmainhas gets the same count here. The 6 extra come fromagc_driver_present_upscale, which opens a device per case.agc_driver_presentation_fsrhad 10.fsr, FSR's start line appears 13 times in each configuration, fromvideo_out_hdr_present, sixguest_video_out_*tests,agc_driver_present_upscaleand three X11 flip and frame-dump tests. With the overlay on as well, the overlay tests pass, and the screen capture below shows the overlay drawn over an FSR presentation.Off is bit-identical on screen too: X screen captures at the third present of
video_out_hdr_present, compared pixel by pixel.ANYPS5_UPSCALEunset,mainand this PR differ in 0 of 2,073,600 pixels, in three capture pairs.fsr, 679,356 pixels differ, all in the presented image: the frame is sharper at the band edges. Adding the overlay tofsrchanges only the panel (11,023 pixels).GitHub's runners at 672308b (run): the presentation, present-upscale and overlay tests pass 10 runs in a row on Linux, with none skipped. On Windows they build and pass, with 5 of the 8 skipping: the runner has no Vulkan device, so the RTX 3070 run below covers those. The libc and libkernel suites pass on both, run with
APS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.Validation,
mainagainst this PR, on GitHub's Linux runner (run):VK_LAYER_KHRONOS_validationwithANYPS5_NO_SHADER_CACHE=1andANYPS5_REQUIRE_VULKAN=1on llvmpipe, for the same tests plusagc_driver_image_table_vouched_lookups. The loader inserted the layer 5 times onmainand 12 with this PR. Both logs report onlyVUID-VkDeviceCreateInfo-pNext-pNext, so this PR adds noVUID-message.RTX 3070 Laptop GPU at this head (driver 617.42, Windows 11 10.0.26300), offscreen, no other GPU work:
agc_driver_presentation_pass(3.7 s),agc_driver_presentation_fsr(10.9 s) andagc_performance_overlay(2.5 s) pass.agc_driver_present_upscaleskips: NVIDIA's Windows driver has noVK_EXT_headless_surface, which it needs. The figures match lavapipe's: EASU's edge rise is 1.52 px against bilinear's 2.20 on both axes, RCAS at sharpness 2.0 changes the image by at most 0.0241, and edge contrast goes 0.2331 → 0.2482 at 0.2. The tests take the first Vulkan device, which on this laptop is the integrated GPU, so the loader was limited to NVIDIA's driver for that run. The cost mode (--fsr-cost) gives the RTX 3070 column above.HDR10 is tested at the component level only: no HDR10 swapchain was available. lavapipe's offers only SRGB_NONLINEAR, and this PC's displays run SDR.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)Overlap:
VulkanDevice::present(commit 2) and covers both on together (commit 3), as described under How.docs/dev/ENVIRONMENT.md.libSceAgcDriver/CMakeLists.txt.docs/user/USAGE.md.PresentationPass.hpp. perf(agc): GPU descriptor heap on VK_EXT_descriptor_heap (APS5_GPU_HEAP) #9508 makesdraw()'sSlot&non-const, and this PR addsrecordInto()next to it; both apply.main. Against perf(agc): GPU descriptor heap on VK_EXT_descriptor_heap (APS5_GPU_HEAP) #9508, which is much larger, it can go first; if perf(agc): GPU descriptor heap on VK_EXT_descriptor_heap (APS5_GPU_HEAP) #9508 lands first, this PR keeps its non-constSlot&.