Skip to content

feat(shader): sample converted unorm images in the shader - #5958

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
ps54all:feat/shader-converted-image-sample
Oct 18, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
ps54all:feat/shader-converted-image-sample

Conversation

@ps54all

@ps54all ps54all commented Oct 14, 2026 •

Copy link
Copy Markdown

What

image_sample on a 10_11_11_UNORM (format 30) image threw sampling, gathering or querying LOD of a converted unorm image is not implemented. The format has no Vulkan equivalent, so it is bound through an R32_UINT view that cannot be filtered. God of War Ragnarök (PPSA08330) stopped on a converted-image sample in compute shader 0x10527dd300, about 60 s in, after its MFSR setup (#5796) and the SINT D16 stores (#5899).

These samples now go through the emulation that color comparisons already use (EmitEmulatedCompareSample, renamed EmitEmulatedSample):

  • One texel (point) or four texels (bilinear) are fetched at LOD 0 through the R32_UINT view.

  • Each texel is decoded and swizzled like image_load (UnpackImageTexel), then blended. Bilinear weights are rounded to the hardware's 8-bit grid (roundEven(fraction * 256) / 256), as adriadam10 measured on gfx1035 in this PR.

  • The sampler state (point or bilinear, wrap or clamp-to-edge) reaches the shader through the existing comparison specialization words, or through the runtime metadata for registered shaders. EmulatedCompareState now fills it for these images (convertedSampleState), and the binding builder provides those words whenever the converted-sample mode is compiled in.

  • The tracker sets two new bits in emulatedCompare:

    • ConvertedSampleUnsupported for gathers and LOD queries;
    • ConvertedSampleRequiresSingleLevel for samples that are not _lz.

    RuntimeImageModes adds the converted UNORM mode for sampled images that these two bits do not exclude.

  • These cases still throw (rejectImage at decode, or when the sampler state is built):

    • gathers and LOD queries;
    • clamp-to-border and mirrored addressing;
    • other filters, min/max reduction, unnormalized coordinates;
    • images selected through an image table;
    • views other than 2D and 2D array;
    • several mip levels with a sample that is not _lz.
  • The R10_G11_B11 float format (43) is not part of this PR: since feat(agc): sample texture format 43 through a lossless B10G11R11 copy #7271 its sampled images use a lossless B10G11R11 copy and the hardware sampler. On the converted path (storage or packed images) a float sample keeps main's refusal.

Cost: none for a title that never samples this format. A shader gets the converted sample mode only once a converted descriptor reaches one of its sampled images:

  • GuestContext::convertedSamples adds it (ImageResource::convertedSamples, set when the program is prepared). The flag is part of every cache key, the disk cache included.
  • Without the flag, a capture that meets a 10_11_11_UNORM descriptor in a sampled image that could take the mode throws ConvertedSampleRequired.
  • The driver (CaptureInvocation) records the program offset in the registered program's PreparedShaders, gets the invocation again (now with the flag, so the static key differs and the existing prepare-at-use path builds the new artifact) and repeats the capture. Recompile(request) retries the same way.
  • So such a shader is prepared twice per run; the record is not kept across runs (the debt entry says so).
  • The emulated color comparison emits the same scalar code as main.

Tested

  • Windows 11, RTX 3080 (NVIDIA 610.74), MinGW-w64 GCC 15.2.0 (winlibs), Release with BUILD_TESTING=ON, on main 91ddd9e (head 8545c23).
  • agc_driver_image_converted_unorm:
    • Replaces the UNORM sample refusal with image_sample_lz checks: point and bilinear filtering, clamp-to-edge and wrap addressing, on-grid and off-grid steps, swizzles X Y Z 1, Z Y X 1 and X 0 1 1. Every component is compared against a decode-and-blend model with 8-bit-grid weights.
    • Main's float point-sampling checks (CheckFloatSample) are kept as they are. Clamp-to-border is refused. The gather and LOD-query refusals stay.
    • The test passes.
    • With the weight rounding removed it fails (off-grid wrapped bilinear image_sample_lz X Y Z 1: thread 2 component 0 is 0.994141, expected 0.992188). With the bilinear neighbours swapped it fails (bilinear image_sample_lz Z Y X 1: thread 0 component 0 is 0.000489, expected 0.000244).
    • It also checks that the sample shader prepared without the flag has no converted mode and with it has exactly one; that an artifact prepared without the flag does not match a request with it; and that a capture of a converted descriptor through it throws ConvertedSampleRequired, while one through the flagged artifact succeeds.
  • Full CTest: everything passes except pr_overlap_tools and check_main_sync (git 2.30 here lacks merge-tree --write-tree), guest_sce_net (DNS timeout) and agc_driver_bda_device, which fails the same way on plain main 91ddd9e (Vulkan result -4; perf(shader): look up the BDA range of a user-data pointer once at program entry #5859 on NVIDIA, fixed by fix(shader): leave the BDA lookup functions through one return #7571).
  • tools/check_conventions.py --base origin/main and tools/check_test_registration.py pass.
  • Earlier versions of this PR (before the rebase onto feat(agc): sample texture format 43 through a lossless B10G11R11 copy #7271): Hollow Knight: Silksong cold start, main against this branch, gave identical SPIR-V (2,500,691 words) and graphics pipeline creation within run-to-run noise. PPSA08330 got past the stop in 0x10527dd300 and ran to about 180 s, to an unrelated stop (runtime storage image mip capacity exceeded). Those runs used the version that also covered the float format and were not repeated after the rebase.
  • adriadam10 measured the weights and keravyn-ops ran the test on an AMD 780M (Windows); both on the earlier version.
  • Not tested: Linux (CI), a console.

Checklist

🤖 Generated with Claude Code

@CnRJay CnRJay left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Routing these samples through the existing compare emulation is a sensible way to get exact-format decoding without a Vulkan equivalent. The test is thorough: it fails when the bilinear neighbours are swapped.

The cost section is the part I'd like quantified, since it affects every title, not only those with converted formats: "a sampled image whose format is unknown at registration now also carries the converted modes in its runtime-mode switch", which is one more decode-and-blend path per sample instruction. Specialization folds the branch away at pipeline creation, but the module still grows. The driver parses and optimizes the larger module before dead-code elimination removes the unused path, so pipeline compile time rises for every registered shader with such samples. Earlier PRs that added arms to hot shader paths (#4322) were closed for exactly this kind of compile-time increase.

Two numbers would settle it:

  • the SPIR-V word count, and the vkCreate*Pipelines time for a few registered pixel shaders of a title that never uses 10_11_11 (any UE or Unity title), before and after;
  • or a cold-cache start of such a title with APS5_TRACE_SECONDS, comparing the shader-compile phases.

If the growth is significant, could the converted modes be added only once a converted descriptor has been seen for that shader, for example by preparing it again at that point? The cost would then fall only on titles that use the format.

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* labels Oct 14, 2026

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run on an Apple GPU, since the PR was tested only on an RTX 3080.

GPUs tested: Apple M2 Max, MoltenVK 1.4.2 (Vulkan 1.4.357), macOS 27.2, x86-64 under Rosetta

main does not run the GPU tests on macOS yet, so I tested on macos-port at b7eb6b6b (mugurc/AnyPS5) plus the ImportWatched change from #1, with this PR's own diff (merge-base..head 8338defe5) applied. Not main. Release build, LunarG Vulkan SDK 1.4.363.0. Suite: ctest -R "^agc" -E "agc_display_buffer_dcc|agc_driver_flip|agc_driver_lds_atomics64" -j1 --timeout 30. Without the PR, 13 tests fail on this setup (7 with the #2704 build), all known macOS failures listed in #1.

The diff does not apply cleanly to that branch. Two hunks conflict:

  • ResourceMaterializer.cpp: the branch has neither rejectImage/PoisonReason nor ImageResource::table.
  • SpirvImageEmitter.cpp: the branch still has the bindless-slot check.

I adapted them: the same new checks, written as throw, with indirectRoot != NoIndirectImage in place of table != NoTable, and the slot check kept. Treat this as an approximation of the PR, not the PR itself.

  • agc_driver_image_converted_unorm passes. That includes the new point and bilinear image_sample_lz checks with clamp-to-edge and wrap, and both formats.
  • With only the new test on the unchanged branch, it fails with samples or gathers a converted float image, or queries its LOD, which is not implemented. So the sample paths run on this device.
  • Suite: the same 13 failures with and without the PR. None new, none fixed.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@ps54all

ps54all commented Oct 14, 2026

Copy link
Copy Markdown
Author

Thanks for the Apple GPU run and for adapting the two hunks by hand. Good to see the point and bilinear image_sample_lz checks pass on MoltenVK and that the test fails there without the change too.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 15, 2026
This was referenced Oct 15, 2026
@gjonn

gjonn commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Another data point from PPSA01317 on current main, Windows 11, RTX 5090 Laptop: this PR fixes its abort, shader 0x145628e00 stage 5: samples or gathers a converted float image, or queries its LOD, which is not implemented, which hits on its first presented frames. With this PR's three commits cherry-picked, the title gets past it and boots further.

There's one limit for later, not a request for this PR: after a few more fixes the same title samples a format-43 image (packed 11_11_10 float) as a cube and stops on samples a converted float image that is not a 2D or 2D array view. So the conversion path only covers 2D and 2D-array views.

AI-assisted: yes

@gjonn

gjonn commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Heads-up: I opened #7271, which samples texture format 43 (10_11_11 float) natively. It copies the texels losslessly into a VK_FORMAT_B10G11R11_UFLOAT_PACK32 image (guest R and B fields swapped, every bit kept) and swaps R and B back in the view. Point and bilinear filtering, every mip and LOD, and cube faces then go through the hardware sampler like any other float texture. This PR stays useful for the other converted formats. Format 43 would just no longer need the in-shader conversion.

The two PRs touch the same test. If both merge, the bilinear R10_G11_B11_FLOAT check in ImageConvertedUnorm.cpp that expects the 8-bit-grid result (1.693848) would have to drop that expectation for format 43. Native filtering gives 1.694336, and since native filtering is what the console's TMU does, 1.694336 is the exact value. Whichever PR merges second can make that change. I'm happy to rebase mine if yours goes in first.

AI-assisted: yes

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#7271 is merged, and it overlaps this PR:

  • Format 43 (R10_G11_B11_FLOAT) sampled images now go through a lossless B10G11R11 copy and the hardware sampler. In ResourceMaterializer.cpp they are nativeRepacked (sampled, non-storage, non-packed) and no longer get conversionFormat, so the float half of the converted-sample emulation added here no longer runs for them.
  • ImageConvertedUnorm.cpp conflicts with main in three hunks. Main replaces the float image_sample_lz refusal with CheckFloatSample (point sampling, bit-exact). The float bilinear and off-grid checks here model weights quantized to roundEven(frac * 256) / 256; the native sampler does not do that on every GPU, so they cannot be kept as they are.
  • libSceAgcDriver/CMakeLists.txt also conflicts (adjacent target_include_directories lines, trivial).

Please rebase on main and decide the float part explicitly:

  • Keep this PR to the 10_11_11 UNORM (format 30) path, which main still refuses, and drop the float emulation and its test runs; or
  • Keep float only for the cases #7271 leaves converted (storage / packed images) and say which test covers them.

The UNORM path itself was already checked (1/256 weights match gfx1035) and needs no change. Re-run agc_driver_image_converted_unorm after the rebase, with the negative check (bilinear neighbours swapped / rounding removed) still failing.

10_11_11_UNORM (format 30) images are read and written through an
R32_UINT view, so the hardware sampler cannot filter them. image_sample
of a 2D or 2D array view now fetches the one or four texels at LOD 0,
decodes them and blends them in the shader, with bilinear weights on
the 8-bit grid, for point or bilinear filtering with wrap or
clamp-to-edge addressing.

A shader gets the converted sample mode only after a converted
descriptor reaches one of its sampled images: that capture throws
ConvertedSampleRequired and the driver prepares the program again
with GuestContext::convertedSamples.

The R10_G11_B11 float format is sampled natively since boykopovar#7271 and keeps
its refusal on the converted path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ps54all
ps54all force-pushed the feat/shader-converted-image-sample branch from 87a9b8d to 8545c23 Compare October 18, 2026 16:40
@ps54all ps54all changed the title feat(shader): sample converted unorm and float images in the shader feat(shader): sample converted unorm images in the shader Oct 18, 2026
@ps54all

ps54all commented Oct 18, 2026

Copy link
Copy Markdown
Author

@adriadam10 rebased on main (91ddd9e) as one commit, 8545c23, taking the first option: this PR now covers only the 10_11_11 UNORM (format 30) path, and the float emulation is gone.

  • Format 43 sampled images are nativeRepacked since feat(agc): sample texture format 43 through a lossless B10G11R11 copy #7271. On the converted path (storage or packed), a float sample keeps main's refusal (samples or gathers a converted float image ...). RuntimeImageModes adds only the UNORM mode once a converted descriptor is seen, and convertedSampleState, the ConvertedSampleRequired trigger and the binding builder check format 30 only.
  • ImageConvertedUnorm.cpp: kept main's CheckFloatSample, and dropped the float bilinear and off-grid runs along with the float expectations on the 8-bit grid. The UNORM point, bilinear, wrapped and off-grid checks are unchanged. CheckConvertedPreparation now expects exactly one converted mode.
  • CMakeLists.txt: the adjacent target_include_directories lines, both kept.

RTX 3080 (NVIDIA 610.74), Windows 11, MinGW GCC 15.2, Release: agc_driver_image_converted_unorm passes. The negative checks still fail:

  • weight rounding removed: off-grid wrapped bilinear image_sample_lz X Y Z 1: thread 2 component 0 is 0.994141, expected 0.992188;
  • bilinear neighbours swapped: bilinear image_sample_lz Z Y X 1: thread 0 component 0 is 0.000489, expected 0.000244.

In the full CTest, agc_driver_bda_device fails here, and also on plain main 91ddd9e (#5859 on NVIDIA, which #7571 fixes). The title and body are updated to match.

AI-assisted: yes

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@spiff637

Copy link
Copy Markdown

Tested this PR at 8545c2301 with another region of the title: God of War Ragnarök PPSA08329 (06.006.000).

  • Setup: Windows 11 Enterprise 10.0.26200, NVIDIA GeForce RTX 3080, driver 617.42.
  • Before (main): it aborted on compute shader 0x1038fdd300: sampling, gathering or querying LOD of a converted unorm image is not implemented. The shader uses image_sample_lz ×5, image_sample_lz_o ×7 and image_gather4_lz ×4; there is no get_lod.
  • With this PR: it ran past that point for 5+ minutes with no driver failure.
  • Caveat: this region's eboot has a direct syscall at code offset 0x116c84, so it was relinked with --skip-syscall-check. That is unrelated to this change.

@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 18, 2026
@adriadam10
adriadam10 merged commit 0fd3555 into boykopovar:main Oct 18, 2026
10 checks passed
adriadam10 pushed a commit that referenced this pull request Oct 19, 2026
The data-hit verification keeps this branch's pending-span query on its ShaderMemory and takes #5958's CaptureInvocation, which retries a capture with converted samples.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs shader core/shader unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants