Skip to content

fix(shader): apply cube output modifiers whenever IEEE mode is off - #5799

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
SleglJan:fix/cube-output-modifiers
Oct 20, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
SleglJan:fix/cube-output-modifiers

Conversation

@SleglJan

@SleglJan SleglJan commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

v_cubeid_f32, v_cubesc_f32, v_cubetc_f32 and v_cubema_f32 refuse any output modifier (mul:2, mul:4, div:2) with VOP3 source modifiers are not implemented. PPSA11386 (007 First Light) stops there on current main, within seconds of start, on v_cubeid_f32 v, v, -v, -v div:2.

Measured on RDNA2 (Ryzen 9 7950X3D iGPU, gfx1036, not the PS5's gfx1013) with tools/hw-oracle, 2048 inputs (edge cases and random), all four instructions with each modifier, in all 16 combinations of IEEE mode, the four f32 denormal values and f16/f64 denormals flushed or kept, with round to nearest even and DX10_CLAMP=1:

  • With IEEE mode off the modifier is applied in all eight denormal combinations, f32 output denormals kept included, while v_mul_f32_e64 ... div:2 in the same kernels follows fix(shader): apply the output modifier only in the float modes where the hardware does #1851's rule (ignored with f32 output denormals kept). With IEEE mode on it is ignored.
  • Where it is applied, the result is main's bit-level f32 output modifier result on every row: of the 24,576 results per mode, 21,768 differ from the unmodified value and all of them match; NaN and infinity pass unchanged, denormal and zero results become +0, div:2 below 2^-125 gives a signed zero, mul overflow a signed infinity. clamp applies after the modifier. The negated-source form 007 uses behaves the same.
  • With fix(shader): quiet signaling NaN cube results in IEEE mode #2043's quieting of signaling NaN results (IEEE mode on only) and the modifier (IEEE mode off only), the two never act on the same result; composed, they match the measurement on all 524,288 results of the 16 modes, 1,088 of them signaling NaNs.

Changes:

  • The decoder accepts the output modifier on the four instructions.
  • The f32 output modifier scaling moves into scaleF32Result. floatCube applies it whenever IEEE mode is off and then writes with the modifier cleared, so clamp is still applied after it.
  • floatCube does not use the generic gate (outputModifierApplies), which drops the modifier with IEEE mode off and f32 output denormals kept; for these four the hardware applies it there. In the usual mode (IEEE off, f32 denormals flushed) the two agree.
  • The refusal check in Vop3CubeClamp.cpp becomes an execution test, and its run and check helpers take the kernel, rows and names instead of being duplicated. The output modifier entry in docs/dev/debt/shader.md records the cube rule.

Only round to nearest even is modelled. The same 2048 rows in the other three round modes (IEEE mode off, f32 denormals flushed and kept) differ from round to nearest even where a result overflows to infinity: the hardware rounds the overflow as the mode says (to the largest finite value when rounding toward zero), in v_cubema_f32's own result as well as in mul:2 and mul:4. The translation gives infinity there, as main already does for v_cubema_f32 without a modifier. The only other difference is the sign of three zero v_cubema_f32 results with a NaN input when rounding toward negative infinity. The debt entry records both.

Overlap: #2463 (closed by the intake pause) accepted the modifier with the generic gate.

Tested

  • Linux (CachyOS, kernel 7.2.9, GCC 16.2.1), Release with BUILD_TESTING=ON. The measurements below were made with this branch on main 23e93c0; it was then rebased without conflicts onto main 145cb82, where tools/check_conventions.py --base upstream/main and git diff --check pass and the 19 vop3 execution tests, agc_driver_vop3_cube_clamp included, pass on lavapipe.
  • Full ctest on lavapipe (Mesa 26.2.4, LLVM 23.1.1; Arch Linux container, GCC 16.2.1) with ANYPS5_ENABLE_SPIRV_TOOLS=ON as in CI: 1107/1121; main 23e93c0 built the same way fails the same 14 (hardware_oracle, guest_syslog, guest_allocator_failures, agc_driver_bc_block_copy, agc_driver_depth_compare_copy, agc_compute_pipeline_replay, autorun_exit_status, guest_none_relocations, guest_module_args, linux_executable_exports, linux_entry_argv, linux_program_header_order, linux_input_program_headers, linux_guest_stack).
  • Full ctest on the RTX 3080 (NVIDIA 615.71.09): no failures out of 1113; 8 of them skip themselves (seven VideoOut tests and agc_driver_stencil_ref_export).
  • agc_driver_vop3_cube_clamp: a 16-instruction kernel (all four instructions with each modifier, the negated-source form, clamp with each modifier) over 32 rows (two of them with a signaling NaN input), compiled with no float mode, IEEE=0 with f32 denormals flushed (0xc0) and kept (0xf0), and IEEE=1 with both, against two tables captured on gfx1036 (IEEE=0 and IEEE=1; the hardware gives the same results with f32 denormals flushed or kept). With only the test applied to main 23e93c0 it fails on the decoder refusal. Mutations caught: the generic f32 gate (fix(shader): accept cube output modifiers #2463's behaviour; fails at IEEE=0 with f32 denormals kept), applying the modifier in IEEE mode, never applying it, and skipping the signaling NaN quieting when a modifier is present (fails on a signaling NaN row with IEEE=1).
  • PPSA11386 v01.018.000 in a bubblewrap sandbox from a cold shader cache, 3 runs each, no environment overrides: on main 23e93c0 every run ends on the decoder refusal (terminate with VOP3 source modifiers are not implemented). With this change every run gets past it, compiles about a thousand shaders and stops at its first presentation on bindless image table has 18960 entries (0 materials), limit 16. Along the way 63 to 72 shaders are logged as not prepared after registration; none of the reasons names a cube instruction or an output modifier (GetImageResource dword 0 is not a valid runtime value, the clip distance limit, static bindless image capacity, bindless storage image tables).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: PPSA11386 007 First Light (PPSA11386) #5714
  • GPUs tested: NVIDIA RTX 3080, driver 615.71.09, CachyOS (Linux 7.2.9); lavapipe (Mesa 26.2.4, LLVM 23.1.1), Arch Linux container; the reference tables come from an AMD Ryzen 9 7950X3D iGPU (gfx1036, RDNA2) through tools/hw-oracle
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@jotitadev

Copy link
Copy Markdown
Contributor

Run on an Apple GPU, as a third device next to the RTX 3080 and lavapipe.

GPUs tested: Apple M2 Max, MoltenVK 1.4.2 (Vulkan 1.4.357), macOS 27.2, x86-64 under Rosetta

main does not run the GPU tests on macOS yet, so I tested on macos-port at b7eb6b6b (mugurc/AnyPS5) with this PR's diff applied, not on main. Release build, LunarG Vulkan SDK 1.4.363.0. The suite is ctest -R agc -E 'flip|display|lds_atomics64' -j1 --timeout 30. Without the PR, 22 tests fail on this branch, all of them known macOS failures listed in #1. Head 2acac0828. One conflict, in RdnaVectorOpDecoder.cpp (supportsNativeVop3ResultModifiers): the branch doesn't have isNativeVop3F16TernaryOpcode in that condition yet, so I kept only this PR's || isCubeOpcode(opcode). Everything else applies cleanly, except docs/.

  • agc_driver_vop3_cube_clamp passes. With only the test file applied to the unchanged branch, it fails with VOP3 source modifiers are not implemented, so the new cases do run on this device.
  • Suite: the same 22 failures with and without the PR. None are new and none are fixed. agc_driver_f32_output_modifier is among them on this branch without the PR, and the cube modifier cases here still match the gfx1036 tables.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

v_cubeid_f32, v_cubesc_f32, v_cubetc_f32 and v_cubema_f32 refused any
output modifier with "VOP3 source modifiers are not implemented";
PPSA11386 stops there on v_cubeid_f32 v, v, -v, -v div:2.

On RDNA2 (gfx1036), over 2048 inputs in every combination of IEEE mode,
the four f32 denormal values and f16/f64 denormals flushed or kept, with
round to nearest even, the four instructions apply mul:2, mul:4 and
div:2 whenever IEEE mode is off, also with f32 output denormals kept,
where v_mul_f32_e64 in the same kernels ignores them, and ignore them
with IEEE mode on. Where applied, the result is the existing bit-level
f32 output modifier result, and clamp is applied after it.

The decoder accepts the modifier on the four instructions. floatCube
applies the shared f32 scaling, now scaleF32Result, in those modes and
writes with the modifier cleared, so clamp is still applied after it.
With IEEE mode on the signaling NaN quieting applies and the modifier is
ignored; with it off the quieting does nothing and the modifier is
applied. agc_driver_vop3_cube_clamp checks each instruction with each
modifier against tables captured on gfx1036.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@SleglJan
SleglJan force-pushed the fix/cube-output-modifiers branch from 2acac08 to e32f5e1 Compare October 14, 2026 18:09
@oneandonlydean oneandonlydean self-assigned this Oct 14, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4072 core/shader/recompiler/Translation/include/Translation/TranslationContext.hpp
core/shader/recompiler/Translation/src/FloatInstructions.cpp
PR 5021 core/shader/recompiler/Translation/src/FloatInstructions.cpp
docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 5426 docs/dev/debt/shader.md
PR 5500 docs/dev/debt/shader.md
PR 5941 docs/dev/debt/shader.md
PR 6009 docs/dev/debt/shader.md
PR 6235 docs/dev/debt/shader.md
PR 6280 docs/dev/debt/shader.md
PR 6412 docs/dev/debt/shader.md
PR 6488 docs/dev/debt/shader.md
PR 6977 docs/dev/debt/shader.md
PR 7085 docs/dev/debt/shader.md
PR 7395 docs/dev/debt/shader.md
PR 7484 docs/dev/debt/shader.md
PR 7684 docs/dev/debt/shader.md
PR 7888 docs/dev/debt/shader.md
PR 7922 docs/dev/debt/shader.md
PR 8004 docs/dev/debt/shader.md
PR 8077 docs/dev/debt/shader.md
PR 8267 docs/dev/debt/shader.md
PR 8281 docs/dev/debt/shader.md

@Scramblrrrr

Copy link
Copy Markdown
Contributor

Windows cross-test on current main 4524944: author-preserved e32f5e1 port at df0e8a7, followed by #6317 and #6332, integration head 00abe7e. GCC 15.2 / Release / SPIRV-Tools OFF / RTX 5070 Ti, Vulkan required. Fresh patched libraries and fixtures built successfully.

agc_driver_vop3_cube_clamp PASS 0.66 s, exercising the included oracle expectations with IEEE on/off and f32 denormals flushed/kept. Seven adjacent controls (shader memory, SPIR-V module, graphics, shared writes, write tracking, position invariance, constant phi) also pass: 8/8 total 6.24 s, no skips. This is a combined-stack Windows cross-test, not a full-suite result or new independent hardware oracle. Source/conventions clean; public integration branch: https://github.com/Scramblrrrr/AnyPS5/tree/codex/first-light-oct16-current-main .

007 is currently preparing a cold shader cache without the earlier cube stop; no rendered-gameplay or FPS claim yet. AI-assisted: Codex; title data stays local.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

The Windows run (@Scramblrrrr, RTX 5070 Ti, agc_driver_vop3_cube_clamp plus seven adjacent controls) and the Apple M2 Max run (@jotitadev, red without the PR and green with it) are a third and fourth device next to the RTX 3080 and lavapipe. Both are on a port or a combined stack, not this head on current main, so we'll run it there.

@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 18, 2026
@oneandonlydean
oneandonlydean merged commit baae664 into boykopovar:main Oct 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants