[thuanquang]

What

agc_driver_f16_fma_denormal_modes (#7034) and agc_driver_fma_mix_f16_denormal_modes (#7040) compare every result bit for bit with the values the RX 6700 XT wrote, including rows whose result is a NaN (inf * 0). The technical debt entry on NaN results says the NaN bits of f16 arithmetic and of v_fma_mix_f32 with f16 sources "are left to the driver", so those rows depend on the driver: on an RTX 5080 (driver 32.0.16.1062) agc_driver_f16_fma_denormal_modes fails on main with v_fma_f16 without a known float mode of 0x7c00, 0x0000, 0x0000 is 0x7fff, expected 0xfe00 (report on #7034), and the v_fma_mix_f32 test has the same kind of row (0xffc00000 on AMD, 0x7fffffff reported on NVIDIA in #6976).

Both checks now accept any NaN where the expected value is a NaN; the f16 one still requires the high half of the register to be zero. Every other row is still compared bit for bit, so the tests keep catching the denormal handling they were written for.

Tested

Windows 11 26200, AMD Radeon RX 6700 XT (gfx1031), driver 32.0.21045.5002, WinLibs GCC 15.2.0 r7, Release, BUILD_TESTING=ON, ANYPS5_REQUIRE_VULKAN=1, main 864729dc0:

• Both tests pass.
• With core/shader/recompiler reverted to 6a32550fd (before fix(shader): flush v_fma_f16 family denormals by the f16 denormal field #7034 and fix(shader): flush v_fma_mix_f32 f16 sources by the f16 denormal field #7040) both still fail on non-NaN rows: v_fma_f16 f16 denormal mode 0, IEEE mode on of 0x3bff, 0x0400, 0x0000 is 0x0400, expected 0x0000 and v_fma_mix_f32 of three f16 sources f32 denormals flushed, f16 denormal mode 0, IEEE mode off of 0x00010000, 0x00000000, 0x3f800000 is 0x33800000, expected 0x00000000.
• Only these two test files change, so the full suite was not rerun.
• python tools/check_conventions.py --base origin/main with this title and body passes.
• Not tested: NVIDIA (no NVIDIA GPU here; the failing value comes from the report above), Intel.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: AMD Radeon RX 6700 XT (gfx1031), driver 32.0.21045.5002, Windows 11 26200
• AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

[fma-rc]

Ran this on Linux, main c9e6c45 plus e5b4b32, Release, GCC 16.2.1. Both tests pass with the PR on an RTX 5080 (driver 615.78.08), a Radeon 610M on RADV and lavapipe.

With main's two test files on the same build, both fail on the RTX 5080: v_fma_f16 without a known float mode of 0x7c00, 0x0000, 0x0000 is 0x7fff, expected 0xfe00 and v_fma_mix_f32 ... of 0x00007c00, 0x00000001, 0x3f800000 is 0x7fffffff, expected 0xffc00000. On RADV they pass either way. Every other row still has to match bit for bit, and only the NaN rows are relaxed.

[akkadbakkad89]

Validation on a second NVIDIA device, since the only NVIDIA datapoint so far is the RTX 5080 failure report: RTX 3070 (driver 617.42), Windows 11, MinGW (WinLibs GCC 15.2.0), Release, head e5b4b326c, serial fresh processes via ctest -R "agc_driver_f16_fma_denormal_modes|agc_driver_fma_mix_f16_denormal_modes":

• agc_driver_f16_fma_denormal_modes: pass (fails on main at v_fma_f16 ... is 0x7fff, expected 0xfe00)
• agc_driver_fma_mix_f16_denormal_modes: pass (fails on main at v_fma_mix_f32 ... is 0x7fffffff, expected 0xffc00000)

Independent confirmation that the NaN-relaxed checks accept NVIDIA's encodings. I had opened #7109 with the same approach before noticing this PR predates the issue itself; closing mine as the duplicate. The actual <= 0xffffu guard on the f16 form is a nice extra — it keeps catching a stray high half that a looser any-NaN check would miss.