[adriadam10]

What

Part of #981. Several instructions always set the quiet bit on a signaling NaN input. The hardware does that only with IEEE_MODE=1; with IEEE=0, the mode titles use, the NaN comes out unchanged. v_ldexp_f32 had the opposite problem and never quieted. Translation now follows GuestContext::floatMode.ieeeMode (#1816) through two helpers, quietNan32 and quietNan16.

Measured on RDNA2 (gfx1035) with the HW oracle at IEEE=0 and IEEE=1, with signaling and quiet NaNs of both signs and with payloads in every operand position:

• Quieted only with IEEE=1:
  • f32 unary: v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/ceil/trunc/rndne/rcp_iflag/frexp_mant_f32
  • f16 unary: v_sqrt/rcp/rsq/log/exp/sin/cos/fract/floor/trunc/rndne/frexp_mant_f16
  • also v_ldexp_f32/f16, v_div_fixup_f32/f16 and v_mullit_f32
• Changed here: floatUnary, float16Unary, vFrexpMantF32, vFrexpF16, vLdexpF16, vDivFixupF32/F16 and vMullitF32 quiet only at IEEE=1, and vLdexpF32 now quiets a NaN at IEEE=1. A shader without a known float mode follows IEEE=0.
• Recorded, not changed: the binary and ternary arithmetic (v_add/mul/fma_f32/f16, v_mul_legacy, v_fma_mix, v_cubema) follows the same rule on hardware, but its NaN bits are still left to the driver. v_cvt_f16_f32 and v_dot2_f32_f16 always quiet, and v_div_scale_f32 never does, as the code already does.

TechnicalDebt: new line for the rule, the v_mullit_f32 line updated, and the float mode line now says only the binary and ternary arithmetic is left.

Tested

Linux (CachyOS, GCC), RADV (gfx1035) and CI's lavapipe (Mesa 25.2.8).

• New agc_driver_nan_quieting: 16 instructions × 32 rows captured on hardware at IEEE=0 and IEEE=1. 105 of the 512 cells differ between the two modes. The test runs with no mode and IEEE=0 (checked against the IEEE=0 table) and with IEEE=1, bit for bit. For v_sqrt and v_rcp, only the NaN cells are compared, because their finite results are the driver's approximation (TechnicalDebt).
• agc_driver_div_fixup_f16, agc_driver_division, agc_driver_sdwa_ldexp_f16 and agc_driver_vop1_float_unary had tables captured with the oracle's old implicit mode (IEEE=1, denormals kept), so they expected quieted NaNs. They now pin that mode (FLOAT_MODE 0xf0, IEEE=1), as InterpolationF16 does since feat(shader): implement v_interp_p1ll_f16, v_interp_p1lv_f16 and v_interp_p2_f16 #697. The tables are unchanged.
• Negative checks, each fails: quieting always in f32, always in f16, never in f32, and v_ldexp_f32 not quieting.
• Full Release ctest: 493/493, on main after feat(shader): flush f32 denormals in the float modes that flush them #1850.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none (feat(shader): carry the shader float mode into the recompile request #1816 merged). Touches the same files as feat(shader): follow DX10_CLAMP in the clamp modifier #1866 without conflicting
• AI-assisted: yes

[adriadam10]

Rebased on main after #1866 (conflict in the f16 unary NaN line, now keeps both the DX10_CLAMP and the IEEE-mode quieting). agc_driver_nan_quieting, agc_driver_dx10_clamp and the four pinned tests pass on RADV and lavapipe.

[SleglJan]

Cross-check of this PR together with #1866 against the hardware, continuing the audit on #981. The kernels of the 26 execution tests the audit found sensitive to DX10_CLAMP or IEEE_MODE (28 kernels, 16,944 result cells per mode) ran through the recompiler of ef4dca9 (the recompiler tree of main at the merge, 42e9148, before #1878) on an RTX 3080 (driver 615.71.09) and on the RADV iGPU (Mesa 26.2.4), Linux (CachyOS, kernel 7.2.9), Release build with CI's flags, with GuestContext::floatMode set to the titles' mode (FLOAT_MODE 0xc0, DX10_CLAMP 1, IEEE 0), to no mode, to DX10_CLAMP 0, to IEEE 1 and to the oracle's former implicit mode (0xf0, IEEE 1), and were compared bit for bit with the oracle in the same mode. Per-cell data, driver and the replay harness with its REPLAY_MODE setting: https://github.com/SleglJan/AnyPS5/tree/audit/oracle-2026-10-08/oracle-audit/check-1866-1867.

• No mode gives the titles' mode in every cell on both hosts. A cell whose oracle value does not depend on a field does not move on the hosts when that field changes either (29,200 host cells for DX10_CLAMP, 31,910 for IEEE_MODE).
• DX10_CLAMP (feat(shader): follow DX10_CLAMP in the clamp modifier #1866). 354 cells in 11 kernels change with the field on the oracle. 58 are in three of the four kernels with a clamped f64 result (DivisionOutputModifiers 14, DivisionResultModifiers 39, F64Transcendental 5); those four (F64Conversions too, with no sensitive cell) throw with DX10_CLAMP=0, as feat(shader): follow DX10_CLAMP in the clamp modifier #1866 says. Of the other 296, RADV matches the hardware in 292 and returns the sNaN quieted in 4 (Float16TernaryClamp); NVIDIA matches in 204 and returns an all-ones NaN (0x7fffffff, 0x7fff in f16) in 92. No other difference.
• IEEE_MODE (this PR). 989 cells in 22 kernels change with the field. The four re-pinned tests match the hardware bit for bit with IEEE 1, and with IEEE 0 in all but Division's 10 v_div_fmas_f32 cells, where the driver's fma returns the input sNaN quieted (0x7fc00001 for 0x7f800001; ternary arithmetic whose NaN bits the description leaves to the driver): DivFixupF16 21 cells, SdwaLdexpF16 36, Vop1FloatUnary 118, Division 21. With IEEE 1, 9 cells differ, Float16TernaryClamp's v_min3/max3/med3_f16 clamp with a NaN operand (the f16 min/max rule the TechnicalDebt entry lists as single-mode), and 40 more are NaN-payload differences on RADV (F32OutputModifier 21, Vop3CubeClamp 12, Float16Misc 4, Vop3PackLdexp 3), 53 on NVIDIA. In the titles' mode 39 cells differ: F64Arithmetic 14 (v_min_f64/v_max_f64 return the quieted sNaN where the hardware returns the other operand) and F64Rounding 25 (v_cvt_f32_f64 mul:2, the output modifier the hardware applies to the f32 result, as noted on fix(shader): apply the output modifier only in the float modes where the hardware does #1851). NaN-payload differences: 94 on RADV, 119 on NVIDIA (F32OutputModifier 21 and Float16Misc 4 more); 71 of the 94 are in the f64 tests (F64Arithmetic 21, F64Division 15, F64NanConversions 13, F64DivisionModifiers 10, F64Transcendental 8, F64Rounding 3, F64Conversions 1), where the tables hold the quieted NaN of IEEE 1 and both hosts produce it in every mode: 43 from the f64 emitter's unconditional quieting (Quiet in SpirvAluEmitterFloat64.cpp, behind add, mul, fma, ldexp, min, max and the unary ops), 11 from v_div_fixup_f64's in Float64Instructions.cpp, 17 in the f32/f64 conversions. Those tables were captured in the same implicit mode as the four this PR pins; as a follow-up, the same floatMode line would make them declare it.

The 28-kernel run predates the #1878 merge; #1878 changes only v_fma_mix_f32, and MixPrecision rerun after it gives the same bits in every cell. Hardware: Ryzen 7950X3D iGPU (gfx1036, RDNA2) through tools/hw-oracle under hsa-rocr 7.2.4; not the PS5's gfx1013.

AI-assisted: yes (Claude Code).