Skip to content

fix(libc): avoid premature scaled hyperbolic overflow - #8023

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
Ondra97:codex/fix-scaled-hyperbolic
Oct 20, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
Ondra97:codex/fix-scaled-hyperbolic

Conversation

@Ondra97

@Ondra97 Ondra97 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

What

_FSinh(x, y) and _FCosh(x, y) currently evaluate the hyperbolic function in binary32 before applying y. This can overflow even when the scaled result fits, or produce NaN for a finite x and a zero scale.

Evaluate the finite scaled product in binary64 before the final binary32 rounding. Preserve the PS5 runtime's NaN preference and payloads, signed zeros, infinite operands, sign application and range-error branches. Extend the existing guest_math fixture and record the remaining console-precision limits in technical debt.

Metric Before (base) After (fix)
Either helper at (90.f, 1e-20f) +inf 6.10201631e18 (0x5ea95d6f)
Either helper at (90.f, 0.f) NaN +0
Mismatches in the same 24 native calls 16 0

The table compares correctness on the same Windows host. The probe applies 12 float input pairs to both exports and compares with a long double product rounded to float: (0,1), (1,1), (-1,2), (20,.25), (90,1e-20), (-90,1e-20), (90,-1e-20), (90,+0), (-90,+0), (90,-0), (100,1e-40), (100,-1e-40).

Static disassembly of PRAGMATA's bundled libc established the special-value behavior and the scaled-exponential paths: _FSinh at original VA 0x80ac0, _FCosh at 0x80680, and _FExp at 0x807c0. I byte-matched the analyzed bodies to the original module. The 7.7f and 8.675f thresholds preserve the two helpers' original range-error branches. Finite results use host math and are not claimed to be bit-identical to the console; title impact remains unverified.

Tested

Windows 11 Pro (build 26300), MinGW-w64 GCC 15.2.0, Ninja Release, BUILD_TESTING=ON, SPIRV-Tools off. Hardware: AMD Ryzen 9 7900X, approximately 32 GB RAM, NVIDIA RTX 4070 Ti SUPER. These checks use the CPU.

Tested base: 8444508241a9e4a6f5d5604d007451ce62d39cc0; tested head: ce81a77d1a21ff534a0ee9db24fecb2805b6c0ed.

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=OFF
cmake --build build --parallel 6
ctest --test-dir build -V -R '^(guest_math|guest_math_exact|guest_floating_point|guest_float_builtins|patched_libraries)$' --timeout 30
  • Full Windows build passed.
  • 5/5 focused CTests passed, zero failed or skipped; guest_math executes 1,835 scaled-hyperbolic checks, including adjacent floats at overflow/underflow boundaries, both signs, NaN payloads, error state, and overflow under all four rounding modes.
  • The same regression failed on the base implementation with got 7f800000, expected 5ea95d6f.
  • All 352 patched libraries loaded successfully. The 24-call probe also passed through the patched NID exports.
  • Linux, actual title execution and the complete CTest suite were not run.

The overlap tool completed with warnings for conflicting older PRs. I manually checked the shared-file patches: they change unrelated functions/debt entries, or only move GuestMath.cpp (#6044); none implements these two helpers' correction.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestMath.cpp
PR 6158 core/libs/prx/libc/src/Math.cpp
core/libs/tests/GuestMath.cpp
PR 6226 docs/dev/debt/libc.md
PR 6469 docs/dev/debt/libc.md
PR 6819 docs/dev/debt/libc.md
PR 7081 docs/dev/debt/libc.md
PR 7201 core/libs/prx/libc/src/Math.cpp
PR 7473 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md

@wachi35676

Copy link
Copy Markdown
Contributor

Accuracy check against a high-precision reference, plus Linux (the description has Windows only).

Accuracy: 143,416 float pairs (x, y) went through the real _FSinh_nid_postfix and _FCosh_nid_postfix exports of the built libc.prx.

  • The pairs were:
    • |x| uniform in [0, 200] with y log-uniform over all finite floats, subnormals included;
    • random finite bit patterns with |x| <= 200;
    • y chosen to put the result near FLT_MAX, FLT_MIN and the smallest subnormal.
  • Each output was compared with sinh(x)*y / cosh(x)*y computed in mpmath at 160 bits and correctly rounded to binary32 (round-to-nearest-even, subnormals, overflow to inf).
  • Windows, Windows 11 Home 26300, Ryzen 5 9600X, WinLibs GCC 15.2.0, Release, BUILD_TESTING=ON, main 9127ea3 with and without this PR:
exact 1 ulp >1000 ulp (inf where the result is finite)
main _FSinh 118,908 9,031 15,477
main _FCosh 118,571 9,368 15,477
this PR _FSinh 143,416 0 0
this PR _FCosh 143,416 0 0

With this PR every output is the correctly rounded value. Example on main: _FSinh(89.4219f, 0.000538265f) returns +inf; the reference is 1.84243e35 (0x7a0def6e), which this PR returns. I can share the probe and comparator if useful.

Tests:

  • Linux (WSL Ubuntu 24.04, GCC 13.3, Release, BUILD_TESTING=ON): guest_math passes 3/3. With Math.cpp from main it aborts at _FSinh(0x1.68p+6, 0x1.79ca1p-67): got 7f800000, expected 5ea95d6f.
  • Windows: guest_math passes 3/3, and the reverted control 0/3.

Prior art: #2050 (closed, not merged) proposed the same binary64-before-narrowing approach from the MSVC STL sources.

AI-assisted: yes (Claude Code); results above were run on this machine.

@oneandonlydean oneandonlydean self-assigned this Oct 20, 2026
@oneandonlydean
oneandonlydean merged commit 542c65d into boykopovar:main Oct 20, 2026
18 checks passed
iamthehimansh added a commit to iamthehimansh/AnyPS5 that referenced this pull request Oct 20, 2026
macOS: guest_math's scaled-hyperbolic special checks mask FE_DENORMALOPERAND out of fetestexcept. Darwin's FE_ALL_EXCEPT includes the x86 denormal-operand flag (glibc's does not), so the new boykopovar#8023 max-float/denorm-min case reported an extra flag even though the result, errno and the standard exception flags all matched. Linux/Windows keep the upstream check.
@Ondra97
Ondra97 deleted the codex/fix-scaled-hyperbolic branch October 22, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants