Skip to content

fix(libkernel): stop losing and deadlocking sceKernelRaiseException deliveries on Windows - #1421

Merged
adriadam10 merged 7 commits into
boykopovar:mainfrom
popcoder565:fix/kernel-raise-exception-ntcontinue
Oct 9, 2026
Merged

adriadam10 merged 7 commits into
boykopovar:mainfrom
popcoder565:fix/kernel-raise-exception-ntcontinue

Conversation

@popcoder565

@popcoder565 popcoder565 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

What

On Windows, sceKernelRaiseException could lose a delivery: it returned 0, but the handler never ran on the target thread. guest_raise_exception hits this in its busy-thread phase in about one run in five on main (9619edf, after #1375) and then hangs until the 30 s ctest timeout. Three paths lost deliveries, and a fourth deadlocked them.

The handler's own return went through NtContinue. A raise suspends the target, points its context at Aps5RedirectedEntryStub, and the stub returns through RtlRestoreContext. On Windows 10 (19045), RtlRestoreContext restores in user mode, with iretq, only when ContextFlags without CONTEXT_XSTATE is exactly CONTEXT_FULL | CONTEXT_SEGMENTS (0x10000F, read from the ntdll disassembly); otherwise it calls NtContinue. The context was read with CONTEXT_FULL | CONTEXT_FLOATING_POINT (0x10000B), so every handler returned through NtContinue. When the next raise suspended the thread inside that system call, SetThreadContext changed the saved user context, and NtContinue then replaced it with the context it was restoring, so the thread went back to its loop without running the handler.

  • The context is now read with CONTEXT_FULL | CONTEXT_SEGMENTS, so the handler returns through the user-mode restore, which builds its iretq frame below the saved context and never enters the kernel.

Other code returns through NtContinue too. An APC or an exception handler that continues execution resumes the thread through NtContinue or NtContinueEx.

  • A raise that finds the thread inside one of those stubs resumes it and suspends it again instead of redirecting a context that is about to be replaced.

The APC path was chosen before the thread had stopped. SuspendThread returns before the thread is suspended; GetThreadContext waits until it is. waitCount was read right after SuspendThread, so a thread leaving its wait could still decrement it and drain its APCs with NtTestAlert before QueueUserAPC queued the delivery, which then waited for an alertable wait that never came.

  • The context is now read first, so waitCount is read from a stopped thread.

A raise could interrupt the target inside winpthreads. MinGW's thread_local goes through emutls and winpthreads' pthread_getspecific, which holds a per-thread spinlock. A handler that reads a thread_local, such as scePthreadSelf or the TimedWait wait state, on a thread interrupted while it held that lock spun on it forever.

  • A raise that finds the target inside libwinpthread-1.dll resumes it and suspends it again, as one inside NtContinue does.

Both retries give up after 1 s. A target that stays inside those NtContinue stubs or winpthreads for that long makes the raise throw, instead of spinning forever. In guest_raise_exception, the longest retry took 7.4 ms.

TechnicalDebt: a new entry records two things: a handler can still reenter other host code the thread was interrupted in, such as the C runtime's heap, and the raise gives up after 1 s. The YMM entry is gone since #1478, which saves the upper halves; with main merged in, its vectors phase passes with this PR's CONTEXT_FULL | CONTEXT_SEGMENTS read.

How this relates to #1562

Since 07610212, #1562 no longer changes Exception.cpp. It now moves libkernel's handler-visible state (the current thread, pthread keys and the TimedWait state) to fiber-local storage, and makes contended mutex and rwlock waits interruptible on Windows. The two PRs complement each other: with #1562, libkernel calls from a handler no longer take winpthreads' TLS lock, and this PR's retry also covers a thread_local in libc or the other prx modules.

In this probe (not part of this PR), a worker loops on libc's strerror, whose buffer is a MinGW thread_local, and the handler calls strerror too:

build runs in which all 500 raises were handled
main 0 of 5 (the handler hung in rounds 1–98)
#1562 at de12706a 0 of 5 (rounds 2–70)
this PR 5 of 5

Tested

Windows 10 (19045), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release, Intel Core i7-12700H.

  • Before: guest_raise_exception failed 2 of 5 runs on its own (ctest timeout), and a loop over the test binary hung on runs 12, 3 and 3. gdb on a hung process showed the main thread still waiting for the 9th delivery to the busy thread and the busy thread back in its loop. A 1 ms sleep takes 15.6 ms there, so ExpectDelivery's 5000 attempts outlast the 30 s timeout instead of aborting.
  • New phase in guest_raise_exception: a thread raises SEH exceptions in a loop and a vectored handler continues each one, so the thread spends much of its time in NtContinue. With the first change only, it hung in 4 of 4 runs; with the retry it passes.
  • The leaving-thread phase hung in 10 of one series of 30 sequential runs with the first two changes. I could not reproduce that on demand afterwards: 84 sequential and 96 parallel runs of the build with and without the third change all passed. The third change follows from SuspendThread being asynchronous, not from a measured before and after.
  • A later attempt to measure the third change also showed no difference. I used a stress tool that is not in this PR:
    • A thread waits on a semaphore and then spins without entering an alertable wait.
    • The main thread signals it, raises after a delay of 0 to 50 µs, and checks that the handler runs while the thread spins.
    • 3 × 20,000 rounds delivered every raise while the thread spun, both with and without the third change.
    • So there is still no measured before and after. The change stays because SuspendThread is asynchronous.
  • New phase: a thread calls scePthreadSelf in a loop while it is raised at, and the handler checks it sees its own thread. Without the winpthreads retry it hung in every run (3 of 3 with the final 100 rounds, 10 of 10 in an earlier version); with it, it passes.
  • After all four: ctest -R ^guest_raise_exception$ --repeat until-fail:10 passed all 10 runs (at most 8.7 s each, the 30 s timeout unchanged), and 20 of 20 runs passed with a 200-round version of the new phase.
  • Full ctest, two rounds: everything passes except, once, agc_driver_dcc_keys_first_write. That test fails without this change too (AGC: on Windows (NVIDIA) a new host import's pages are marked written after ImportWatched's window, so write-backs drop GPU results #1434) and never raises an exception.
  • Retry bound: a new phase keeps a thread inside NtContinue and checks that the raise throws after 1 s without running the handler. The thread continues a context whose Rip is NtContinue and whose Rcx points at that same context, so each call restores itself.
    • Without the bound, 2 of 2 runs hung in that phase until the ctest timeout.
    • With it, guest_raise_exception passed 10 of 10 runs on the branch, and 10 of 10 merged onto main 63c57d2.
  • Retry durations, traced:
    • Over 5 runs, about 60 raises per run retried. The longest took 666 retries in 7.4 ms, and the average was 0.9 ms.
    • With four copies of the test running at once, the longest took 5 ms.
  • Full ctest merged onto main 63c57d2: 452 of 454 pass. The failures are agc_driver_dcc_keys_first_write and agc_driver_srgb8_color_target, the NVIDIA import flakes in AGC: on Windows (NVIDIA) a new host import's pages are marked written after ImportWatched's window, so write-backs drop GPU results #1434.

No title tested.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

A raised exception redirects the target thread to a stub that runs the handler and returns through RtlRestoreContext. On Windows 10 (19045) RtlRestoreContext restores in user mode only when the context flags are exactly CONTEXT_FULL | CONTEXT_SEGMENTS (CONTEXT_XSTATE aside); for the CONTEXT_FULL | CONTEXT_FLOATING_POINT context it was given, it calls NtContinue. When the next raise suspended the thread inside that system call, NtContinue then replaced the redirected context with the one it was restoring, so the handler never ran although sceKernelRaiseException had returned 0. guest_raise_exception lost such a delivery to its busy thread in about one run in five and hung until the ctest timeout.

The context is now read with CONTEXT_SEGMENTS, so the handler returns through the user-mode restore, and a raise that finds the thread inside NtContinue or NtContinueEx (an APC or an exception handler returning) resumes it and suspends it again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
popcoder565 and others added 2 commits October 8, 2026 00:53
…EH exceptions

The new phase's vectored handler continues every exception the thread raises, so the thread spends much of its time in NtContinue, where a redirect is lost unless the raise retries. Without the retry the phase hangs in every run; with it the test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elivery

SuspendThread returns before the thread has stopped, and GetThreadContext waits until it has. waitCount was read right after SuspendThread, so a thread that was leaving its wait could still decrement it and drain its APCs with NtTestAlert before QueueUserAPC queued the delivery, which then waited for an alertable wait that never came. The context is now read first, so waitCount is read from a stopped thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hreads

MinGW's thread_local goes through emutls and winpthreads' pthread_getspecific, which holds a per-thread spinlock. A raise that redirected a thread holding it ran the handler on that same thread, and a handler reading any thread_local, scePthreadSelf for one, spun on the lock forever. A raise that finds the target inside libwinpthread-1.dll now resumes it and suspends it again, as one inside NtContinue does.

guest_raise_exception's new phase raises at a thread that calls scePthreadSelf in a loop and checks that the handler sees its own thread. It hung in every run without the retry. TechnicalDebt records that a handler can still reenter other host code the thread was interrupted in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@popcoder565 popcoder565 changed the title fix(libkernel): keep sceKernelRaiseException redirects out of NtContinue fix(libkernel): stop losing and deadlocking sceKernelRaiseException deliveries on Windows Oct 8, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the diff, the guest_raise_exception registration (Windows only) and CI on the current head (windows, linux, conventions, report green; Linux ctest 457/457 locally, the test itself doesn't run there). Two things before merging:

  • The retry in RaiseOn (Exception.cpp:219-243) has no bound: a thread that stays inside NtContinue/NtContinueEx or winpthreads makes the raise spin on SwitchToThread forever, a hang instead of an error. Please bound it (a retry count or a deadline) and throw when it runs out, with the test covering that path if it can be forced.
  • #1562 fixes the same MinGW TLS deadlock in a different way and conflicts with this one. Please say in the description how the two relate: whether this one is still needed with #1562, or #1562 with this one. If they're alternatives, we'll take only one.

Also, the third change (reading the context before waitCount) has no before/after measurement; if you have one, add it to Tested.

…inpthreads

The retry that waits for a target to leave NtContinue, NtContinueEx or
libwinpthread-1.dll had no bound, so a target that stayed there made
sceKernelRaiseException spin forever. It now throws after 1 s. A new
guest_raise_exception phase keeps a thread inside NtContinue by
continuing a context that calls NtContinue again, and checks that the
raise throws and that the handler did not run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@popcoder565

Copy link
Copy Markdown
Contributor Author

@adriadam10 thanks for the review. I pushed 24dc98d and updated the description.

  • Retry bound: the retry now gives up after 1 s and throws.
    • A new guest_raise_exception phase forces this path deterministically. A thread keeps continuing a context that calls NtContinue again, so it never leaves the stub. The raise must throw without running the handler.
    • Without the bound, that phase hangs.
    • Traced retries in the test took at most 7.4 ms.
  • fix(libkernel): keep Unity GC delivery out of MinGW TLS waits #1562: the two aren't alternatives, and there is a new section in the description.
  • Third change: I tried to measure it with a stress tool (3 × 20,000 rounds) and saw no difference with or without it. Tested now says so.

🤖 Addressed by Claude Code

@popcoder565

Copy link
Copy Markdown
Contributor Author

The linux failure isn't from this PR. It stops at the test-registration step before building: guest_formatting_tests is run by ctest now; remove it from KNOWN_UNRUN. #1435 started running guest_formatting on Linux without updating tools/check_test_registration.py, so main fails the same way. #1603 fixes it, and I'll re-run CI here once that lands.

🤖 Addressed by Claude Code

@popcoder565

Copy link
Copy Markdown
Contributor Author

I updated the comparison with #1562 in the description. Its current head de12706a builds and passes this PR's phases, apart from the NtContinue loop phase, which checks this PR's bound. One case still hangs with it: a handler that uses another module's thread_local (here libc's strerror buffer) while the target was stopped inside winpthreads. That case hangs in 5 of 5 runs with #1562 and on main, and passes 5 of 5 with this PR. So this PR's own parts are now the winpthreads retry and the 1 s NtContinue bound. I can rebase them onto #1562 if you prefer its delivery.

🤖 Addressed by Claude Code

…ption-ntcontinue

Keeps this branch's CONTEXT_FULL | CONTEXT_SEGMENTS read, context-first
order and bounded retry in RaiseOn, and adds boykopovar#1478's ymm save fields to
both deliveries. The test keeps both phase sets; the vectors phase runs
after the continuing, selfing and looping phases and counts their
deliveries. The TechnicalDebt entry about ymm upper halves is dropped,
since boykopovar#1478 preserves them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 9, 2026
…ption-ntcontinue

Keeps both new TechnicalDebt entries: this branch's note that a raise
can still reenter host code it interrupted, and boykopovar#2107's sigaltstack note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the new bound (Exception.cpp:230-255, 1 s deadline that throws), the guest_raise_exception phases covering it (NtContinue loop, scePthreadSelf in winpthreads, SEH continue) and the TechnicalDebt line; merges cleanly with current main and the Linux build and ctest are green (the test itself is Windows-only, so the Windows behaviour rests on your runs). Merging.

@adriadam10
adriadam10 merged commit 76bff21 into boykopovar:main Oct 9, 2026
6 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…exception-ntcontinue

fix(libkernel): stop losing and deadlocking sceKernelRaiseException deliveries on Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants