Linux: closing the window crashes in AJM/APR calls made after static teardown #1925

Summary

Closing the game window calls LibcRequestExit_nid_postfix(0) (libSceVideoOut/src/VideoOutDriver.cpp:496). That starts a thread that runs the shutdown callbacks and then std::exit (libc/src/Process.cpp:112-116). std::exit destroys every host library's static objects, but the title's own threads keep running and keep calling those libraries. Two libraries keep their state in namespace-scope statics that guest threads reach:

• libSceAjm.native/src/Ajm.cpp:143-149: g_instances (the decoder instances), g_batches and their mutexes.
• libkernel/Apr/src/Apr.cpp:42-50: g_files, g_idsByPath, g_directories and the AMM state (_amm()).

After they are destroyed, an audio job thread looks up a freed Instance and gets a garbage codec. sceAjmBatchWait no longer finds the batch it just started and returns SCE_AJM_ERROR_INVALID_BATCH (0x80930004). An APR read resolves its file id through the freed vector.

#1197 fixed the same race for NGS2, and the same fix works here.

Impact on Demon's Souls (PPSA01341 v1.07)

Closing the window at the attract movie ended in a crash on every one of 15 runs on our test stack once #1071 had removed the driver-side fault. The faults:

• terminate ... AJM: decoding codec 4026830656 is not implemented (and codec 27): a freed Instance.
• *** NCA DecoderManager: sceAjmBatchWait() failed with error code 0x80930004 which is fatal ..., halting now ***, then SIGSEGV in a job-worker thread.
• APR: cannot open <garbage>: a freed g_files entry.
• double free or corruption (out).

#1658 runs the guest exit handlers before the host ones. With it, 3 of 3 closes still failed (runs 317, 319, 321), because the host statics are still destroyed while the title's threads run.

Each crash also writes a core of about 1 GB, and these shutdown cores bury the real crashes in coredumpctl.

Fix

Branch: https://github.com/fisherjoey/AnyPS5/tree/fix/exit-teardown-order

AJM's instance and batch tables and APR's file, directory and AMM state are allocated once and never destroyed, as #1197 did for NGS2. New test guest_ajm_teardown (UNIX): an atexit handler registered before libSceAjm.native is dlopened runs after the library's statics are gone, and it uses an instance created before exit. It fails on main and passes with the change. On our Demon's Souls stack with the commit on top, 5 of 5 window closes exit 0 with no core.

Other libkernel state is in namespace-scope statics too (Equeue.cpp g_equeues, SyncOnAddress.cpp g_waiters, DirectMemory.cpp), and a title thread could hit it the same way. A general option would be to end the process with _Exit after the guest exit handlers and the shutdown callbacks, so host statics are never torn down while guest threads run. That changes shutdown for everything, so it is not in this branch. I can open a discussion if you'd prefer that approach.

Found on Linux 7.2 (Nobara 44), GCC 16, RTX 5070 Ti (595.91.07).

AI-assisted: yes.