Skip to content

fix(libs): flush pending GPU results before file writes - #6234

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
Ondra97:codex/fix-file-write-gpu-coherence
Oct 18, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
Ondra97:codex/fix-file-write-gpu-coherence

Conversation

@Ondra97

@Ondra97 Ondra97 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

Explicit-size file writes can return the full byte count while storing stale guest memory after a GPU storage-image write, even after a draining suspend and device-idle wait. I make the file-write paths prepare their source ranges before host I/O.

A small libc callback bridge invokes the existing AGC read guard and range-specific FlushGpuWrites without reversing the libc/libkernel dependency direction. It covers kernel/POSIX scalar, positioned and vectored writes, AIO writes and libc fwrite. Preparation precedes positioned-transfer, shared-memory and native FILE locks. fwrite checks multiplication and address overflow before preparing its range.

The GPU regression checks actual file contents through 12 API paths, interior source addresses, multiple iovecs, positioned offsets and element counts. Controls cover empty/invalid requests, an unrelated CPU source, retention of another pending image, guard refusal before file mutation, and ordinary writes after shutdown and hook reset.

The rebase preserves every added and removed line of the original patch and retains main's queue-zero late-interrupt test linkage. The merged append/vector/descriptor-lifetime changes remain intact. The current #6287 append-stream implementation changes stream opening rather than this source-preparation path; no prerequisite is required.

Direct guest CPU loads, other host readers, and freeing/reusing pending storage-image memory remain covered by the existing technical debt. No retail title is claimed as fixed.

Tested

Base: c0a7468f6f666a46b1c3e2fb14bb651fed05afc5. Head: 9b89c92a2e4a26b01cf5c21d4a071deb13e36da7. Release, BUILD_TESTING=ON.

Windows (native host) — Windows 11 Pro 10.0.26300, AMD Ryzen 9 7900X, 32 GB RAM, NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 617.42 / Windows driver 32.0.16.1742, WinLibs MinGW-w64 GCC 15.2.0 r7.

  • Full build, all libraries and NID patching passed: cmake --build build-pr6234 --parallel 4.
  • 34 selected filesystem, scalar/vector/positioned I/O, AIO, stream, descriptor-lock, write-watch and GPU coherence tests passed; 0 failures, 0 CTest skips, 7.95 s total, longest 0.86 s.
  • All 12 API paths in agc_driver_storage_file_write reported zero incorrect bytes. GPU logs confirm the NVIDIA device. GPU runs used ANYPS5_REQUIRE_VULKAN=1, ANYPS5_GPU=NVIDIA GeForce RTX 4070 Ti SUPER and ANYPS5_NO_SHADER_CACHE=1; the regression also sets APS5_SUSPEND_DRAIN=1 and has a 30 s timeout. Reproduce it with ctest --test-dir build-pr6234 -V -R '^agc_driver_storage_file_write$' under those settings.

Before / after, Windows (native host) — one run of the same scalar-write case, on the same machine and configuration, after a real PM4 compute dispatch and draining suspend. The source begins 13 bytes inside a 131,072-byte surface; the write transfers 131,035 bytes.

Metric Before (base) After (fix)
Incorrect bytes in the first successful sceKernelWrite 85,957 / 131,035 0 / 131,035

For the negative control, all seven production files were restored byte-for-byte from the base and the regression selected only its sceKernelWrite case. The control failed on the actual byte comparison. All seven production files and the complete 12-API fixture were then restored and SHA-256 checked before the final full build and passing suite. This demonstrates file-content correctness, not a performance or game-compatibility improvement.

Linux (VirtualBox) — Ubuntu 24.04.5 LTS, kernel 7.0.0-38-generic, GCC 13.3.0, VirtualBox 7.2.20r175154 on the Windows host above, 4 vCPUs and 8 GiB RAM. Graphics: not exercised.

  • Full Release build and NID patching passed: cmake --build build --parallel 3.
  • 13 registered CPU/runtime tests passed; 0 failures, 0 CTest skips, 0.67 s total. They cover scalar/vector file I/O, positioned-transfer races, AIO, streams, append writes, filesystem/timestamps, write-watch coverage and loading patched libraries. The seven additional AIO/descriptor-lock modes in the Windows selection are registered only on Windows.
  • Both agc_driver_storage_file_write_tests and agc_driver_queue_zero_late_interrupt_tests compiled; neither was executed in VirtualBox. SHA-256 matched all 16,037 transferred source/dependency files before the build.

Linux graphics/Vulkan tests: NOT RUN — intentionally excluded from VirtualBox validation. Native/bare-metal Linux was not tested. Native Linux GPU validation: not tested. No retail title was run.

Conventions, main-sync, whitespace and test registration checks passed; all 954 test executables have CTest registrations. Full hosted Windows/Linux PR builds require collaborator dispatch under CONTRIBUTING.md; no such run was triggered here.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 617.42 / Windows 32.0.16.1742, Windows 11 Pro 10.0.26300
  • AI-assisted: yes — Codex. I am responsible for the change.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6326 core/libs/prx/libSceAgcDriver/CMakeLists.txt L1862
PR 6341 core/libs/prx/libSceAgcDriver/CMakeLists.txt L1862
PR 6926 core/libs/prx/libc/include/GuestArena.hpp L58
core/libs/prx/libc/src/GuestArena.cpp L33, L538-539
docs/dev/debt/libSceAgcDriver.md L87
PR 7432 docs/dev/debt/libSceAgcDriver.md L87-89
PR 7505 docs/dev/debt/libSceAgcDriver.md L87-88
PR 7604 docs/dev/debt/libSceAgcDriver.md L87-88

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1341 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1368 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1835 core/libs/prx/libc/src/GuestArena.cpp
PR 2434 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 4448 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 4650 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 4891 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5021 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
core/libs/prx/libc/src/GuestArena.cpp
core/libs/prx/libkernel/File/src/Open.cpp
core/libs/prx/libkernel/File/src/Stdio.cpp
PR 5058 docs/dev/debt/libSceAgcDriver.md
PR 5371 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5426 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5431 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 5925 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6044 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6185 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6186 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6188 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6192 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6194 docs/dev/debt/libSceAgcDriver.md
PR 6218 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6223 docs/dev/debt/libSceAgcDriver.md
PR 6273 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6280 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6300 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6360 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6412 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6447 core/libs/prx/libc/src/GuestArena.cpp
PR 6454 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6472 docs/dev/debt/libSceAgcDriver.md
PR 6481 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6486 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6501 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6529 docs/dev/debt/libSceAgcDriver.md
PR 6535 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6548 docs/dev/debt/libSceAgcDriver.md
PR 6549 docs/dev/debt/libSceAgcDriver.md
PR 6563 docs/dev/debt/libSceAgcDriver.md
PR 6606 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6615 core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6683 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6701 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
PR 6747 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6786 docs/dev/debt/libSceAgcDriver.md
PR 6788 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6856 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6867 docs/dev/debt/libSceAgcDriver.md
PR 6868 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6879 core/libs/prx/libc/src/GuestArena.cpp
core/libs/prx/libkernel/File/src/Stdio.cpp
PR 6936 core/libs/prx/libkernel/File/src/Open.cpp
core/libs/prx/libkernel/File/src/Stdio.cpp
PR 6937 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6941 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 6946 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6947 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6963 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6977 core/libs/prx/libkernel/File/src/Stdio.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7002 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 7008 docs/dev/debt/libSceAgcDriver.md
PR 7081 core/libs/prx/libc/src/FileAndHeap.cpp
PR 7085 core/libs/prx/libSceAgcDriver/CMakeLists.txt
docs/dev/debt/libSceAgcDriver.md
PR 7201 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
core/libs/prx/libc/src/FileAndHeap.cpp
core/libs/prx/libc/src/GuestArena.cpp
core/libs/prx/libkernel/File/src/Open.cpp
core/libs/prx/libkernel/File/src/Stdio.cpp
PR 7223 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7250 docs/dev/debt/libSceAgcDriver.md
PR 7256 docs/dev/debt/libSceAgcDriver.md
PR 7258 docs/dev/debt/libSceAgcDriver.md
PR 7332 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
core/libs/prx/libc/src/FileAndHeap.cpp
core/libs/prx/libkernel/File/src/Open.cpp
core/libs/prx/libkernel/File/src/Stdio.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7390 docs/dev/debt/libSceAgcDriver.md
PR 7615 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libc/src/GuestArena.cpp
PR 7642 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7644 core/libs/prx/libSceAgcDriver/CMakeLists.txt

EagleBearerYS added a commit to EagleBearerYS/AnyPS5 that referenced this pull request Oct 15, 2026
…w-snapshot one

The entry sat right below "Pending storage-image results", which boykopovar#6234
rewrites, so the two changes conflicted. Above "Draw input snapshots",
two entries up, it merges cleanly with boykopovar#6234 and with main.
This was referenced Oct 15, 2026
EagleBearerYS added a commit to EagleBearerYS/AnyPS5 that referenced this pull request Oct 16, 2026
GuestArenaSetHostWriteObserver_nid_postfix becomes GuestArenaSetHostWriteHook_nid_no_patch, taking the address as std::uintptr_t, and the driver's GuestMemory::FlushBeforeHostWrite becomes PrepareHostWrite, so they read as one pair with the host read hook boykopovar#6234 adds (GuestArenaSetHostReadHook_nid_no_patch and PrepareHostRead). The setter is a helper that must not become a NID, so it takes the _nid_no_patch suffix. Behaviour is unchanged.
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 17, 2026
@Ondra97
Ondra97 force-pushed the codex/fix-file-write-gpu-coherence branch from 0f5651f to 9b89c92 Compare October 17, 2026 21:59
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 17, 2026
@adriadam10 adriadam10 self-assigned this Oct 18, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@adriadam10
adriadam10 force-pushed the codex/fix-file-write-gpu-coherence branch from 9b89c92 to a64d038 Compare October 18, 2026 23:00
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@adriadam10
adriadam10 merged commit a4a1021 into boykopovar:main Oct 18, 2026
17 checks passed
@Ondra97
Ondra97 deleted the codex/fix-file-write-gpu-coherence branch October 19, 2026 04:24
@mugurc mugurc mentioned this pull request Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants