Skip to content

fix(agc): drain library prebuilds before device replacement - #2

Closed
MarcZX10 wants to merge 5761 commits into
mainfrom
codex/agc-library-prebuild-replacement-race
Closed

MarcZX10 wants to merge 5761 commits into
mainfrom
codex/agc-library-prebuild-replacement-race

Conversation

@MarcZX10

Copy link
Copy Markdown
Owner

What

Closes boykopovar#6805. Device replacement now drains queued registration library prebuilds before clearing the old device's pipeline caches. Pipeline-library clearing also waits for in-flight library part builds, including the interval where vkCreateGraphicsPipelines runs without the library-store mutex, before erasing the per-device maps. A deterministic mock test blocks pipeline creation while cache clearing runs, then checks that clearing waits and releases all pipeline objects.

Open boykopovar#5925 and boykopovar#6641 touch nearby lifecycle code; neither drains registration library prebuilds during device replacement.

Tested

  • Linux, Debian 12, GCC 12.2: agc_driver_graphics_tests built successfully. This host lacks the PS5 CPU feature set, so I temporarily bypassed the test binary's startup CPU guard to run the mock-only test; the guard change was restored and is not in this PR.
  • ctest --test-dir /tmp/build-6805 -R '^agc_pipeline_library_clear_waits_for_build$' --output-on-failure: passed, 1/1. The run used the same temporary local CPU-guard bypass. No GPU was used; the regression test uses mocked Vulkan calls.
  • git diff --check and python3 tools/check_conventions.py --base upstream/main: passed.

Checklist

fdfawef324 and others added 30 commits October 15, 2026 07:39
Use checked host stdio for BMP output and stop requesting dumps after the first write failure. Keep pixel validation, allocation and non-I/O failures on the existing fatal path.

Co-Authored-By: Codex <noreply@openai.com>
…-misses

feat(agc): trace draw-key cache misses
…without a format

A sampler voice set up with an all-zero format has no channels. Adding no
waveform blocks to it, or setting a matrix with no levels, threw
std::invalid_argument, which ends the title.

Accept both on such a voice: the empty add only applies RESET and the
CONTINUE state, and the empty matrix clears that matrix, which rendering
already treats as unset. Blocks with data, APPEND, and an empty matrix on
a voice with a format are still refused.

Steins;Gate Re:Boot (PPSA36591) clears a voice this way when a button is
pressed at its title screen.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ception

Co-Authored-By: Claude <noreply@anthropic.com>
…location-retirement

fix(agc): retire fully released native depth owners
…e surfaces lock

The bind-time HTILE read covered one word per 8x8 tile from DB_HTILE_DATA_BASE, but main models the
HTILE as the pipe-aligned SW_64KB_Z_X metadata (HtileMetadataBytes, the extent HtileFillCovers takes):
at 1920x1080, 1856 of the 32400 tile words lie past the first tiles*4 bytes, so they kept ZMask 0
after a store-back. The read and the store-back now cover the whole metadata and run before the
surfaces mutex is taken, since both wait for recorded work over the range. The tests' HTILE arrays
hold the whole metadata.

PPSA25344 (Unreal Engine 5) decodes its depth through HTILE in its Nanite depth export after the
depth prepass and takes ZMask 0 as a tile holding the clear value; the new
agc_driver_depth_fast_clear_submit case checks that a draw after a fast clear leaves every word
expanded.
…its words are uniform

A fill or clear the driver recognizes over an HTILE drops the range's last-read generation, so the
next bind reads the words and stores them back even when the write tracker did not see the fill's
stores. A depth view of slice 1 or later reads the HTILE at its own address
(DB_HTILE_DATA_BASE + slice * HtileMetadataBytes); the HTILE word layout is only modelled for slice 0,
so such a slice takes the clear and the store-back only when all its words are equal and keeps its
fill-time fast clears otherwise. DepthTarget::slice carries DB_DEPTH_VIEW SLICE_START.

agc_driver_depth_fast_clear_submit: an unrecognized kernel clears the HTILE on two frames in a row and
both clears are taken; slice 1 with words that are not all equal is refused and stored back untouched;
slice 1 with equal words takes the clear. agc_driver_graphics checks the decoded slice.
…-vcc-tuple

fix(shader): accept buffer resource tuples ending in VCC
…ve-at9-channel-config

fix(libSceAjm.native): reject ATRAC9 config data with a reserved channel configuration
…ore-back

A multisampled depth target is created as, or already is, the native depth owner, and a target whose HTILE overlaps a resident depth guard aliases one. Neither may have its HTILE read or rewritten by the generic store-back: the expanded words would be a guest write over the owner's range. DepthSurfaceView decides this before the store-back, which still runs before GpuMutex() and the surfaces mutex are taken. NativeMultisampleDepth binds the owner and a single-sample alias of its HTILE and checks the words stay untouched.
….native-memory-data-stride

fix(savedata): read every sceSaveDataSetSaveDataMemory2 data entry at its 64-byte stride
VideoOut flips on an RTX 3060 now show that a first frame dump that cannot be opened no longer stops presentation. Header, pixel and close failures remain measured only by the CPU fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t-import-registry

fix(relinker): record declared host imports in guest registries
…r-count

fix(agc): accept geometry programs with fewer GS and ES input VGPRs
…ves the write watch

On Windows the write watch is dropped in whole 64 KiB blocks
(WriteWatchCoverage::Exclude), but the unwatch hook was given the page
range of the import. An image with results pending in the same block as
an imported page, outside that page, lost the watch without its guest
bytes saved, and storing its results threw.

The hook now gets the span of blocks Exclude drops (ExcludedSpan, the same
block math), so every image with results pending in those blocks saves its
bytes first.
…n-conflicts

fix(tools): report comparisons blocked by base conflicts
Since boykopovar#5364, Draw/DrawRegisterKey.hpp includes Shaders/ShaderRegistry.hpp,
which reaches ShaderMemory.hpp and its "Recompiler.hpp". tests/Pm4.cpp
includes DrawRegisterKey.hpp, but agc_driver_pm4_tests only links
libSceAgcDriver, libc and libkernel, so the recompiler's include
directories are not on its path and Pm4.cpp no longer compiles. The test
targets that already reach ShaderMemory.hpp link shader_recompiler_agc_driver;
this one now does too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(libkernel): implement rtprio_thread and the pthread concurrency controls
…odelay-nosigpipe-nbio

feat(libkernel): answer TCP_NODELAY, SO_NOSIGPIPE and SO_NBIO on a guest socket
…al-source-stamp

fix(tools): retain source commit when Git status fails
MichonGoddijn231849 and others added 28 commits October 16, 2026 12:53
…IPELINE_DISPATCH)

With pipelined draws, a DISPATCH_DIRECT on queue 0 whose dispatch-cache entry hits with a
recipe runs as an ordered commit item instead of draining the pipeline first. Misses, reads
that overlap writes still pending in the pipeline, verified data hits and writes that cannot
be ranged still drain. A dispatch that is resolved ahead (or adopted from a group resolved
ahead) and copy kernels keep the drained path; with the switch on, queue 0 does not resolve
dispatch groups ahead, since that drains the pipeline at every group. Off by default.
agc_driver_pipelined_dispatch counts drains per opcode (each dispatch drains with the switch
off); agc_driver_pipelined_dispatch_item checks that repeated dispatches do not drain, and that
a dispatch whose descriptor table a pending fill retargets drains and reads the new buffer.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Pixel invocations stopped by a BDA fault now end with OpTerminateInvocation when the target offers SPV_KHR_terminate_invocation, and keep OpKill otherwise. OpKill is not enough on AMD driver branch 21045, where a guest atomic after the kill still lands once per faulting pixel.

Based on the approach in poonasor/AnyPS5@73969879.

Refs boykopovar#2009
…tion

Shader validation accepts the extension only in fragment shaders and only when the device enabled shaderTerminateInvocation. The flag is part of the validation memo key.

Refs boykopovar#2009
The validator only checked SPV_KHR_terminate_invocation when it saw the
OpExtension, so a module using OpTerminateInvocation without declaring
the extension, or in a non-fragment stage, was accepted. Require the
fragment stage, the enabled shaderTerminateInvocation feature and the
declared extension at the instruction itself.

Refs boykopovar#2009
The device queries and enables shaderTerminateInvocation when available, advertises SPV_KHR_terminate_invocation to the recompiler and validation, and the shader device profile rejects a target that advertises the extension without the enabled feature.

Based on the approach in poonasor/AnyPS5@73969879.

Refs boykopovar#2009

(cherry picked from commit 1866a3f)
Restore the shaderDrawParameters feature guard while adding shaderTerminateInvocation support. Cover enabled and disabled DrawParameters profiles, and pass the termination feature to the correct shader validation argument.
…lined-dispatch

perf(agc): dispatch-cache recipe hits as ordered commit items
image_store with D16 data to a storage image of a SINT format threw
"stores 16-bit data to an image of a SINT format", both when the
descriptor is decoded and in the SPIR-V image emitter, and the runtime
image modes left out the SINT conversions for such images. God of War
Ragnarok (PPSA08330) stops on it in compute shader 0x1052a17900 right
after its MFSR setup.

Each 16-bit half is now converted like the D16 format buffer stores
(EmitD16StoreComponent): sign-extended, then saturated to the
component width as for 32-bit data. The buffer form was measured on
RDNA2 in boykopovar#331; the image form is not measured (TechnicalDebt).

agc_driver_image_store_sint now stores D16 data with DMASK 0xf and 0x5
to all eight SINT formats and after an image_atomic_add on 32_SINT,
instead of expecting the refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…journal

perf(agc): journal the pipelines of a run and build them ahead on the next one
…ry-line-strip-output

feat(agc): draw line-strip output of NGG geometry programs as mesh lines
…zed-sampler-without-mip-filter

fix(agc): read an unnormalized sampler without a mip filter at its base level
…ck-terminate-2009

fix(agc): stop pixel invocations on BDA faults when supported
boykopovar#5884 added MeshConfiguration::outputPrimitiveSize, and boykopovar#5875's pipeline
journal guards the layouts it encodes. The two merged separately, so on
Linux with libstdc++ PipelineJournal.cpp no longer compiles:
MeshConfiguration is 40 bytes, ShaderStages (which holds an optional
one) 112. The guards now name those sizes, and the recipe codec writes
and reads the new field, so a replayed line-strip geometry pipeline
keeps its two-vertex primitives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The layout guards only build on Linux with libstdc++, so a Windows or
macOS build of main after boykopovar#5884 wrote journals whose mesh records lack
outputPrimitiveSize under the new shader-cache source version. The
journal format is now 3, so such a file is discarded instead of being
read with the new codec. MarcZX100 pointed this out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re-d16-sint

feat(shader): store D16 data to SINT storage images
…bounds

fix(shader): fold fixed 32-lane vertex bounds
…pare

perf(agc): allow shader preparation on first use
…h-output-size

fix(agc): journal the mesh output primitive size
On Windows the atexit handlers of a DLL run after the process has ended
its other threads, so the logger thread can be gone with lines still
queued and the unbounded Flush waits forever (guest_shader_link timed
out on 4 of the last 7 completed main builds since boykopovar#6371). Run the bounded
FlushAtEnd there instead.

Co-Authored-By: Claude <noreply@anthropic.com>
…ded-atexit

fix(agc): bound the async log's exit flush
@MarcZX10

Copy link
Copy Markdown
Owner Author

Closing this accidentally created PR on the fork; the intended target is boykopovar/AnyPS5.

@MarcZX10 MarcZX10 closed this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AGC: since #5875, a registration library prebuild racing the window attach segfaults in LinkPipelineFromLibraries (Stray PPSA02100, warm shader cache)