Repository navigation
Conversation
|
Windows / AMD results for this head, in a scene where queue worker 0 is the limit: ASTRO BOT (PPSA21564 01.007.000), Host: Windows 11 Pro 26200, Ryzen 7 5800X3D, Radeon RX 9070 XT (AMD proprietary driver 32.0.23033.1002), 32 GB, WinLibs GCC 15.2.0 r7, Release. Builds.
Method. No input, warm shader cache (1,904 hits, 0 misses),
Seven launches with this PR went through boot, loading and at least 3 minutes in the hub: the two measured above, and five more held 3 minutes each, at 6.62-6.67 FPS. None had a stop, a hang or an error in the log. AI-assisted (Claude Code); the builds and runs are on my machine. |
6170a0c to
54b0ce0
Compare
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
… report boykopovar#6292 moves the APS5_PROFILE_DRAW check in Driver::timed to the top and rewrites the report's condition, the two lines this branch's driver-time line sat between, so the two conflicted. The line now follows the report. It reads the same flag and times, so it counts the same.
7ffce1c to
c14f4c4
Compare
c14f4c4 to
562e4c8
Compare
|
Evidence from Stellar Blade on Windows (MinGW build): I measured this change as part of #6615's stack, which includes it. Ported onto a downstream perf branch on origin/main 4ffb0cd (RX 9070 XT, 4K, warm cache, two base/port pairs, 49-150 s of gameplay, whole traces), the stack gave +4.5% and +9.5% presents (about +7%). p50 dropped 9-13% and p99 16-22%. Peak commit was 0.75-1.2 GB higher. I couldn't isolate #6292 on its own. Full table and trace_digest output: #6615 (comment). |
Ports boykopovar#6292 and boykopovar#6615 (boykopovar#6522 is already present). Draw-path mutexes and the GPU-lock gate become atomic locks with explicit recursion, and per-draw thread-locals move to native TLS slots (TEB slots on Windows) instead of emutls. The same conversion also covers the newer sb5 paths: guest allocation registry, trace thread binding, own-stack checks, collect counters, recorder sync attribution, held-read scratch, packet profile, draw keys, supplied shader-memory blocks and the changed-block scratch. There is no runtime switch; compare builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What
On Windows the driver is built with MinGW:
std::mutexlocks and unlocks through winpthreads (pthread_mutex_lock,pthread_mutex_unlock);thread_localis emulated: every access calls__emutls_get_address, which callspthread_getspecific.A draw takes a few dozen different mutexes, and every packet reads several thread-locals. On the S2 page of Sonic Origins Plus's main menu, the queue worker spent these samples per frame (APS5 sampler, 15 s):
pthread_mutex_lock1.22,pthread_mutex_unlock0.82,__emutls_get_address2.15 (1.80 of it inpthread_getspecific), and the device use gate 0.37. That is about 10% of its frame.AgcDriver::Mutex(Execution/include/Mutex.hpp) is an inline mutex on one atomic word.std::atomic::wait, which spins 12 pauses and 4 yields itself before it sleeps). Likestd::mutex, it is neither recursive nor fair.std::mutexwith 4 to 8 threads on one lock, and that waiting at once beatsstd::mutexat every thread count. Without the spin, main'sagc_driver_multisample_fill_host_onlyfailed most runs: waiters that sleep at once widen a race main already has (agc_driver_multisample_fill_host_only fails intermittently: the sentinel fill after an absorbed multisampled clear does not land #6512), a multisampled target's clear recorded on another device's recorder. fix(agc): record a multisampled target's commands on its own device #6522 fixes that race, so this PR depends on it. The game numbers below come from the spinning version; on that page the contended path takes no samples (0.00 per frame in three sampled runs of my fork's dev), so they hold.std::mutexes of the draw path. perf(agc): pipeline queue-0 draw commits behind the packet worker #1201 already made two more aSlimMutex(the pending storage writes and the unit shadows), and those stay as they are.std::mutex: the five astd::condition_variablewaits with, and the sparse and settle queue mutexes, which the context hands out asstd::mutex*.DeviceUseGatekeeps its user count and the replacement flag in one atomic word instead of a mutex and a condition variable. A packet enters and leaves with one atomic instruction each.GpuMutextakes anAgcDriver::Mutexand counts its own recursion. Its owner token is the thread environment block's address (CurrentThreadToken) instead of a thread-local's.HostThreadSlot(libc/include/HostThreadSlot.hpp) holds a per-thread value of up to 8 bytes.TlsAlloc), read and written through the thread environment block in one or two instructions. Elsewhere it is athread_local.ThreadOwned).Driver::timedreads no clock unlessAPS5_PROFILE_DRAWasks for its report. A wait packet reads it only forAPS5_TRACE_GPU's slow-wait line (suggested by @simonbronk).agc_driver_locks(tests/execution/Locks.cpp) checks:try_lock, and a sleeping waiter's wake;With this change, per frame:
pthread_mutex_lock1.22 to 0.31,pthread_mutex_unlock0.82 to 0.20,__emutls_get_address2.15 to 1.48, and the device use gate 0.37 to 0.05. What remains is the mutexes kept or outside the draw path, and thread-locals this change does not touch.Tested
Windows 11 (10.0.26300), WinLibs GCC 15.2.0 (UCRT), Release, NVIDIA GeForce RTX 2080 Ti, driver 617.14.
AdoptLatestCollectEpochandlastBumpedEpochread the collect epoch through theHostThreadSlot.LockSpinning) inside its recursion guard.dumpRequest's new suffix and the descriptor pool'sfreeDescriptorSetsstay, withAgcDriver::Mutex.AcquireCachetakes theAgcDriver::Mutex.agc_driver_bda_devicefails (vkWaitForFences graphics: Vulkan result -4) on plain main 0bc3003 as well, 0 of 4 runs alone.drawCacheMutex, main added the draw relocation fields (perf(agc): find cached draws across moved user-data pointers #4943), and perf(agc): journal the pipelines of a run and build them ahead on the next one #5875 keyed the compute pipelines byComputePipelineKeyand dropped the pipeline store'sscratchKey.drawCacheMutex,computePipelinesMutexand the pipeline store's mutex stayAgcDriver::Mutexes, and every lock of them is astd::lock_guardorstd::unique_lockas before. The collect memo ring goes through its thread slot as before. perf(agc): pipeline queue-0 draw commits behind the packet worker #1201's per-epoch collect map next to it keeps itsthread_local, like the other thread-locals main added since.tools/check_conventions.pypasses:HostThreadSlottakesTlsAllocandTlsSetValuefrom<processthreadsapi.h>, and the GPU mutex's unlock keeps main's comment where it was.agc_driver_gpu_indirect_arguments_misaligned_writer_ahead, which main itself does not compile with MinGW), including the newagc_driver_locks.agc_driver_multisample_fill_host_onlyfailed 0 of 200 runs (8 at a time) on main 72c5eed. On 44df216 it also failed 0 of 400 withAPS5_PIPELINED_DRAWS=0(4 at a time) and 0 of 40 alone, and without fix(agc): record a multisampled target's commands on its own device #6522's line it failed 40 of 200 and 7 of 40.APS5_TRACE_SECONDS=200:15on the same machine in one session, caches warm. Traces attached.This pair overstates the change. The baseline's window ran slow outside the draw path:
Driver.Packetminus its draw part is 37.0 ms against 11.3 ms with this change. Its title FPS, 6.70, is also below every other run of that build (7.51 to 8.34). The sampled pair, 7.75 to 8.38 FPS (+8%), is the better estimate.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)Traces and summary lines (baseline b8acbb0, this change 824084e):
locks-traces.zip
The two summary lines, with
tools/trace_digest.pyfor each trace:Baseline (summary commit a38ad5e)
seconds 200:15 from present 2260: 95 presents, frame p50 156.7 / p90 163.0 / p99 190.3 / max 190.3 ms, GPU busy 78.6 ms per present
NVIDIA GeForce RTX 2080 Ti, NVIDIA 617.14, Windows 10.0.26300, 3840x2160, commit a38ad5e (dirty), dropped: yes
{"commit":"a38ad5e9fc89","dirty":true,"env":{"APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"200:15"},"gpu":"NVIDIA GeForce RTX 2080 Ti","driver":"NVIDIA 617.14","os":"Windows 10.0.26300","resolution":"3840x2160","window":{"spec":"seconds 200:15","first_present":2260,"ms":15034.646},"presents":95,"frame_ms":{"p50":156.721,"p90":163.045,"p99":190.275,"max":190.275},"gpu_busy_ms_per_present":78.608,"worker_busy_pct":{"queue worker 0x0":98.389},"top_phases_ms_per_frame":{"Driver.Packet":155.677,"Driver.Packet.draw":118.731,"Driver.Draw":117.267,"Driver.Draw.draw_and_resource_release":54.084,"Graphics.Draw":52.74,"Driver.Draw.shader_compile_and_link":45.397,"Graphics.Draw.draw_and_resource_release":42.561,"Shader.DrawStage":38.974},"trace_dropped":true,"trace":"aps5-trace-a38ad5e9fc89-37256.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): batch 78.5, Driver.Packet 34.4, Driver.Packet.draw 27.8, Driver.Draw 27.5, Driver.Present 27.0, Driver.Present.render_fence_wait 26.4, Vulkan.Retire 26.4, Vulkan.Retire.wait 26.4, Driver.Draw.draw_and_resource_release 6.1, Graphics.Draw 6.0
Longest waits
This change (summary commit b8acbb0)
seconds 200:15 from present 2475: 115 presents, frame p50 129.7 / p90 139.3 / p99 153.5 / max 153.5 ms, GPU busy 85.8 ms per present
NVIDIA GeForce RTX 2080 Ti, NVIDIA 617.14, Windows 10.0.26300, 3840x2160, commit b8acbb0 (dirty), dropped: yes
{"commit":"b8acbb0d65c3","dirty":true,"env":{"APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"200:15"},"gpu":"NVIDIA GeForce RTX 2080 Ti","driver":"NVIDIA 617.14","os":"Windows 10.0.26300","resolution":"3840x2160","window":{"spec":"seconds 200:15","first_present":2475,"ms":15060.273},"presents":115,"frame_ms":{"p50":129.749,"p90":139.339,"p99":153.497,"max":153.537},"gpu_busy_ms_per_present":85.85,"worker_busy_pct":{"queue worker 0x0":98.33},"top_phases_ms_per_frame":{"Driver.Packet":128.738,"Driver.Packet.draw":117.44,"Driver.Draw":116.398,"Driver.Draw.draw_and_resource_release":52.218,"Graphics.Draw":51.006,"Driver.Draw.shader_compile_and_link":45.265,"Graphics.Draw.draw_and_resource_release":41.533,"Shader.DrawStage":39.233},"trace_dropped":true,"trace":"aps5-trace-b8acbb0d65c3-37744.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): batch 85.7, Driver.Present 32.0, Driver.Present.render_fence_wait 31.4, Vulkan.Retire 31.4, Vulkan.Retire.wait 31.4, Driver.Packet 23.6, Driver.Packet.draw 22.7, Driver.Draw 22.5, Driver.Draw.draw_and_resource_release 4.9, Graphics.Draw 4.8
Longest waits