Skip to content

test(agc): share native depth fixture backing on Linux - #9849

Open
JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:test/native-depth-shared-fixtures-20261022
Open

JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:test/native-depth-shared-fixtures-20261022

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

The Linux native depth fixtures allocate anonymous private mappings, so they cannot provide a shared backing descriptor to the existing guest-memory importer. On this RADV host, agc_driver_native_depth_bda_clear skips; the other depth tests can pass without exercising the tracked HTILE dispatch.

Give the two Linux fixtures sealed, shared memfd backing, retain the file offset after aligning each mapping, and register a bounded resolver that returns a duplicated descriptor. Remove the resolver on scope exit and unregister write watching before unmapping the BDA fixture. This changes test allocation only; no production rendering code or guest exports change. Windows and other POSIX allocation paths retain their existing backing.

The clear test now logs when its tracked HTILE dispatch actually completes, so a passing run can be distinguished from an unavailable path.

Tested

2026-10-24 review correction: head 88be59f9904bc75a576e084b7a9a8969c4384b56 adds one diagnostic-string change on top of the previously tested 8ee44075d8. The skip now begins skipped, the device does not retain guest write watching ..., preserving the watched/resolver counts and matching the existing Linux DEVICE_LIMITATION classifier. No production code, test branch conditions or importer selection changed. Both focused executables rebuilt successfully. A JUnit check reproduces rejection of the old wording, accepts the new wording, and still rejects an unrelated unexpected skip. Conventions and diff checks pass; local merge-tree against main c077c7aaa1 is clean. Final-head GPU reruns completed through the shared lock (ticket 551): RADV 5 passed/0 skipped; lavapipe 4 passed/1 expected BDA-clear skip. Actual Khronos insertion is verified in every focused test, with zero VUID errors on both drivers. The unchanged tools/check_skips.py --job linux passes on both actual JUnit reports. The earlier paired main/change evidence below retains its own exact heads; this final rerun does not represent a fresh integration build on latest main.

Why lavapipe can still skip with shared backing: its probe logs that a GPU fill resolves host-import write protection (4/4 scratch pages reported written), so decideImportWatch selects comparisons for direct host-pointer imports. The dma-buf probe separately retains protection. With these different policies, import first attempts the direct host pointer and consults shared backing only if that attempt fails. A successful direct import therefore leaves the fixture resolver unused (zero descriptor returns) and marks the imported range unwatched. This follows the existing GuestBufferMemory.cpp selection and agrees with the watched=0 diagnostics; shared memfd backing alone does not force dma-buf selection. The tracked HTILE assertion is expected to remain skipped on this observed lavapipe path. It is not a passing assertion and no forced-watch override or broadening of check_skips.py is introduced. Hosted CI should confirm its own device behavior.

Final-head logs, actual JUnit reports and classifier outputs, SHA-256 2c39a3287b1efa5d82ee7b16a08dabcab2554beb0beb5b67879fef0c3c8a9fe2.

Historical paired GPU validation follows:

Refreshed onto main 1da03c6f34fcff493a5d09966fa30b4269b4da93, final candidate 8ee44075d8838849df7209fce2dfd8f23ca4affc (two linear commits, no merge commits). Both targets rebuilt on main and the final candidate. Same Arch Linux 7.2.9 machine, Ryzen 9 7950X, RX 6650 XT / RADV Mesa 26.2.4; lavapipe Mesa 26.2.4 also tested on that machine.

Five focused native-depth CTests Main Candidate
RADV 4 passed; BDA clear skipped 5 passed, zero skips
lavapipe 4 passed; BDA clear skipped 4 passed; same BDA clear skip

The five tests are agc_driver_native_depth_bda_clear, agc_driver_native_multisample_depth, agc_driver_native_depth_retirement, agc_driver_native_depth_clears, and agc_driver_native_depth_clear_refusals. Khronos validation actually loaded in every test on both revisions and drivers; zero VUID- messages in all four logs. Shader caches disabled and Vulkan required. The retained lavapipe skip is not counted as a pass.

On the RADV candidate, all three BDA owner/texture/attachment consumers report Native full BDA clear accepted; completed clear coverage reports checked=27 unavailable=0, and Native HTILE tracked dispatch completed confirms that path ran. Lavapipe still loses write watching after warm-up (depth and metadata watched=0); the added diagnostic records shared backing present and resolver descriptor returns of zero. Those counts describe resolver calls, not successful Vulkan imports; the change does not override the existing host-pointer importer selection. This retains the reviewer-observed driver limitation.

tools/check_conventions.py and git diff --check pass on the final candidate. Only the two intended test fixture files change; no production renderer behavior changes.

Full refreshed paired logs, exact heads, validation-loader counts, skip diagnostics and final overlap snapshots (refresh-* files). Earlier evidence remains historical and is superseded by these final-head runs.

Reproduce after building agc_driver_native_depth_bda_clear_tests and agc_driver_native_multisample_depth_tests:

VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_LOADER_DEBUG=layer VK_DRIVER_FILES=/usr/share/vulkan/icd.d/radeon_icd.json ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 APS5_WRITE_WATCH_IMPORTS=probe ctest --test-dir BUILD -V --no-tests=error -R '^agc_driver_native_(depth_bda_clear|multisample_depth|depth_retirement|depth_clears|depth_clear_refusals)$'

The validation layer must be installed/discoverable. For lavapipe use its ICD and, on this machine, GALLIUM_OVERRIDE_CPU_CAPS=avx. No Windows runtime, title bring-up or performance gain is claimed.

Manual overlap review refreshed after validation: #6300, #6808, #9533, #9717 and #9742 still use anonymous Linux TargetMemory backing and have no equivalent shared resolver. Three peer heads changed during validation; their full final fixture blobs were retrieved and rechecked. They change depth/stencil/HTILE behavior and coverage; this PR supplies independent Linux fixture backing and diagnostics. Integrating shared fixture edits requires preserving those branches' added test cases. This is a manual shared-file review, not proof that every complete branch merges or jointly passes; Depends on remains none. #6727 is also historical shared-file overlap.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; silent stubs are listed in technical debt (no production paths changed)
  • No notes, investigation .md files or images added to the repository
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none; this improves native test coverage
  • GPUs tested: AMD Radeon RX 6650 XT, RADV 26.2.4 (109060100), Arch Linux
  • AI-assisted: yes

@Dextroid17

Copy link
Copy Markdown
Contributor

Linux data point on CI's device class: a71cc0436 merged with main 9b1ed888d (local merge e142bb770, clean, the 2 files of the diff only), llvmpipe (lavapipe, Mesa 26.2.4) with GALLIUM_OVERRIDE_CPU_CAPS=avx, CachyOS, GCC 16.2.1, Release, ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 APS5_WRITE_WATCH_IMPORTS=probe, ctest -j1.

Result: the skip the PR reports removing on RADV stays on lavapipe, on both arms. The five tests (agc_driver_native_depth_bda_clear, _multisample_depth, _depth_retirement, _depth_clears, _depth_clear_refusals) give 4 passes and 1 skip in 3 of 3 runs on the merge and 3 of 3 on main, and the per-test output is identical between the arms: agc_driver_native_depth_bda_clear prints skipped, the device does not retain guest write watching for native clear imports both times, agc_driver_native_depth_clears skips its HTILE write-coverage assertion both times, and the case counts are checked=16 unavailable=8 (multisample, retirement) and Native depth completed clear cases checked=19 unavailable=8 both times.

Why, from strace of the merge binary: the fixture does what the PR says (two memfd_create("native-depth-bda-clear", MFD_CLOEXEC|MFD_ALLOW_SEALING), F_SEAL_SHRINK, MAP_SHARED mappings), but the only two UDMABUF_CREATE calls are lavapipe's own 4 KiB lp_dma_buf and the 16 KiB aps5-dma-buf-import-probe of the write-watch probe; the 1 MiB fixture blocks (2 MiB memfd mappings) never reach the shared-backing importer. With this probe outcome (host imports compared, dma-buf imports watched) core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp:1759 calls createHostPointerImport alone and :1770 consults the shared-backing resolver only when that fails; createImport at :834 has the same order. On lavapipe VK_EXT_external_memory_host accepts the MAP_SHARED memfd mapping, and the probe decides host imports resolve write protection: yes (... 4 after a GPU write); imported ranges are compared, so the blocks are unwatched after the warm-up clear and the test skips at tests/execution/NativeDepthBdaClear.cpp:185. So the PR does what it says and is inert here: the sealed backing can only matter on a driver where the host-pointer import refuses the mapping. I did not run RADV, so I cannot confirm that is what happens in your table; on CI's Linux lane the test stays skipped for a reason the fixture cannot change. If the skip message distinguished "ranges unwatched after a host import" from "no shared backing", the next reader would not need the strace.

The VK_LAYER_KHRONOS_validation layer is not installed on this host (loader: Layer "VK_LAYER_KHRONOS_validation" was not found), so no layer run from me. I did not run a title.

AI-assisted: yes

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 9533 core/libs/prx/libSceAgcDriver/tests/execution/NativeMultisampleDepth.cpp L855-856, L1288
PR 9717 core/libs/prx/libSceAgcDriver/tests/execution/NativeMultisampleDepth.cpp L855-856, L1288
PR 9742 core/libs/prx/libSceAgcDriver/tests/execution/NativeMultisampleDepth.cpp L855-856, L1288

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6300 core/libs/prx/libSceAgcDriver/tests/execution/NativeMultisampleDepth.cpp
PR 6808 core/libs/prx/libSceAgcDriver/tests/execution/NativeMultisampleDepth.cpp

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed paired run and strace. I checked the import ordering and the post-warm-up Memory::Watched check against this branch; they agree with your explanation. I have added your lavapipe result and its missing-validation-layer limitation to the description, and explicitly scoped the zero-skip result to the tested RADV host. This fixture change does not override host-pointer import selection.

The current skip message does not identify which import path lost write watching. I am retaining that diagnostic improvement as a follow-up; I have not changed production importer ordering or claimed the fixture removes the lavapipe skip.

@MarcZX10

Copy link
Copy Markdown
Contributor

The paired RADV evidence and explicit lavapipe limitation make the scope of this fixture change clear. The branch is now 78 commits behind current main (3bbd2321), and the overlap check already flagged several shared depth-test files. Please rebase and rerun the five focused native-depth CTests on the final head, then refresh the overlap assessment. The current report is useful; this is to ensure the fixture still exercises the intended importer path after the intervening depth-test changes.

@JJenkx
JJenkx force-pushed the test/native-depth-shared-fixtures-20261022 branch from a71cc04 to 8ee4407 Compare October 22, 2026 20:57
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 1da03c6f34fcff493a5d09966fa30b4269b4da93 without merge commits; final head 8ee44075d8838849df7209fce2dfd8f23ca4affc. Rebuilt both revisions and reran all five focused CTests on RADV and lavapipe with the actual Khronos layer loaded, caches disabled and Vulkan required. RADV: main 4 passes/1 skip, final head 5 passes/0 skips. Lavapipe: both 4 passes/1 skip. Zero VUID errors in all four logs. The new diagnostic makes the retained lavapipe write-watch limitation explicit; it does not claim Vulkan import success.

Refreshed the manual shared-file overlap assessment against current heads of #6300/#6808/#9533/#9717/#9742, including the three that changed during validation. None duplicates the Linux shared-backing resolver; shared fixture integrations must retain each branch's test cases. This is not a complete-branch joint-runtime claim. Description and public final-head receipts updated (refresh-* files).

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 8ee44075d8838849df7209fce2dfd8f23ca4affc. This is limited to Linux test-fixture backing; the shared memfd resolver is scoped to the two fixtures and removed on exit, and write watching is unregistered before unmapping. The report now distinguishes the RADV result from lavapipe's retained host-pointer/write-watch limitation and does not claim the fixture forces the latter importer path.

The author reports both revisions rebuilt and the five focused tests rerun with Vulkan required, caches disabled, and the Khronos validation layer loaded: RADV main 4 pass/1 skip versus candidate 5 pass/0 skips; lavapipe 4 pass/1 skip on both. No VUIDs were reported. Current main has no later edits to either changed fixture file since the candidate base. I have no local GPU device for independent execution. Hosted policy checks pass; product jobs are skipped.

@JJenkx

JJenkx commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Windows check that the Windows allocation path is unaffected.

Static. Comparing only the code a _WIN32 build sees, at the merge base and at this head:

  • NativeDepthBdaClear.cpp changes in a blank line and in the skip message's wording. The skip condition is the same two Memory::Watched checks.
  • NativeMultisampleDepth.cpp adds one progress puts and a blank line.
  • The Windows allocation (CreateFileMappingW / GuestArenaMapDirect / write-watch registration) is byte-for-byte the same. All the memfd code is under __linux__.

Runtime. Head 8ee44075d8 merged onto main ee8a7349b5 (clean, giving 7c8f5d87a7), with a main baseline at ee8a7349b5. Windows 11 Enterprise 10.0.22621, WinLibs GCC 15.2.0 UCRT, Release, BUILD_TESTING=ON, AMD Radeon RX 6700 XT (1002:73df, AMD proprietary driver 26.10.1). Every run had VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (Khronos layer 1.4.309), ANYPS5_NO_SHADER_CACHE=1 and ANYPS5_REQUIRE_VULKAN=1, used ctest -V and ran three times. No log reports the layer as missing.

Test main ×3 this PR ×3
agc_driver_native_depth_bda_clear pass pass
agc_driver_native_multisample_depth pass pass
agc_driver_native_depth_retirement pass pass
agc_driver_native_depth_clears pass pass
agc_driver_native_depth_clear_refusals pass pass

No VUID- messages and no skipped tests in either tree. Both trees print the same internal message, in all six runs:

skipped native HTILE write-coverage assertion: the write tracker does not cover the imported metadata range; the production depth-sampling path may still decline this clear

That line inside agc_driver_native_depth_clears predates this PR on Windows, and this PR doesn't change it. Data only; I haven't touched the branch.

@github-actions github-actions Bot added the agc libSceAgc* label Oct 23, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 23, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@JJenkx, a full CI run with this head (8ee44075d8, run 40071220525, merged onto main with four unrelated PRs) fails the Linux check_skips step:

error: agc_driver_native_depth_bda_clear skipped
1 test(s) skipped for a reason that is not expected: the job lost that coverage and ctest still reports every test as passed

Two things combine here:

  1. The test still skips on the Linux CI device with the memfd backing, the same as on main (it also skips in run 40071205858, which doesn't include this PR).
  2. The new skip line, skipped, native clear imports lost guest write watching after warm-up (...), no longer matches DEVICE_LIMITATION in tools/check_skips.py, which accepts the device does not .... On main the old wording matched, so the skip was counted as expected.

Could you keep a the device does not ... prefix on that line (the extra watched/resolver counts can follow it), and check why the shared backing doesn't keep write watching on the CI device (llvmpipe)? If it's expected to skip there, say so in the description. We'll rerun full CI on the next head.

@JJenkx

JJenkx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in 88be59f9904bc75a576e084b7a9a8969c4384b56: the diagnostic again begins skipped, the device does not ...; all watched/resolver counts remain. The existing tools/check_skips.py rejects the old message and accepts the new one in a JUnit fixture, while an unrelated unexpected skip is still rejected. Both focused executables rebuild; conventions/diff and merge-tree checks pass. Only the string changed.

The observed lavapipe skip is still expected: its host-import probe reports write protection resolved by GPU fill, so direct imports use comparisons. GuestBufferMemory.cpp first tries the host pointer; its shared-backing fallback runs only on failure. Since that direct import succeeds, the memfd resolver is not called, its return counts stay zero, and the ranges become unwatched. This explains why shared fixture backing does not by itself retain watched HTILE coverage on this path. The description now says this explicitly and keeps the skip distinct from passing coverage.

The earlier main/change RADV/lavapipe logs remain attached with their exact old heads. The new-head reruns are queued in the shared GPU FIFO; I will attach their results when terminal. Please rerun hosted CI on this corrected head; no production importer change or skip-classifier relaxation is included.

@JJenkx

JJenkx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Final-head reruns are now complete on 88be59f9904bc75a576e084b7a9a8969c4384b56: RADV5/5; lavapipe4 passed plus the expected BDA-clear skip. Every focused test loads Khronos validation; zero VUID errors on both drivers. tools/check_skips.py --job linux passes on both actual CTest JUnit outputs, including the lavapipe skip with the retained diagnostic counts. The earlier classifier fixture still rejects both the old wording and an unrelated unexpected skip.

Final-head logs, actual JUnit reports and classifier outputs, SHA-256 2c39a3287b1efa5d82ee7b16a08dabcab2554beb0beb5b67879fef0c3c8a9fe2.

Description updated. This addresses the requested wording/skip-classification change and documents the existing host-pointer path limitation; it does not force watched imports or count the unavailable HTILE assertion as passing. Hosted full CI remains for the maintainer to rerun.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants