Skip to content

fix(tests): stage linked .prx runtime files next to test executables on Windows - #7645

Open
akkadbakkad89 wants to merge 1 commit into
boykopovar:mainfrom
akkadbakkad89:fix/test-prx-dll-staging
Open

akkadbakkad89 wants to merge 1 commit into
boykopovar:mainfrom
akkadbakkad89:fix/test-prx-dll-staging

Conversation

@akkadbakkad89

@akkadbakkad89 akkadbakkad89 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What

add_test_executable gains a POST_BUILD step that copies $<TARGET_RUNTIME_DLLS> next to the test exe on Windows. The step is gated on CMAKE_VERSION >= 3.21 (the generator expression's floor) so the declared cmake_minimum_required(VERSION 3.20) keeps configuring; on <3.21 the step is absent and the existing per-test ENVIRONMENT_MODIFICATION PATH prepends apply unchanged (note the prepend property itself is a 3.22 feature that ctest silently ignores on 3.20-3.21, so this widens rather than narrows the working range; the loader trap below 3.22 predates this PR). The executable directory outranks PATH in the DLL search order, so a test whose registration forgot the ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>" boilerplate still resolves libc.prx/libkernel.prx/libSceAgcDriver.prx at load - instead of spawning a modal "entry point not found" (0xC0000139) dialog that blocks until ctest reports a timeout.

The prepend is manual per-test boilerplate repeated across ~95 CMakeLists files; new registrations keep missing it (live instance fixed in #7627's agc_driver_depth_bounds_state). With staging, no registration needs it, future omissions stop mattering, and hand-running a test exe works with no environment setup. Existing ENVIRONMENT_MODIFICATION lines stay harmless.

Tested

Windows 11, MinGW (WinLibs GCC 15.2.0), Release, RTX 3070 (driver 617.42).

  • objdump evidence + generated build line: agc_driver_graphics_tests links libc/libkernel/libSceAgcDriver and the copy stages exactly those three unpatched/*.prx into tests/; exes with no shared-library deps (nid_buffer_validation_tests) build clean (empty $<TARGET_RUNTIME_DLLS> is a no-op).
  • Direct run with PATH scrubbed to C:\Windows\System32;C:\Windows: agc_driver_graphics_tests.exe passes the full graphics suite - the loader resolves the .prx imports from the exe directory.
  • ctest sweep: agc_write_tracking, agc_buffer_pool (+2 small-device variants), agc_import_watch, agc_import_watch_other_family, agc_host_import_unmap, agc_host_import_retirement, agc_host_import_second_mapping, agc_host_import_retire, guest_kernel_sys, guest_cpu_time_clocks, nid_buffer_validation - 14/14 pass, agc_host_import_sparse skipped by design.
  • CMake 3.20.5 (real binary, minimal fixture): the un-gated $<TARGET_RUNTIME_DLLS> form fails generation (Expression did not evaluate to a known generator expression); the gated form configures, generates and builds clean.
  • python3 tools/check_conventions.py --base origin/main: clean; git diff --check: clean.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: Windows: tests whose exe imports a .prx hang under ctest when the registration lacks the unpatched-dir PATH prepend #7637
  • Unblocks: none
  • GPUs tested: n/a (test-infrastructure change; graphics suite run on RTX 3070 / 617.42 / Windows 11 only as a staging check)
  • AI-assisted: yes

@MarcZX10

Copy link
Copy Markdown
Contributor

I rechecked the submitted diff against the cross-target reproduction from #7637. The PR currently still uses cmake -E copy -t "$<TARGET_FILE_DIR:${target}>" $<TARGET_RUNTIME_DLLS:${target}>. With COMMAND_EXPAND_LISTS, CMake 3.25.1 expands this to copy -t <exe-dir> file1.prx file2.prx file3.prx, which fails: Target (for copy command) ".../file3.prx" is not a directory. I reproduced it with one and three MinGW Windows runtime targets. The tested working form is cmake -E copy_if_different $<TARGET_RUNTIME_DLLS:${target}> "$<TARGET_FILE_DIR:${target}>/" (also with COMMAND_EXPAND_LISTS); it generated the expected copy command and staged all three files next to the exe in the cross-target fixture. Please update the post-build command; the current Windows run in the PR description may have used a different command than the submitted one.

@MarcZX10

Copy link
Copy Markdown
Contributor

One more edge case from the same cross-target: a test target with no linked runtime DLLs also fails in the submitted form. TARGET_RUNTIME_DLLS expands empty, leaving cmake -E copy -t <exe-dir>; CMake 3.25.1 exits with Error copying file "-t" to ".../build-win". Please guard the post-build command when the list is empty, or use a helper that conditionally stages only nonempty lists. The copy_if_different correction above handles the multi-file case when the list is nonempty; it also needs the empty-list guard.

@akkadbakkad89
akkadbakkad89 force-pushed the fix/test-prx-dll-staging branch from 8ee1664 to d2fbfd9 Compare October 18, 2026 19:24
@MarcZX10

Copy link
Copy Markdown
Contributor

I tested the updated copy_if_different form from the current head d2fbfd9 in the same MinGW Windows cross-target fixture. With three linked .prx targets, the generated command now copies all three alongside the executable and succeeds. A no-runtime-DLL test also builds successfully; including $<TARGET_FILE:${target}> keeps the command nonempty, and CMake handles copying that file to its existing output directory. This closes both failures I reported in the submitted command. The hosted windows app-build lane is still skipped, so this is focused cross-target validation, not a full project Windows build.

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Thanks for the fast re-verification on the 3.25.1 cross-target - good that the copy_if_different form plus the $<TARGET_FILE> guard closes both the multi-file and empty-list cases. Confirmed locally on cmake 4.4.4 as well: full graphics suite passes with PATH scrubbed, 14/14 on the ctest sweep including nid_buffer_validation (the no-DLL case). Current head d2fbfd925 carries exactly the command you validated.

@github-actions github-actions Bot added the build label Oct 18, 2026
@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 8043 CMakeLists.txt L16

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 674 CMakeLists.txt
PR 1835 CMakeLists.txt
PR 3786 CMakeLists.txt
PR 5021 CMakeLists.txt
PR 7201 CMakeLists.txt
PR 8161 CMakeLists.txt
PR 9508 CMakeLists.txt

@github-actions github-actions Bot added the fix error Closes an issue (Fixes #N) label Oct 18, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One compatibility issue to resolve before merging: the root CMakeLists.txt declares cmake_minimum_required(VERSION 3.20), but $<TARGET_RUNTIME_DLLS:...> was added in CMake 3.21 (CMake 3.21 release notes). On Windows with CMake 3.20 this generator expression is evaluated during generation and is unknown, so the newly added if(WIN32) command can make configuration fail for a version the project currently claims to support. Could you either preserve 3.20 compatibility (for example, gate an alternate staging path by CMake version) or raise the project minimum to 3.21 and update the documented/tooling requirement? The current CMake 4.4.4 validation does not cover the declared minimum.

@akkadbakkad89

akkadbakkad89 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor Author

Overlap bot report checked: all 13 shared-file PRs touch CMakeLists.txt regions unrelated to add_test_executable (lines 11-25). Closest: #7642 edits request_serializer_tests includes at ~line 92, #7039 adjusts a TIMEOUT at ~line 173, #3998 calls add_test_executable for a new test (benefits from the staging automatically). No real overlap.

…on Windows

A test registration that omits the WIN32 PATH prepend for libs/unpatched
makes the executable block on the Windows entry-point dialog (0xC0000139)
until ctest reports a timeout; the dialog is invisible to headless and CI
runs. The prepend is per-test boilerplate repeated across ~95 CMakeLists
files, so new registrations keep missing it (boykopovar#7627).

add_test_executable now copies $<TARGET_RUNTIME_DLLS> next to the exe at
link time (copy_if_different rather than copy -t, whose -t option older
cmake -E lacks; the exe itself is listed as a source so a target with no
shared-library dependencies still receives a valid command and copies
nothing). The loader resolves the .prx imports from the executable
directory ahead of PATH, so no registration needs the prepend and a test
exe can be run by hand with no environment setup. Fixes boykopovar#7637.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@akkadbakkad89
akkadbakkad89 force-pushed the fix/test-prx-dll-staging branch from d2fbfd9 to ee69ac6 Compare October 18, 2026 20:08

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking the real CMake 3.20.5 behavior; the version gate correctly avoids the unknown-generator-expression failure. One remaining scope point: on 3.20 the staging command is disabled, so a test registration without the manual PATH prepend can still hit the same loader dialog, and direct-running the exe still lacks the new guarantee. Could you either add a 3.20-compatible staging fallback, or state that this general protection applies only with CMake >=3.21 (and retain the per-test PATH setup as required below that)? The current reported CMake 3.20 validation proves configuration/build, not the no-PATH runtime behavior.

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Good catch — fixed in ee69ac6d0 by gating the staging step on the generator expression's floor:

if(WIN32 AND NOT CMAKE_VERSION VERSION_LESS 3.21)
    add_custom_command(TARGET ${target} POST_BUILD ...)
endif()

Chose the gate over bumping the minimum so this fix stays scoped. On <3.21 the command is never created, so the unknown genex can't fail generation — and the per-test ENVIRONMENT_MODIFICATION PATH prepends still apply, so behavior there is exactly today's.

Verified against a real CMake 3.20.5 binary (PyPI wheel) with a minimal fixture replicating the pattern:

  • Un-gated form: generation fails with $<TARGET_RUNTIME_DLLS:...> — Expression did not evaluate to a known generator expression (reproduces the failure you predicted on the declared floor).
  • Gated form: configures, generates and builds clean on 3.20.5.
  • Regression-checked on CMake 4.4.4 + MinGW: reconfigured, relinked agc_driver_graphics_tests, all three .prx files staged into tests/, and the suite still passes with PATH scrubbed to System32.

Side note for context, not pushback: the repo already relies on ENVIRONMENT_MODIFICATION (3.22) in ~92 registrations, so the de-facto floor for working tests is higher than 3.20 — but those fail silently at runtime rather than breaking generation, so the gate is still the right call here.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current head addresses the CMake 3.20 generator-expression failure by gating the staging command at 3.21, and the updated body clearly records that the existing per-test PATH setup remains in effect below that version. The reported MinGW staging checks cover linked and no-runtime-DLL targets, and the current conventions, relinker platform, and ASan checks pass. I approve this focused test-infrastructure change.

@akkadbakkad89

Copy link
Copy Markdown
Contributor Author

Fair point — taken the second option and scoped the claim in the PR body: staging applies on CMAKE_VERSION >= 3.21; below that, behavior is byte-for-byte today's.

One nuance worth noting on the fallback side: ENVIRONMENT_MODIFICATION is itself a CMake 3.22 feature, so on 3.20–3.21 ctest accepts the per-test PATH=path_list_prepend:... lines but silently ignores them — meaning .prx-importing tests already hit this loader trap on those versions today, prepend or no prepend. The gate therefore can't regress anything: 3.20 keeps exactly its current (already-broken-for-ctest) behavior, while 3.21 actually gains protection the prepend could never deliver there.

A true 3.20 fallback would need copy_directory of the whole unpatched dir per exe — a second code path staging ~all prx rather than the linked set — to benefit a toolchain where ctest can't honor the existing registrations anyway. Happy to add it if you'd rather have that, but the scoped claim seemed the honest fit given the prepend is a no-op below 3.22 regardless.

@Neithennn

Copy link
Copy Markdown
Contributor

Full-suite Windows run, to cover every test executable now that each one gets the staging step (the description lists a 14-test sweep).

Windows 11 Pro 26200, NVIDIA GeForce RTX 4080 (driver 617.42), WinLibs GCC 15.2.0 r7, CMake from WinLibs, Ninja, Release, bundled dependencies, BUILD_TESTING=ON, ANYPS5_NO_SHADER_CACHE=1. Head ee69ac6d0 merged onto main 36f60df9e; the full build succeeds and stages 108 .prx files in build/tests/.

  • Full ctest -j 8 --timeout 120: 1629/1631. The two failures are shader_cache_runtime_sources and hardware_oracle_compare, both hitting my 120 s limit under parallel load; alone on this tree they pass 3/3 each (55 to 58 s and 11 s). On unchanged main the same run gave 1631/1631, with shader_cache_runtime_sources at 79 s, so that one is close to the limit either way.
  • Targeted, with ANYPS5_REQUIRE_VULKAN=1: agc_driver_graphics, agc_write_tracking, agc_import_watch, agc_import_watch_other_family, guest_kernel_sys, nid_buffer_validation: pass.

I did not repeat the scrubbed-PATH direct run.

AI-assisted: yes (runs done with Claude Code).

@THEMANnt THEMANnt left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed ee69ac6d0e6f1fb9a08c9a6d9790b12d35da6985 against 8d17389a73852ee0e462c640589b47d49c68e8d6. No actionable findings in this focused review.

Source review: Inspected the complete helper change and its existing runtime output/wrapped-main setup. Windows CMake 3.21+ stages linked runtime dependencies beside the test. COMMAND_EXPAND_LISTS preserves the DLL list, VERBATIM handles paths, and including the executable itself keeps copy_if_different valid when the dependency list is empty. Older CMake and non-Windows behavior are unchanged.

Validation: Windows, CMake 4.3.1 / MinGW GCC 15.2: a disposable project using the exact head helper and WindowsTestExit source built a shared .prx plus linked and standalone tests in directories containing spaces. Both head executables ran with exit 0; the linked .prx was staged. The same baseline fixture built but did not stage that file. Conventions and whitespace passed. This older script-registration CLI rejects --scripts; no registration pass is claimed. Effective hosted relinker matrix and ASan succeeded; full-product jobs were skipped.

Scope: Actual bounded CMake/build/runtime fixture; full AnyPS5 libraries and all test targets were not rebuilt locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build fix error Closes an issue (Fixes #N)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows: tests whose exe imports a .prx hang under ctest when the registration lacks the unpatched-dir PATH prepend

5 participants