Skip to content

test: add in-repo test framework and labelled test layout - #2630

Closed
elhajahmadj wants to merge 2 commits into
boykopovar:mainfrom
elhajahmadj:test/unit-test-architecture
Closed

elhajahmadj wants to merge 2 commits into
boykopovar:mainfrom
elhajahmadj:test/unit-test-architecture

Conversation

@elhajahmadj

Copy link
Copy Markdown

What

Adds a small in-repo test framework (core/testing) and moves back-end tests into a labelled layout, so ctest -L unit runs fast tests that don't need a device.

  • core/testing: named cases, a runner that keeps going after a failed case, --list, --case <name>, positional arguments, skip exit code 77 (skipped, <reason>, read by tools/check_skips.py), RequireEqual, RequireThrows* and TemporaryDirectory. It needs only the C++20 standard library, so the relinker-only build can use it. It is an INTERFACE library compiled into each test, so tests built with the core/libs unwind flags can still catch failures.
  • anyps5_add_test and anyps5_add_library_test register tests with one of the labels unit, integration, gpu, e2e or tooling. Only unit tests get an explicit 30 s timeout; the other labels use ctest --timeout.
  • Tests move into tests/{unit,integration,gpu,e2e} folders. New unit tests cover nid (SHA-1, NID compute and resolve, the patchers, export exclusions), ElfReader, GuestImageReader and the JPEG and PNG decoders.
  • docs/dev/TESTING.md describes the layout. CONTRIBUTING.md and README.md link to it.
  • .gitignore: Testing/ (the CTest output folder) also matched core/testing/include/Testing/, so this adds negations for the framework folder.

Fixes found by the new tests (fix: commit):

  • ElfReader checks the section header entry size, shstrndx < shnum and section name bounds. It rejects virtual addresses that wrap or map outside the file.
  • Jpeg::Decode zero-initialises stb allocations, so a truncated scan no longer returns uninitialised pixels. It returns nullopt when there is no start-of-scan marker.

Found but not fixed here (each needs its own PR):

  • Recorder may crash when the first occlusion DumpSamples runs inside a render pass.
  • The driver may keep cached colour-target state after guest memory is freed (Srgb8 test).
  • Two Recorder storage-refresh cases already fail on main.
  • A repeated sceVideoOutOpen after a failure may corrupt the heap.
  • The PNG decoder does not check chunk CRCs.
  • guest_sce_net is flaky under parallel load.
  • Tests leak a GuestBlock in FlatStore, FlatCacheBits and GlobalAtomicCsub.

Follow-up: 75 test files still use the old main() + Require style through CUSTOM_MAIN or the old helpers. Converting them goes in a separate PR.

Tested

  • Windows 11, MinGW GCC 15.2, full build (all + libs): no errors. ctest -L unit passes 49 of 49. tools/check_test_registration.py reports 500 test executables, none built and not run.
  • WSL Ubuntu, a clean clone of this branch, -DANYPS5_RELINKER_ONLY=ON: builds without warnings, and all 61 tests pass.
  • tools/check_conventions.py passes.
  • I did not run the full GPU suite on hardware.

Checklist

  • Based on current main; no other open PR implements the same functions. This branch is about 370 commits behind main and touches files that changed there, so it will need a rebase before merge.
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

ElfReader validates the section header entry size, the string table index and section name bounds, and rejects virtual addresses that wrap or map outside the file. Jpeg decoding zero-initialises stb allocations and returns nullopt when the stream has no start of scan.
Adds core/testing with named cases, a runner that keeps going after failures, skip exit code 77 and the anyps5_add_test helper. Tests move into unit, integration, gpu and e2e folders with matching CTest labels, and new unit tests cover nid, ElfReader, GuestImageReader and the JPEG and PNG decoders. docs/dev/TESTING.md describes the layout.
@github-actions github-actions Bot added libc libkernel build relinker docs prx agc libSceAgc* intake paused Closed while PR intake is limited to authors with at least 10 PRs merged into main labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown

Thank you for your interest in the project! We are sorry, but we cannot accept more PRs while we work through the existing backlog. This PR has been automatically closed and labeled "intake paused". This is not a judgment of the quality of your changes.

@github-actions github-actions Bot closed this Oct 10, 2026
@github-actions

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#1384 Allocate
Release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs intake paused Closed while PR intake is limited to authors with at least 10 PRs merged into main libc libkernel prx relinker

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant