Skip to content

feat(libSceNpCppWebApi): complete the InGameCatalog and Leaderboards offline surface - #13206

Open
edubox wants to merge 10 commits into
boykopovar:mainfrom
edubox:feat/npcpp-catalog-leaderboards
Open

edubox wants to merge 10 commits into
boykopovar:mainfrom
edubox:feat/npcpp-catalog-leaderboards

Conversation

@edubox

@edubox edubox commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What

Completes the remaining imported offline surface of sce::Np::CppWebApi::InGameCatalog::V5 and sce::Np::CppWebApi::Leaderboards::V1, 21 NIDs with no implementation:

Namespace Exports Behaviour
InGameCatalog::V5 14 the twelve Container/Sku accessors read the measured fields; the two ParameterToGetContainer setters store the value and an is-set flag through the implementation pointer
Leaderboards::V1 7 the four request setters store the value and an is-set flag; the three response accessors read the measured fields

The offsets and widths come from the module bundled with PPSA21564/PPSA23732 (builds J03093737/J03341606): Container::getId is the pointer at +0x38, skusIsSet is the byte at +0x220, GetRankingResponseBody::getTotalEntryCount is the 32-bit value at +0x58 and RecordScoreRequestBody::needsTmpRankIsSet is the byte at +0x42, plus the other offsets recorded in the debt. The two parameters now allocate a host implementation of the module layout so setfields and the Container setters can store through +0x8; setfields returns 0x80553502 before initialize and 0x80553503/0x80553504 when the string copy fails. The host allocator stands in for the module's LibContext allocator.

Tested

Linux x86-64, GCC 16.2.1, Release with BUILD_TESTING=ON, rebased on main ab6f5d32ed:

  • ctest --test-dir build -R '^guest_np_cpp_web_api': 11/11 passed, including the new guest_np_cpp_web_api_catalog_leaderboards. It pins the measured offsets, the is-set flags, the vector reference-count increment and the parameter lifecycle/error codes instead of asserting throws or null-self success.
  • python3 tools/check_conventions.py --base origin/main passes.
  • python3 tools/pr_overlap.py --branch feat/npcpp-catalog-leaderboards --jobs 32: no overlap with active open pull requests. The tool could not compare a set of stale pull requests that already conflict with main in core/libs/CMakeLists.txt.
  • No title run.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Progress report (9f2b939 → eecab38)

📈 System libraries: 30.02% (+0.04%, +21 implemented functions)

✅ 21 implemented
Library Name
libSceNpCppWebApi _ZN3sce2Np9CppWebApi12Leaderboards2V121GetRankingRequestBody17setNpServiceLabelERKj
libSceNpCppWebApi _ZN3sce2Np9CppWebApi12Leaderboards2V122RecordScoreRequestBody15setWaitsForDataERKb
libSceNpCppWebApi _ZN3sce2Np9CppWebApi12Leaderboards2V122RecordScoreRequestBody17setNpServiceLabelERKj
libSceNpCppWebApi _ZN3sce2Np9CppWebApi12Leaderboards2V19BoardsApi29ParameterToGetBoardDefinition9setfieldsEPKc
libSceNpCppWebApi _ZN3sce2Np9CppWebApi13InGameCatalog2V512ContainerApi23ParameterToGetContainer14setkeepHtmlTagEb
libSceNpCppWebApi _ZN3sce2Np9CppWebApi13InGameCatalog2V512ContainerApi23ParameterToGetContainer20setuseCurrencySymbolEb
libSceNpCppWebApi _ZN3sce2Np9CppWebApi13InGameCatalog2V59Container7getSkusEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi12Leaderboards2V122GetRankingResponseBody12entriesIsSetEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi12Leaderboards2V122GetRankingResponseBody18getTotalEntryCountEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi12Leaderboards2V122RecordScoreRequestBody17needsTmpRankIsSetEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V53Sku10getEndDateEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V53Sku12endDateIsSetEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container13childrenIsSetEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container14getDescriptionEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container14getDisplayNameEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container14getReleaseDateEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container16releaseDateIsSetEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container5getIdEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container7getTypeEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container8getLabelEv
libSceNpCppWebApi _ZNK3sce2Np9CppWebApi13InGameCatalog2V59Container9skusIsSetEv

Changed libraries (implemented exports)

Library Before After Change
libSceNpCppWebApi 1055/1430 (73.78%) 1076/1451 (74.16%) +0.38%

@JJenkx

JJenkx commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Windows run of this head (744f1938d5), merged onto main 7d4006607a (clean merge, local only). Windows 11 Enterprise 10.0.22621, Ryzen 9 5950X, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON, no Vulkan driver visible.

  • This PR: guest_np_cpp_web_api_catalog_leaderboards passes 3 of 3 (ctest --repeat until-fail:3).
  • Negative control: not possible. With this PR's source files (core/libs/prx/libSceNpCppWebApi/Export.cpp, core/libs/prx/libSceNpCppWebApi/include/Api.hpp) put back to main's version, the build fails against the rest of the PR, so there is no source-only control.

AI-assisted: yes (Claude Code).

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@edubox thanks. The 21 names all hash to the expected NIDs, but the bundled module (the libSceNpCppWebApi.prx shipped with PPSA21564, build J03093737, SHA-256 0666df3c...67c2) behaves differently from this PR for most of them:

  • The 15 accessors are plain field reads, not throws. For example Container::getId is lea rax,[rdi+0x38]; ret, Container::skusIsSet is movzx eax,byte [rdi+0x220]; ret, GetRankingResponseBody::getTotalEntryCount is mov eax,[rdi+0x58]; ret, and RecordScoreRequestBody::needsTmpRankIsSet is movzx eax,byte [rdi+0x42]; ret. That last one reads the title's own request object, so "only exists after a successful response" doesn't apply to it.
  • The setters store state. The request-body setters write the value plus an is-set flag (for example RecordScoreRequestBody::setWaitsForData stores to +0x79 and sets +0x7a). The two ParameterToGetContainer setters write through the implementation pointer at +0x8, and do nothing when it's null. ParameterToGetBoardDefinition::setfields returns 0x80553502 when the parameter isn't initialized or the copy fails, where this PR returns 0.
  • The test pins unfinished behaviour. It asserts that all 15 accessors throw and that every setter returns 0 for a null self. The project's tests shouldn't pin unimplemented calls to a throw, and a null self isn't a valid call. Please drop those assertions, and pin source-backed values instead (for example setfields before initialize returns 0x80553502).
  • Debt. docs/dev/debt/libSceNpCppWebApi.md isn't updated. Any setter that still returns success without storing needs a Silent stubs entry, plus the provenance line the other namespaces have.

The field reads and stores are small enough to implement directly on a matching layout. Where you'd rather not, please document the gap instead of pinning a throw.

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/CMakeLists.txt
PR 7275 core/libs/CMakeLists.txt
PR 8650 core/libs/CMakeLists.txt
PR 9156 core/libs/CMakeLists.txt
PR 9293 core/libs/CMakeLists.txt
PR 9297 core/libs/CMakeLists.txt
PR 9302 core/libs/CMakeLists.txt
PR 9383 core/libs/CMakeLists.txt
PR 12642 core/libs/CMakeLists.txt
PR 13486 core/libs/CMakeLists.txt
core/libs/prx/libSceNpCppWebApi/Export.cpp
core/libs/prx/libSceNpCppWebApi/include/Api.hpp
docs/dev/debt/libSceNpCppWebApi.md
PR 14212 docs/dev/debt/libSceNpCppWebApi.md
PR 14798 core/libs/CMakeLists.txt
PR 15029 core/libs/prx/libSceNpCppWebApi/Export.cpp
docs/dev/debt/libSceNpCppWebApi.md
PR 15111 core/libs/CMakeLists.txt
PR 15836 core/libs/CMakeLists.txt
PR 15838 core/libs/CMakeLists.txt
PR 16436 core/libs/prx/libSceNpCppWebApi/Export.cpp

@edubox

edubox commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (91db3f79f7); head is now 26623a0535 (was 744f1938d5). The commit reapplied cleanly with no code changes.

Verified on this exact head (Linux x86-64, GCC 16.2.1, Release):

  • ctest --test-dir build -R '^guest_np_cpp_web_api' passes 10/10, including guest_np_cpp_web_api_catalog_leaderboards.
  • python3 tools/check_conventions.py --base origin/main: All conventions checks passed.
  • git diff --check origin/main...HEAD: clean.

AI-assisted: yes

@edubox

edubox commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (73ffb45110); head is now 25ab63ee5a (was 26623a0535). The commit reapplied cleanly with no code changes.

Verified on this exact head (Linux x86-64, GCC 16.2.1, Release):

  • ctest --test-dir build -R '^guest_np_cpp_web_api' passes 10/10, including guest_np_cpp_web_api_catalog_leaderboards.
  • python3 tools/check_conventions.py --base origin/main: All conventions checks passed.
  • git diff --check origin/main...HEAD: clean.

AI-assisted: yes

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch 2 times, most recently from 25ab63e to a0bc673 Compare October 30, 2026 21:20
@edubox

edubox commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (e584b5a7eb); head is now a0bc6734b8 (was 25ab63ee5a). The commit reapplied cleanly with no code changes.

Verified on this exact head (Linux x86-64, GCC 16.2.1, Release):

  • ctest --test-dir build -R '^guest_np_cpp_web_api' passes 10/10, including guest_np_cpp_web_api_catalog_leaderboards.
  • python3 tools/check_conventions.py --base origin/main: All conventions checks passed.
  • git diff --check origin/main...HEAD: clean.

AI-assisted: yes

@edubox

edubox commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (9605300f29); head is now 807e3b8897a. No source changes in the rebase.

Focused tests pass: ctest --test-dir build -R '^guest_np_cpp_web_api' 10/10 (including the new guest_np_cpp_web_api_catalog_leaderboards). python3 tools/check_conventions.py --base origin/main and git diff --check origin/main...HEAD are clean. Pushed with --force-with-lease.

AI-assisted: yes (Claude Code).

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from a0bc673 to 807e3b8 Compare October 30, 2026 22:22
@edubox

edubox commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (6b48aee) since the branch was 62 commits behind.

  • ctest --test-dir build -R '^guest_np_cpp_web_api': 10/10 passed.
  • python3 tools/check_conventions.py --base origin/main: passed.

Head is now 044e3fb783d.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch 2 times, most recently from 044e3fb to 30208c7 Compare October 31, 2026 02:03
@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (4b7af94e1d); the branch was 213 commits behind.

  • ctest --test-dir build -R '^guest_np_cpp_web_api': 10/10 passed.
  • python3 tools/check_conventions.py --base origin/main: passed.

Head is now 30208c76c99.

@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (2364892547); head is now 2609de9484. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from 30208c7 to 2609de9 Compare October 31, 2026 04:01
@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (3d1330179a); head is now 990b0a0ff1. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from 2609de9 to 990b0a0 Compare October 31, 2026 05:02
@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (de83e8f270); head is now 0423039423. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch 2 times, most recently from 0423039 to 68c7a3e Compare October 31, 2026 10:01
@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (45f4f82449); head is now 68c7a3edcc. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox

edubox commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (1847a6f9da); head is now 86bbf0c931. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from 68c7a3e to 86bbf0c Compare October 31, 2026 11:02
@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (1e24993fa3a); head is now d6eb5616e8. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from 13aaa08 to d6eb561 Compare November 5, 2026 06:57
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ AMD GPU test: passed

PR head d6eb561 merged onto main e772fb2 · tested 2026-11-05 14:53 UTC · 23 min

GPU AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 3,989 passed · 15 failed (0 new, 10 also fail on main, 5 unstable on main) · 71 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Vulkan validation ✅ no new validation IDs vs main (1417 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.4 s, ⚠️ agc_driver_graphics 90.8 s, ⚠️ agc_driver_graphics_maps_fallback 88.8 s, ⚠️ public_headers 68.4 s, ⚠️ agc_graphics_profile_draw 60.1 s
Unstable on main (5, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_job_manager_compute_capacity: failed in 8 of the last 10 main runs
agc_driver_pipeline_journal: failed in 9 of the last 10 main runs
agc_driver_two_sample_alpha_to_coverage_sample_mask_export: failed in 5 of the last 10 main runs
agc_submit_progress: failed in 9 of the last 10 main runs
agc_submit_progress_no_timeline: failed in 9 of the last 10 main runs
Failing on main too (10, not counted against this PR)
agc_driver_alpha_to_coverage_sample_mask_export
agc_driver_guest_memory_lifetime_direct
agc_driver_image_table
agc_driver_native_depth_bda_clear
agc_driver_native_depth_buffer_clear
agc_driver_native_depth_clears
agc_driver_native_multisample_depth
agc_driver_submit_progress
guest_mmap_executable
guest_socket_options_multicast
Passed on retry (2, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)
agc_driver_emulated_color_compare: Timeout in the parallel run; also passed only on retry in 53 other runs in the last 7 days
agc_graphics_profile_draw: Timeout in the parallel run; also passed only on retry in 4 other runs in the last 7 days
Validation IDs already present on main (4 IDs in 812 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (812 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_draw_space_reuse, agc_driver_address_draw_space_reuse_held, agc_driver_address_draw_space_reuse_off, agc_driver_address_draw_space_reuse_transitions, agc_driver_address_draw_space_reuse_verify, agc_driver_address_passes, and 804 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (71)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_copied_writer_label_order_drain: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_gcn_display_present: skipped, no display: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
agc_driver_image_table_vouched_lookups: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a watched imported table cannot be vouched for
agc_driver_linear_target_direct_write_completion: skipped, the device does not keep host imports write-watched, so no completion stamp over the target is tracked
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes
agc_driver_recorded_draw_staging: skipped, the device does not keep host imports write-watched, so the target is never resident
agc_driver_scattered_runs: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_driver_unit_shadow_budget: skipped, the device does not import watched host memory for unit shadows
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_idle_publish_wait: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a forced watch of the import reads its submissions as CPU stores
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_shadowed_key_stores: skipped, the device imports no watched host memory under a unit shadow
agc_small_tiled_shadow_label: skipped, host imports are unavailable: the test block is not imported and watched
agc_storage_clear_hand_over: skipped, the device imports no watched host memory for the clear-key hand-over
agc_storage_pending_key_refresh: skipped, refreshes over a pending key store are not testable here
agc_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_net_ctl_disconnected: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_device_capability: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_disabled: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROL
Reproduce
git fetch origin d6eb5616e81a1a9a06c9ce5ea13bb749b3b45836 e772fb25cc72ee5faf77847f06bbc69533016e82
git checkout e772fb25cc72ee5faf77847f06bbc69533016e82 && git merge --no-ff d6eb5616e81a1a9a06c9ce5ea13bb749b3b45836
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (03cfa4c0679); head is now 9da2f25d11. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from d6eb561 to 9da2f25 Compare November 5, 2026 15:02
@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Addressed at head 9da2f25. The 15 accessors now read the offsets and widths measured in the bundled module instead of throwing, the request-body setters store the value plus the is-set flag, and ParameterToGetBoardDefinition::setfields returns 0x80553502 before initialization and 0x80553503/0x80553504 on the argument/copy failures. The test drops the throw and null-self pins and asserts the measured fields, flags and parameter lifecycle instead; docs/dev/debt/libSceNpCppWebApi.md records the host allocator standing in for the module LibContext allocator. guest_np_cpp_web_api_catalog_leaderboards passes locally, and the hosted conventions and windows jobs are green.

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ AMD GPU test: passed

PR head 9da2f25 merged onto main 813f46c · tested 2026-11-05 19:48 UTC · 29 min

GPU AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 4,014 passed · 13 failed (0 new, 12 also fail on main, 1 unstable on main) · 71 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Vulkan validation ✅ no new validation IDs vs main (1484 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.2 s, ⚠️ public_headers 76.1 s, ⚠️ agc_driver_graphics 70.5 s, ⚠️ agc_driver_graphics_maps_fallback 70.3 s, ⚠️ agc_driver_bda_device 60.3 s
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_two_sample_alpha_to_coverage_sample_mask_export: failed in 6 of the last 10 main runs
Failing on main too (12, not counted against this PR)
agc_driver_alpha_to_coverage_sample_mask_export
agc_driver_guest_memory_lifetime_direct
agc_driver_image_table
agc_driver_native_depth_bda_clear
agc_driver_native_depth_buffer_clear
agc_driver_native_depth_clears
agc_driver_native_multisample_depth
agc_driver_pipeline_journal
agc_driver_submit_progress
agc_graphics_profile_draw
guest_mmap_executable
guest_socket_options_multicast
Validated but not compared (67, not in the main baseline's validation pass)
guest_http2_http1_cookies
guest_job_manager_worker_spin
public_headers
public_headers_compile_0
public_headers_compile_1
public_headers_compile_10
public_headers_compile_11
public_headers_compile_12
public_headers_compile_13
public_headers_compile_14
public_headers_compile_15
public_headers_compile_16
public_headers_compile_17
public_headers_compile_18
public_headers_compile_19
public_headers_compile_2
public_headers_compile_20
public_headers_compile_21
public_headers_compile_22
public_headers_compile_23
public_headers_compile_24
public_headers_compile_25
public_headers_compile_26
public_headers_compile_27
public_headers_compile_28
public_headers_compile_29
public_headers_compile_3
public_headers_compile_30
public_headers_compile_31
public_headers_compile_32
public_headers_compile_33
public_headers_compile_34
public_headers_compile_35
public_headers_compile_36
public_headers_compile_37
public_headers_compile_38
public_headers_compile_39
public_headers_compile_4
public_headers_compile_40
public_headers_compile_41
public_headers_compile_42
public_headers_compile_43
public_headers_compile_44
public_headers_compile_45
public_headers_compile_46
public_headers_compile_47
public_headers_compile_48
public_headers_compile_49
public_headers_compile_5
public_headers_compile_50
public_headers_compile_51
public_headers_compile_52
public_headers_compile_53
public_headers_compile_54
public_headers_compile_55
public_headers_compile_56
public_headers_compile_57
public_headers_compile_58
public_headers_compile_59
public_headers_compile_6
public_headers_compile_60
public_headers_compile_61
public_headers_compile_62
public_headers_compile_63
public_headers_compile_7
public_headers_compile_8
public_headers_compile_9
Validation IDs already present on main (4 IDs in 812 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (812 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_draw_space_reuse, agc_driver_address_draw_space_reuse_held, agc_driver_address_draw_space_reuse_off, agc_driver_address_draw_space_reuse_transitions, agc_driver_address_draw_space_reuse_verify, agc_driver_address_passes, and 804 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store_image_copy, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_partially_unmapped_plane, agc_driver_depth_plane_unmapped_plane, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (71)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_copied_writer_label_order_drain: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_gcn_display_present: skipped, no display: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
agc_driver_image_table_vouched_lookups: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a watched imported table cannot be vouched for
agc_driver_linear_target_direct_write_completion: skipped, the device does not keep host imports write-watched, so no completion stamp over the target is tracked
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes
agc_driver_recorded_draw_staging: skipped, the device does not keep host imports write-watched, so the target is never resident
agc_driver_scattered_runs: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_driver_unit_shadow_budget: skipped, the device does not import watched host memory for unit shadows
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_idle_publish_wait: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a forced watch of the import reads its submissions as CPU stores
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_shadowed_key_stores: skipped, the device imports no watched host memory under a unit shadow
agc_small_tiled_shadow_label: skipped, host imports are unavailable: the test block is not imported and watched
agc_storage_clear_hand_over: skipped, the device imports no watched host memory for the clear-key hand-over
agc_storage_pending_key_refresh: skipped, refreshes over a pending key store are not testable here
agc_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_net_ctl_disconnected: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_device_capability: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_disabled: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROL
Reproduce
git fetch origin 9da2f25d11bffa7d350cd8c6998f8045d8491956 813f46cd685f49aafa9ccac7ca7d65b9179b4974
git checkout 813f46cd685f49aafa9ccac7ca7d65b9179b4974 && git merge --no-ff 9da2f25d11bffa7d350cd8c6998f8045d8491956
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

…rds fields

The 15 response/model accessors now read the offsets and widths measured in
the module bundled with PPSA21564/PPSA23732 (builds J03093737/J03341606)
instead of throwing, and the request-body setters store the value plus an
is-set flag. ParameterToGetBoardDefinition and ParameterToGetContainer now
allocate a host implementation so setfields and the two Container setters
can store through +0x8; setfields returns 0x80553502 before initialize and
0x80553503/0x80553504 when the string copy fails. The host allocator stands
in for the module's LibContext allocator, which the debt records.

The test pins the measured offsets, the is-set flags, the vector reference
count increment and the parameter lifecycle/error codes instead of asserting
throws or null-self success.
@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased automatically onto current main (151a2cb6e27); head is now 05f1a18310. Focused tests pass; check_conventions.py --base origin/main and git diff --check are clean.

@edubox
edubox force-pushed the feat/npcpp-catalog-leaderboards branch from 9da2f25 to 05f1a18 Compare November 5, 2026 20:03
@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

The review points are implemented at head 05f1a18310: the 15 accessors read the measured offsets/widths instead of throwing, the request-body setters store the value plus the is-set flag, ParameterToGetBoardDefinition/ParameterToGetContainer allocate a host implementation and store through +0x8 (including the 0x80553502/0x80553503/0x8055350f/0x80553510 lifecycle codes), and GuestNpCppWebApiCatalogLeaderboards.cpp pins those measured values instead of throw/null-self assertions. The remaining stateless setters are documented as silent stubs in docs/dev/debt/libSceNpCppWebApi.md, with the module builds and the host-allocator gap recorded.

Verified on this head: guest_np_cpp_web_api_catalog_leaderboards passes, and check_conventions.py --base origin/main reports 0 errors (two debt-length warnings on the edited entries). AI-assisted: yes.

…aderboards

# Conflicts:
#	core/libs/prx/libSceNpCppWebApi/Export.cpp
#	docs/dev/debt/libSceNpCppWebApi.md
@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Rebased the merge conflict onto current main (merge commit 7d5773e6da). Conflicts:

  • core/libs/prx/libSceNpCppWebApi/Export.cpp: this branch already moved the ParameterObject helpers to the top of the file, so I removed main's duplicate copy and reused ParameterOutOfMemory for the three allocation-failure returns that main's new vector code introduced.
  • docs/dev/debt/libSceNpCppWebApi.md: kept main's updated Vector/Iterator<IntrusivePtr<BasicPresence>> wording and this PR's new Leaderboards::V1 and InGameCatalog::V5 measured fields entry.

Verified on the new head: guest_np_cpp_web_api_catalog_leaderboards passes, git diff --check is clean, and python3 tools/check_conventions.py --base origin/main reports 0 errors (two existing debt-length warnings). AI-assisted: yes.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Nov 5, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 5, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Nov 6, 2026
…merged from main

Co-authored-by: edubox <3165165+edubox@users.noreply.github.com>
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Hi @edubox, main moved under this branch (#14136 gave the BasicPresence accountIdIsSet declaration a signature in Api.hpp), so I merged main into it. The one conflict was in libSceNpCppWebApi/include/Api.hpp: the merge keeps main's prototyped accountIdIsSet declaration and all of your Leaderboards and InGameCatalog declarations. #14136 also added four list-parser lines in Export.cpp that still used the old ParameterAllocationFailed name your branch renamed to ParameterOutOfMemory (same 0x80553504 value), so I switched those four to your name in a second commit. None of your own code changed, and the NpCppWebApi tests pass on the merged tree.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 6, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Nov 6, 2026
…aderboards

# Conflicts:
#	core/libs/prx/libSceNpCppWebApi/Export.cpp
#	docs/dev/debt/libSceNpCppWebApi.md
@edubox

edubox commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Merged current main (1da78eb048) to clear the conflict.

  • Export.cpp: kept main's newer vector-helper logic (direct-body/adopted-body handling) with this branch's ParameterOutOfMemory name, and dropped the reference-count constants main now defines once.
  • docs/dev/debt/libSceNpCppWebApi.md: kept main's wording for the DataStatus/header getters that main now implements, plus this branch's measured Container parameter/accessor detail.

Verified on the merge: guest_np_cpp_web_api* 15/15 pass, check_conventions.py --base origin/main reports 0 errors (2 debt-length warnings), and git diff --check is clean. AI-assisted: yes.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants