Skip to content

fix(module): resolve the default dlsym scope against the main program - #864

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
jeffrey12cali:fix/dlsym-default-scope
Oct 8, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
jeffrey12cali:fix/dlsym-default-scope

Conversation

@jeffrey12cali

@jeffrey12cali jeffrey12cali commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

What

Titles pass handle 0 (RTLD_DEFAULT) for a process-wide lookup, and the loader rejected it: the handle is not in the module map, so every such lookup failed even when a loaded module exports the symbol. Handle 0 and -2 now take the default scope - the global modules by plain name, NID and #guest form, then the process-wide scope.

The process-wide scope of the real loader also contains glibc and libstdc++, so an unfiltered fallback would answer malloc, dlopen and pthread_create where PS5 reports the symbol as missing. Handle 0 is the main program, whose lookup sees the program's exports and not the host C++ runtime (same model as KytyPS5's FindProgramById(handle)->export_symbols). A result from the process scope is therefore accepted only when dladdr attributes it to a converted or replacement module - a .prx library or the executable.

The scripting heap from the previous revision of this branch is removed; see the reply to the review for the measurement that decided it.

Tested

Arch Linux, GCC, Release, cmake -DBUILD_TESTING=ON: ctest 392/392 passed on this head.

guest_dynamic_loader now asserts both directions of the change. Handle 0 must resolve a guest export (dlsym(0, "GuestModuleAdd")), and the host names malloc, dlopen and pthread_create must stay null. Verified by reverting the feature itself, not a variant of it: with DynamicLoader.cpp taken from main, the test fails (0% tests passed, 1 tests failed). With this branch's loader it passes.

One gap is named rather than hidden. GuestModuleAdd is found by the module loop before the process-scope fallback runs, so no assertion exercises DefaultScopeSymbol itself yet. That case needs a module that is in the process scope but absent from the loader's module map, which this fixture set does not provide.

On a title (PPSA28997, native Linux relink): booting with the scripting heap removed reaches the same milestone as with it - managed bootstrap completes, level0/level1 open, no abort - which is why the export was dropped rather than documented.

Checklist

One note on freshness: I could not rebase onto the current tip - pushing main's commits that touch .github/workflows/conventions.yml to my fork is refused because the OAuth token has no workflow scope - so this head is based on 931109b3. I will rebase as soon as the fork's main is synced, and re-run the suite after.

Refs #694.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and ran ctest merged onto current main (Linux, all green); guest_dynamic_loader fails without the change, good. A few things before this can go in:

  • Handle 0 reaches host symbols. On Linux MainProgramModule() is ::dlopen(nullptr, ...) (DynamicLoader.cpp:73), i.e. the whole process scope, and FindSymbol tries the plain name before the NID. A probe in guest_dynamic_loader_tests gets non-null for dlsym(0, "pthread_create"), "dlopen" and "malloc": host glibc wins over the guest exports. This contradicts "nothing reaches for host glibc" in the description. On PS5, handle 0 is the main program and the lookup only searches its export table (KytyPS5 KernelDlsym, FindProgramById(handle)->export_symbols). Handle 0 should only see guest exports (NID / #guest names), not the host process scope.
  • The references disagree on scriptingGetMem. prosper (docs/archive/ROADMAP_HISTORY.md, IL2CPP bring-up) made sceKernelDlsym return ESRCH for scriptingGetMem "instead of fake success, keeping the caller's fallback", and the title then finishes il2cpp_init. Only KytyPS5 special-cases the name. Since the name exists in none of the title's binaries, ESRCH looks like what real hardware does. If you have evidence that the export exists on PS5, please cite it. Otherwise the default-scope fix alone (with an honest not-found) may be enough. Please check whether your title still boots that way.
  • Windows. GetModuleHandleW(nullptr) only covers the executable, while scriptingGetMem lives in the libc DLL, so the new assertions in guest_dynamic_loader (also registered on Windows) look like they will fail there. CI hasn't run on this head yet.
  • If the export stays: the (alignment, bytes) signature is inferred, so it needs a line under "Unknown function info" in docs/dev/TechnicalDebt.md. The test should also check the requested alignment (& 0xFFF), not just 16 bytes (GuestDynamicLoader.cpp:55), and cover scriptingFreeMem.

@jeffrey12cali

Copy link
Copy Markdown
Contributor Author

Thank you for the careful review - the handle-0 finding is a real bug and the scriptingGetMem challenge is correct. I measured it before replying, and the answer changes what this PR should contain.

scriptingGetMem is not needed. Removing it. I gated both exports out of the vendored libc (so the lookup fails honestly, as prosper's docs/archive/ROADMAP_HISTORY.md describes) and re-ran PPSA28997 with scripted input:

with the export without it
managed bootstrap milestone ("Finished Initializing …") reached reached
scene opens level0, level1 level0, level1
stdout lines in the same window 323 314
crash / abort / unhandled none none

So I have no evidence the export exists on PS5 and no evidence the title needs it - your reading is right and the fake allocator was my inference, not a measurement. I will rework this PR to be the default-scope fix only, with an honest not-found, and drop the scripting heap (kept locally as a patch in case a later symptom points at it). If it comes back at all it goes in with the (alignment, bytes) signature recorded under "Unknown function info" and tests covering the alignment and scriptingFreeMem, as you said.

Handle 0 reaching host symbols is a genuine defect and I will fix it, not document it. You are right that it contradicts my description, and the probe is decisive: dlsym(0, "pthread_create"), "dlopen", "malloc" returning non-null means the host process scope wins over guest exports. MainProgramModule() being dlopen(nullptr) is exactly the wrong scope for PS5 semantics, where handle 0 is the main program's own export table (same model as KytyPS5's FindProgramById(handle)->export_symbols). The fix I intend: for handle 0, resolve only against the main program's guest exports - the #guest/NID names - and never fall through to the host scope, so a plain name that only exists in glibc resolves to nothing rather than to glibc. I will add those three names to guest_dynamic_loader_tests as negative assertions so this cannot regress.

Windows. Agreed, and I had not verified it: GetModuleHandleW(nullptr) covers only the executable while the libc exports live in another module, so the new assertions should fail there. I will gate the assertions to the platform whose semantics they test, or make the Windows lookup cover the runtime modules as well - whichever the handle-0 model ends up being - and get CI green on the head before asking for another look.

One question so I fix the right thing: for handle 0, should a name that exists both as a guest export and as a host symbol resolve to the guest (my reading of PS5 semantics), and should RTLD_DEFAULT-style guest lookups ever see the host process scope at all, or only the guest link map list?

@jeffrey12cali
jeffrey12cali force-pushed the fix/dlsym-default-scope branch 2 times, most recently from 654f9c4 to 7f5eacc Compare October 7, 2026 17:38
@jeffrey12cali

Copy link
Copy Markdown
Contributor Author

All four points accepted; the head is rewritten. 7f5eacc4, two files, +29/-1.

Handle 0 no longer reaches host symbols. The default-scope fallback is now filtered by dladdr: a process-scope result is accepted only when it comes from a converted or replacement module (a .prx library or the executable). dlsym(0, "malloc"), "dlopen" and "pthread_create" are asserted null in guest_dynamic_loader. I checked that the assertion actually pins the behaviour rather than merely passing: replacing the filter with a bare ::dlsym(RTLD_DEFAULT, ...) makes guest_dynamic_loader fail, and restoring it makes it pass. Your FindProgramById framing is what I implemented, minus the per-program table, because on Linux the guest module list plus a filtered process scope is the same set.

scriptingGetMem was an inference, and it is gone. You were right to push on it. I gated both exports out so the lookup fails honestly and re-ran PPSA28997 with scripted input:

with the export without it
managed bootstrap milestone reached reached
scene opens level0, level1 level0, level1
stdout in the same window 323 314
crash / abort / unhandled none none

So the default-scope fix stands on its own, the scripting heap is dropped from this PR, and no TechnicalDebt line is needed for an inferred signature that no longer exists. I have the code parked locally if a later symptom ever points back at it, but prosper's ESRCH reading is now the one I have evidence for.

Windows. The assertions and the process-scope fallback are both inside #ifndef _WIN32 now, so nothing on that platform depends on GetModuleHandleW(nullptr) covering the runtime modules. If the default scope should also exist on Windows, that is a separate question about the PE import model and I would rather raise it in its own PR than smuggle it in here.

Comments. Removed - you were right that the description claimed there were none while the diff carried them. The head has no added comments.

CI. Not green on the head yet, and one blocker I cannot fix from my side: I cannot rebase onto current main because pushing main's commits that touch .github/workflows/conventions.yml to my fork is refused (the OAuth token has no workflow scope). This head is therefore based on 931109b3; I will rebase once the fork's main is synced and re-run the suite. If there is a preferred way to handle that for contributor forks, tell me and I will use it.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and ran ctest merged onto current main (Linux, 400/400 green). Three things left:

  • The test doesn't cover the fix. With the DynamicLoader.cpp changes reverted, guest_dynamic_loader still passes: on main handle 0 is rejected outright, so dlsym(0, "malloc") etc. are already null (GuestDynamicLoader.cpp:92-94). Your "verified to bite" check swapped in a different broken version, not main. The new behaviour, handle 0 finding an export, isn't asserted anywhere (handle -2 on a global module was already covered at line 40). Please add positive cases: dlsym(0, "GuestModuleAdd") on the RTLD_GLOBAL module from the test, and one export that is only reachable through the process-scope fallback (e.g. a .prx export by NID).
  • leaf == "app.elf" (DynamicLoader.cpp:59) depends on the file name. The output name is the user's choice (relinker input.elf output.elf; app.elf in USAGE.md is only an example), so a renamed executable has its exports silently rejected. Identify the main program by the object itself instead, e.g. compare info.dli_fbase with the main executable's base (dladdr on a symbol of the executable, or the first dl_iterate_phdr entry).
  • CI / rebase: no checks have run on this head. Pushing a branch that contains main's workflow commits needs the workflow scope (gh auth refresh -s workflow), or use "Sync fork" in the GitHub UI and rebase on that.

@Dextroid17

Copy link
Copy Markdown
Contributor

@adriadam10's first point — "the new behaviour, handle 0 finding an export, isn't asserted anywhere" — is a one-line addition next to the handle -2 check he notes is already covered (GuestDynamicLoader.cpp:40):

    Require(dlsym_nid_postfix(reinterpret_cast<void*>(-2), "GuestModuleAdd") == reinterpret_cast<void*>(add));
    Require(dlsym_nid_postfix(nullptr, "GuestModuleAdd") == reinterpret_cast<void*>(add));

Nothing else changes.

Verified by reverting only DynamicLoader.cpp, keeping the test (applied onto a rebase against current main, since no checks have run on this head):

build result
with the change 100% tests passed out of 1
DynamicLoader.cpp reverted 1/1 Test #343: guest_dynamic_loader ..... Subprocess aborted — 0% tests passed

The revert is confirmed to be a real one: grep -c 'handle == nullptr' DynamicLoader.cpp goes from 2 to 0 before the rebuild. On main, handle 0 never enters the search at all, so the new assertion fails there and passes with the change — which is the property the first attempt was missing.

Two honest limits.

  • I did not test Windows. The test is registered there too and I have no way to run it; the reviewer's own warning about guest_dynamic_loader on Windows stands, and this line is subject to it.
  • I did not add the second case he asked for — an export reachable only through the process-scope fallback. GuestModuleAdd is found by the module loop before DefaultScopeSymbol is reached, so it exercises handle 0 being accepted, not the fallback itself. A genuine case needs a symbol the module scan misses; I could not construct one from this test's fixtures and would rather say so than add an assertion that passes for the wrong reason.

The remaining points on the review — handle 0 reaching host symbols, the leaf == "app.elf" file-name dependence, and the scriptingGetMem question — are unaffected by this and still need your call. This only removes the "test doesn't cover the fix" blocker.

Happy to open a PR against jeffrey12cali:fix/... with exactly this if that is easier than applying it by hand; otherwise the line above is the whole change and it is yours to use.

@jeffrey12cali
jeffrey12cali force-pushed the fix/dlsym-default-scope branch from 7f5eacc to c3ffd7f Compare October 8, 2026 00:10
@jeffrey12cali

Copy link
Copy Markdown
Contributor Author

You were right, and my earlier "verified to bite" claim was wrong. I checked it your way this time. Head is c3ffd7f1.

What I did wrong. My check replaced the filter with a bare ::dlsym(RTLD_DEFAULT, ...). That tests a different broken build, not main. On main, handle 0 never enters the search, so the three host-name assertions pass without any change. I have now confirmed that directly: I copied DynamicLoader.cpp from upstream/main (grep -c 'handle == nullptr' returns 0), kept the test, rebuilt, and guest_dynamic_loader passed. The assertion did not pin the fix.

What I added. The positive case @Dextroid17 proposed:

    Require(dlsym_nid_postfix(nullptr, "GuestModuleAdd") == reinterpret_cast<void*>(add));

Verified in both directions on this head:

build result
DynamicLoader.cpp from main, new tests kept 0% tests passed, 1 tests failed
this branch's loader 100% tests passed out of 1
full suite with the fix 100% tests passed out of 392

One limit I share with @Dextroid17. GuestModuleAdd is found by the module loop before DefaultScopeSymbol runs. So the new assertion proves handle 0 is accepted and resolves, but it does not prove the process-scope fallback works. I also could not build a case that reaches only the fallback from this test's fixtures. A real one needs a module that is in the process scope but not in the loader's module map, which the current fixture set does not give us. If you can point at a shape that does, I will add it. I would rather leave the gap named than add an assertion that passes for the wrong reason.

The file-name dependency is gone. leaf == "app.elf" was wrong, because the output name is the user's choice. The main program is now identified by its load base: dl_iterate_phdr records the entry with an empty dlpi_name, and FromGuestModule compares info.dli_fbase against it. .prx files are still accepted by suffix. I could not test the renamed-executable case, because the test binary exports no guest symbols. If you want that covered, a fixture executable with an export would do it, and I will add it.

CI and rebase. I still cannot rebase onto current main. Pushing a branch that carries main's commits for .github/workflows/conventions.yml is refused, because my token has no workflow scope. I did not try gh auth refresh -s workflow, because it needs an interactive browser authorisation from this session. Two options, and I will take whichever you prefer: you run "Sync fork" on jeffrey12cali/AnyPS5, and I rebase and re-run, or I authorise the extra scope locally and push. This head is based on 931109b3.

@Dextroid17

Copy link
Copy Markdown
Contributor

Thanks — the corrected negative control is exactly the check that was missing, and the table is the shape I would want: loader from main with the new tests kept, 1 failed; this branch, 100% passed. That is what separates a test that pins the fix from one that passes on both sides, and your description of why the earlier attempt tested a different broken build (::dlsym(RTLD_DEFAULT, …) never reaching handle == nullptr) is the useful part of the record.

Agreed on the limit, and I would leave it named rather than papered over. GuestModuleAdd resolving through the module loop before DefaultScopeSymbol runs is a real gap in what the assertion proves, and an assertion that passes for the wrong reason is worse than an acknowledged hole. The shape that would close it — a module present in the process scope but absent from the loader's module map — is not something this fixture set gives either of us, so I have nothing better to offer than the honest note.

On the rebase blocker, the facts are on your side. GitHub refuses any push that modifies .github/workflows/ from a credential without the workflow scope, and that check is on the credential, not the branch — there is no rebase-side workaround, which is why the two routes you listed are the only ones:

  • gh auth refresh -s workflow gives your token the scope in this session, after which the rebase and push go through normally. It is interactive, as you say, but it is the route that unblocks you without anyone else.
  • A sync of jeffrey12cali/AnyPS5 from main performs the same update server-side, so your branch can then be rebased onto your own main without you ever pushing a workflow file.

I have no preference between them and it is not mine to set — either gets the head onto current main, and the choice belongs to the maintainer. What I would not do is drop the workflow file from the branch to work around it, since that is the file the conventions job runs from.

Nothing else from me on the code. The load-base identification (dlpi_name empty, compared via dli_fbase) is a better anchor than the output file name, and I could not test the renamed-executable case either, for the same reason you give.

Written with AI assistance.

@fisherjoey

Copy link
Copy Markdown
Contributor

This seems to cover the handle == 0 part of #694, where dlsym(NULL, name) returns nil. Adding Refs #694 to the description would link them. The DT_NEEDED visibility part of #694 is not mentioned in this PR.

@hikazey

hikazey commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

On the remaining gap, a test where only the process-scope fallback can resolve the symbol: the test executable itself can provide one. DefaultScopeSymbol accepts symbols whose dli_fbase is the main program, and the main program isn't in the loader's module map (the dlopen(nullptr) handle is closed at line 37). So an export defined in GuestDynamicLoader.cpp can only be found through the fallback once the executable exports its symbols (ENABLE_EXPORTS, non-Windows only):

--- a/core/libs/prx/libkernel/CMakeLists.txt
+++ b/core/libs/prx/libkernel/CMakeLists.txt
@@ add_dependencies(guest_dynamic_loader_tests guest_module_fixture)
+    if(NOT WIN32)
+        set_target_properties(guest_dynamic_loader_tests PROPERTIES ENABLE_EXPORTS ON)
+    endif()
--- a/core/libs/tests/GuestDynamicLoader.cpp
+++ b/core/libs/tests/GuestDynamicLoader.cpp
@@ static void Require(bool value) { if (!value) std::abort(); }
+#ifndef _WIN32
+extern "C" int APS5_VABI GuestDefaultScopeOnly(int a, int b) { return a * b; }
+#endif
@@     Require(dlsym_nid_postfix(nullptr, "pthread_create") == nullptr);
+    Require(dlsym_nid_postfix(nullptr, "GuestDefaultScopeOnly") == reinterpret_cast<void*>(&GuestDefaultScopeOnly));
+    Require(dlsym_nid_postfix(reinterpret_cast<void*>(-2), "GuestDefaultScopeOnly") == reinterpret_cast<void*>(&GuestDefaultScopeOnly));

Checked on Ubuntu 24.04, GCC 14.2, on top of c3ffd7f1:

build guest_dynamic_loader
this branch's loader + the test passed
DynamicLoader.cpp from main, test kept Subprocess aborted
this branch's loader with both DefaultScopeSymbol calls removed (handle 0 still accepted) Subprocess aborted

The third row is the one that was missing: the test fails when only the fallback is removed. Full suite on this head plus the test: 392/392 passed. tools/check_conventions.py passes on the commit. Not tested on Windows; the case is #ifndef _WIN32 like the fallback itself.

Feel free to take it into this PR. AI-assisted: yes (Claude Code).

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and ran ctest merged onto current main (413/413 green); with DynamicLoader.cpp reverted, guest_dynamic_loader now fails, so the handle-0 → guest export assertion pins the fix. Thanks for redoing that check.

One thing left:

  • The new fallback is never executed by any test. MainProgramBase, FromGuestModule and DefaultScopeSymbol (DynamicLoader.cpp:54-84, call site :163-167) are only reached when the module loop finds nothing, and the three host-name asserts (GuestDynamicLoader.cpp:93-95) pass identically without that path, so the dladdr filter itself is unpinned. The existing fixture can cover it: ::dlopen the test guest .prx directly with RTLD_GLOBAL (so it is not in the loader's module map) and assert dlsym_nid_postfix(nullptr, "<its export>") returns it, and that a host symbol still returns null.
  • Minor: info.dli_fbase == dlpi_addr (:75) only matches for PIE executables (non-PIE has dlpi_addr == 0). Either compare against the executable's actual mapping start or note the PIE assumption in TechnicalDebt.

Not verified on our side: the title run (PPSA28997).

@hikazey

hikazey commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Following up with @adriadam10's suggestion (host dlopen of the fixture with RTLD_GLOBAL). Writing it turned up a bug: the .prx branch of FromGuestModule never matches.

return path.size() > 5 && path.compare(path.size() - 5, 5, ".prx") == 0;

".prx" is 4 characters, so this compares the path's last 5 characters ("t.prx") with it, and the result is never 0. On this head the fallback therefore only returns symbols from the main program. Symbols from DT_NEEDED-loaded .prx libraries, the #694 case, are still rejected. I traced it inside DefaultScopeSymbol: ::dlsym(RTLD_DEFAULT, "3qcG+yBRS+s") returns the export from /…/anyps5-host-loaded-module-for-test.prx, and FromGuestModule then rejects it.

Fix plus test (on top of c3ffd7f1 and my previous comment's test):

@@ -74,7 +74,7 @@ bool FromGuestModule(void* address) {
     if (address == nullptr || !::dladdr(address, &info) || info.dli_fname == nullptr) return false;
     if (reinterpret_cast<std::uintptr_t>(info.dli_fbase) == MainProgramBase()) return true;
     const std::string path(info.dli_fname);
-    return path.size() > 5 && path.compare(path.size() - 5, 5, ".prx") == 0;
+    return path.ends_with(".prx");
 }
 
 void* DefaultScopeSymbol(const char* symbolName) {
@@ -4,6 +4,9 @@
 #include <fstream>
 #include <stdexcept>
 #include <thread>
+#ifndef _WIN32
+#include <dlfcn.h>
+#endif
 extern "C" {
 void* APS5_VABI dlopen_nid_postfix(const char*, int);
 void* APS5_VABI dlsym_nid_postfix(void*, const char*);
@@ -98,5 +101,15 @@ int main(int argc, char** argv) {
     Require(dlsym_nid_postfix(nullptr, "pthread_create") == nullptr);
     Require(dlsym_nid_postfix(nullptr, "GuestDefaultScopeOnly") == reinterpret_cast<void*>(&GuestDefaultScopeOnly));
     Require(dlsym_nid_postfix(reinterpret_cast<void*>(-2), "GuestDefaultScopeOnly") == reinterpret_cast<void*>(&GuestDefaultScopeOnly));
+    const auto hostLoaded = std::filesystem::absolute("anyps5-host-loaded-module-for-test.prx");
+    std::filesystem::copy_file(argv[1], hostLoaded, std::filesystem::copy_options::overwrite_existing);
+    void* host = ::dlopen(hostLoaded.c_str(), RTLD_NOW | RTLD_GLOBAL);
+    Require(host != nullptr);
+    void* hostAdd = ::dlsym(host, "3qcG+yBRS+s");
+    Require(hostAdd != nullptr);
+    Require(dlsym_nid_postfix(nullptr, "GuestModuleAdd") == hostAdd);
+    Require(dlsym_nid_postfix(nullptr, "malloc") == nullptr);
+    Require(::dlclose(host) == 0);
+    std::filesystem::remove(hostLoaded);
 #endif
 }

The fixture is libguest_module_fixture.so, so the test copies it to a .prx name, as the .guest.prx case already does. It exports GuestModuleAdd only as its NID, so the case also covers the NID retry.

guest_dynamic_loader (Ubuntu 24.04, GCC 14.2):

loader result
this head + the one-line fix passed
this head as is aborted (the suffix bug)
main aborted
fallback calls removed aborted
NID retry removed aborted
.prx check returns false aborted
filter removed (any RTLD_DEFAULT result accepted) aborted

Full suite with the fix and both tests: 392/392 passed. check_conventions.py passes. Not tested on Windows; the new case is inside the existing #ifndef _WIN32 block.

I didn't address the PIE point (dli_fbase vs dlpi_addr for a non-PIE executable). Both test executables here are PIE.

AI-assisted: yes (Claude Code).

@Dextroid17

Copy link
Copy Markdown
Contributor

Confirmed from the head, and the bug is exactly as you describe. FromGuestModule ends with

return path.size() > 5 && path.compare(path.size() - 5, 5, ".prx") == 0;

compare(pos, count, s) compares count characters of the substring against the NUL-terminated ".prx" (4 characters), so the comparison can never return 0 for a 5-character slice like "t.prx" — the whole guard is path.size() > 5 && false. On this head the .prx branch therefore never matches and DefaultScopeSymbol only ever returns main-program symbols; the DT_NEEDED case from #694 stays rejected, which matches your trace inside the function. path.ends_with(".prx") is the right fix, and your table (aborts when the .prx check returns false, passes with it) is the assertion the earlier threads were missing.

That also closes the fallback gap I noted a few comments up — between your GuestDefaultScopeOnly case and this one, both halves of DefaultScopeSymbol are now pinned.

Same limitation on my side: this is from reading the head, not a rebuild (my box is NVIDIA-only), and I can't run the Windows path either. The case is inside the existing #ifndef _WIN32 block, so nothing there depends on GetModuleHandleW.

Written with AI assistance.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Built and ran ctest merged onto current main (440/440 green); with DynamicLoader.cpp reverted, guest_dynamic_loader fails, so the handle-0 assertions pin the change. Two things left:

  • The .prx branch of FromGuestModule never matches (DynamicLoader.cpp:77): path.compare(path.size() - 5, 5, ".prx") compares 5 characters against a 4-character string, so it is never 0. The fallback therefore only returns symbols from the main program, and symbols from .prx libraries (the #694 case) are still rejected. path.ends_with(".prx") fixes it.
  • The fallback is still not executed by any test. DefaultScopeSymbol is reached only when the module loop finds nothing, and the malloc/dlopen/pthread_create asserts pass without it. hikazey posted two tests in this thread that cover it: an exported symbol in the test executable (ENABLE_EXPORTS, non-Windows) and a host dlopen(..., RTLD_GLOBAL) of the fixture. Please add them (with the .prx fix, the second one fails without it) and check they fail with both DefaultScopeSymbol calls removed.

Also add Refs #694 to the description, as fisherjoey suggested.

@jeffrey12cali
jeffrey12cali force-pushed the fix/dlsym-default-scope branch 2 times, most recently from cf33f7d to 5aa7a5b Compare October 8, 2026 11:53
@jeffrey12cali

Copy link
Copy Markdown
Contributor Author

Both points are fixed, and I verified each control myself. Head is 5aa7a5b9, three files, +67/-1.

The .prx branch was dead code, and you were both right. path.compare(path.size() - 5, 5, ".prx") compares five characters against a four-character string, so it can never return 0. I confirmed it in isolation before changing anything:

compare(size-5, 5, ".prx") == 0 ? 0      <- the branch never matched
compare(size-4, 4, ".prx") == 0 ? 1
ends_with(".prx")                ? 1

FromGuestModule now uses path.ends_with(".prx"). This was my regression: the earlier revision used ends_with, and my rewrite to identify the main program by load base replaced it with the broken compare call.

Both tests are in, and I re-ran every control @hikazey asked for. I took @hikazey's two tests as posted - the ENABLE_EXPORTS symbol in the test executable, and the host dlopen(RTLD_NOW | RTLD_GLOBAL) of the fixture - plus the libkernel property change. On this head:

build guest_dynamic_loader
this head 100% tests passed out of 1
.prx check reverted to the compare form 0% tests passed, 1 tests failed
both DefaultScopeSymbol calls removed, handle 0 still accepted 0% tests passed, 1 tests failed
DynamicLoader.cpp from main, tests kept 0% tests passed, 1 tests failed
full suite on this head 100% tests passed out of 438

The second row is the one that proves the .prx fix is pinned, and the third proves the fallback is executed. So the gap I called unbuildable in my last comment is closed - @hikazey found the shape I was missing, and @Dextroid17 confirmed the bug independently from the head. Both are credited in the commit.

Windows. I could not compile for Windows here: no MinGW toolchain on this machine. The two new assertions and the fallback itself are inside #ifndef _WIN32, and the ENABLE_EXPORTS property is set under if(NOT WIN32), so nothing new reaches that path. CI will have to confirm it.

Refs #694 is now in the description, as suggested.

@blawesom blawesom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the design and the test's ordering assumptions against the source before commenting. Two things worth an answer, one small must-fix nit.

What I checked (all correct):

  • Test ordering for the final assertions: by the time the new block runs, the 0x100-global fixture is dlclosed and the pinned module was opened without 0x100 (so RTLD_LOCAL, invisible to RTLD_DEFAULT); the host RTLD_GLOBAL copy is the only one the default scope can find, so dlsym(nullptr, "GuestModuleAdd") == hostAdd is sound.
  • ENABLE_EXPORTS ON on the test target is required for GuestDefaultScopeOnly to be visible to RTLD_DEFAULT.
  • MainProgramBase() via dl_iterate_phdr (empty dlpi_name = main program) is the right convention; static-local init is thread-safe.
  • Host symbols from libc.so (malloc, dlopen, pthread_create) are correctly rejected by FromGuestModule.

1. (question, design) FromGuestModule accepts any default-visibility symbol in the main program. The relinked executable contains the whole AnyPS5 runtime, so a guest dlsym(NULL, ...) can now resolve a same-named runtime symbol that lives in the exe — the test only proves symbols from libc.so are filtered. Real hardware dlsym(NULL) is also main-scope so this is probably fine, but it's the kind of resolution change that can silently differ from the console, so it would be good to state that this is intended.

2. (parity gap) DefaultScopeSymbol tries the plain name and the NID, but FindSymbol also tries name#guest and nid#guest. If a DT_NEEDED-loaded prx exports only #guest-suffixed symbols, a handle lookup finds them but the default scope cannot. Should the fallback mirror all four forms?

3. (nit, must-fix) Formatting glitch in GuestDynamicLoader.cpp — two Require statements on one line:

Require(dlsym_nid_postfix(nullptr, "pthread_create") == nullptr);    Require(dlsym_nid_postfix(nullptr, "GuestDefaultScopeOnly") == reinterpret_cast<void*>(&GuestDefaultScopeOnly));

4. (nit) The hardcoded NID "3qcG+yBRS+s" could be Nid::ComputeNid("GuestModuleAdd", "") so it can't drift from the fixture.

5. (forward-compat note) link.h / dl_iterate_phdr doesn't exist on macOS; the #ifndef _WIN32 guard will need narrowing when the macOS port lands (that port ships a dl_iterate_phdr emulation). Fine for now.

6. (scope note for #694) The issue's local fix also added the scriptingGetMem/scriptingFreeMem exports, and the "intentionally omitted?" question is still open — the issue probably shouldn't be considered fully closed by this PR alone.

Overall the approach (filter the real RTLD_DEFAULT instead of registering every libs/*.prx in the modules map) looks right and less invasive than the alternative discussed in #694.

jeffrey12cali and others added 2 commits October 8, 2026 14:30
Titles pass handle 0 (RTLD_DEFAULT) for a process-wide lookup, and the loader rejected it: the handle is
not in the module map, so every such lookup failed even when a loaded module exports the symbol. Handle 0
and -2 now take the default scope, searching the global modules by plain name, NID and `#guest` form, and
then the process-wide scope.

The process-wide scope of the real loader also contains glibc and libstdc++, so an unfiltered fallback
would answer `malloc`, `dlopen` and `pthread_create` where PS5 reports the symbol as missing: handle 0 is
the main program, whose lookup sees the program's exports, not the host C++ runtime. A result from the
process scope is therefore accepted only when `dladdr` attributes it to a converted or replacement module
(a `.prx` library or the executable), which keeps the fallback for guest symbols and removes the host
leak. `guest_dynamic_loader` asserts the three host names are not found; disabling the filter makes that
test fail.

The scripting heap that an earlier revision of this branch added has been dropped: the title reaches the
same boot milestone with the lookup failing honestly, so the export was an inference rather than a
requirement, and prosper's bring-up notes report `ESRCH` for that name on hardware.
@adriadam10
adriadam10 force-pushed the fix/dlsym-default-scope branch from 5aa7a5b to 0ecd75f Compare October 8, 2026 12:34

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked on Linux merged onto current main: build + ctest 450/450 green; with DynamicLoader.cpp reverted, guest_dynamic_loader fails, so the handle-0 / default-scope assertions pin the change. The .prx fix (ends_with) and the two tests from @hikazey (executable export via ENABLE_EXPORTS, host dlopen with RTLD_GLOBAL) cover the fallback; thanks to @hikazey and @Dextroid17 for finding the dead branch.

Added on top (rebased on main):

  • Split two statements that shared one line in GuestDynamicLoader.cpp (pthread_create assertion / GuestDefaultScopeOnly).

Not verified here: Windows runtime (everything new is under #ifndef _WIN32; covered only by the Windows CI job). Merging once CI is green on the pushed head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants