Skip to content

fix(http): preserve context and epoll state on allocation failure - #9630

Open
Ondra97 wants to merge 18 commits into
boykopovar:mainfrom
Ondra97:codex/fix-http-state-allocation
Open

Ondra97 wants to merge 18 commits into
boykopovar:mainfrom
Ondra97:codex/fix-http-state-allocation

Conversation

@Ondra97

@Ondra97 Ondra97 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

What

Allocation failure can leave HTTP context and epoll ownership partly committed. sceHttpInit inserts a context before initializing its option registry; a failed second insertion leaves a context that was never returned. sceHttpCreateEpoll publishes its output before inserting the epoll into its registry, so an insertion failure leaves a handle to destroyed state. sceHttpTerm removes the last context before allocating its temporary epoll list, and can then fail without finishing cleanup.

I roll back the context insertion if option initialization fails, publish an epoll only after its registry insertion succeeds, and detach the existing epoll map with swap before waking and destroying its entries. This avoids allocating a teardown list after the last context has been removed. The current shared-epoll lifetime across multiple contexts is retained.

These changes make host bookkeeping consistent under C++ allocation failure. PS5 out-of-memory error codes and failed-call output values remain unmeasured and are recorded in debt; creation can still throw std::bad_alloc. The test's replacement allocator now disarms the injected budget before throwing, so exception unwinding cannot recursively trigger the same synthetic fault.

The updated overlap scan finds no comparable line/export collision but cannot compare conflicting #6044 and #8620. Their exact shared-file patches change test paths/registration and cookie setters, respectively. I also reviewed #9417's request-header removal, which changes a distinct export; #9229's proxy change has merged into this base. None implements this allocation fix. #8357 remains open and adds a pool-size registry at sceHttpInit/sceHttpTerm; the allocation fix should land first, then #8357 should rebase and retain rollback across every registry. No joint composition with #8357 was built.

The planned PPSA90210 request-transport work also changes initialization and termination. The coordination plan is this fix first, then the transport PR declares #9630 and integrates its net/SSL registries into initialization rollback and per-context teardown, preserving the shared epoll lifetime. #8357 is another downstream registry change; its order relative to the transport work needs the actual transport patch before comparison. No joint transport build or runtime validation has been performed.

Tested

Windows 11 Pro 10.0.26300; AMD Ryzen 9 7900X, 31.66 GiB RAM, NVIDIA GeForce RTX 4070 Ti SUPER. WinLibs GCC 15.2.0, Ninja Release, BUILD_TESTING=ON, ANYPS5_RELINKER_ONLY=OFF, bundled dependencies.

Current main base: b7d1d47a5f9f32c9c388ad8d47a0055a90ae6e33. Tested head: f66f955da716869c32b74a0d822e4ddea71ca43e. The regression executable compiles production Export.cpp and Options.cpp directly so its replacement C++ allocator reaches the real bookkeeping code. The red control and the Before values below were measured on the earlier base d6eafa54d0464258574fd2f71889754d05a5bc93; I did not rerun that control on the new base. The HTTP implementation and regression source were unchanged by the main merges. The original HTTP tests separately exercise the shared library.

Metric Before (base) After (fix)
New lifecycle regressions passed 1/7 7/7
Epoll creation budgets changing output after failure 2/8 0/8

With the one-shot injector, the unchanged base failed 6/7 new lifecycle tests in 0.25 s: initialization left partial state at budget 2 without an existing context and at budget 1 with one; epoll insertion changed the output in two failed calls; termination lost the context at budgets 0-2. The fix passed all 7 new tests and 28 budget scenarios: 8 for each initialization mode, 8 for epoll creation and 4 for termination. The base initialization probes stop at their first invalid state, so they do not complete all 28 scenarios. Successful epolls support polling/deletion, existing contexts survive failed creation, failed outputs remain untouched, and a fresh context cannot use epolls from a completed termination. No title or console was run.

The full all-target Windows build passed on the final head with BUILD_TESTING=ON, bundled dependencies and the complete default AGC execution-test set. #8310 is merged in the base; no source overlay was used. An initial sandboxed build stopped in the libusb version generator because Git Bash/MSYS could not create a named object (0xC0000022); the same incremental build completed outside that sandbox.

Final selected runtime tests: 12/12 passed, 0 skipped; 367 patched libraries loaded, 0 failed. Both changed translation units compile; conventions, registration (1,176 test executables, none built and unregistered), main-sync and whitespace checks pass. Registration is an executable count, not an executed-test count.

Commands, with the supported WinLibs toolchain on PATH and the full Release build configured in a sibling ../build directory:

cmake --build ../build --target guest_http_state_allocation_tests guest_http_tests guest_http_response_header_tests guest_http_options_tests --parallel 10
cmake --build ../build --parallel 10
ctest --test-dir ../build -R "^(guest_http2$|guest_http$|guest_http_(response_header|options|state_.*)$|patched_libraries$)" --parallel 4 -V
python tools/check_conventions.py --base origin/main
python tools/check_test_registration.py ../build
python tools/check_main_sync.py --base origin/main --strict
python tools/pr_overlap.py --branch codex/pr9630-final-sync-20261022 --repo boykopovar/AnyPS5 --jobs 8
git diff origin/main...HEAD --check

All commands above used the final submitted sources. The earlier red control temporarily compiled the regression target with unchanged earlier-base Export.cpp, then restored the source before the final build.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 10535 core/libs/prx/libSceHttp/CMakeLists.txt L25

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6044 core/libs/prx/libSceHttp/CMakeLists.txt
PR 9822 core/libs/prx/libSceHttp/Export.cpp
PR 9826 core/libs/prx/libSceHttp/Export.cpp

@fogydes

fogydes commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

The two failure paths are fixed the right way, verified against the diff: sceHttpCreateEpoll only writes *eh after the map insert succeeds (so nothing is published and the caller's value survives a throw), sceHttpTerm_nid_postfix swaps the whole map out under the lock instead of a move-out loop (no half-moved epolls left behind by a throw), and a failed AddHttpContext in sceHttpInit rolls back the context id it just registered. The injected bad_alloc tests cover both directions (rejected and shrinking-budget success), including the output invariants.

Two nits, neither blocking:

  1. CMakeLists.txt now carries two if(BUILD_TESTING) blocks whose tail duplicates the Windows PATH fixup for the same test directory; one block would be simpler.

  2. The term test walks budgets 0..3 while the init/create tests walk 0..7, so the term path sees fewer allocation-fault positions than its siblings do.

No conventions problems: no comments added, the debt entry records exactly what remains unverified (console OOM error values and failed-call outputs), and the inline // No network is emulated... comment predates the PR.

@MarcZX10 MarcZX10 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocation-failure fixes match the ownership invariants: init rolls back the inserted context, epoll output is published only after registry insertion, and termination swaps the existing map instead of allocating a teardown vector. The one-shot allocator avoids recursive fault injection during unwinding. The author reports 7/7 regressions on the fix versus 1/7 on base, plus 0/8 output changes after failed epoll creation, 11/11 selected runtime tests and a full Windows build. An independent review agrees with the key failure paths. I ran exact-head git diff --check and current-main conventions; both pass. Full hosted product jobs are skipped on this head, so the Windows results are author-reported; I did not rerun them locally. Minor test coverage note: termination explores budgets 0–3 while init/create explore 0–7, and CMake has duplicated Windows PATH setup; neither changes the core disposition. The change is generic with Unblocks: none. Coordinate merge order with #8357 as stated in the description.

I’ve noticed that holding this PR because of #5239 isn’t necessary. It’s fine for PRs to keep coming in; they’ll be handled in due course. Please disregard my earlier request to hold or defer it.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The allocation-failure fixes and regression results remain convincing. I independently ran exact-head whitespace/conventions checks, test registration (171 scripts), and Linux syntax checks for the changed HTTP implementation, options unit, and allocation test; all passed. The author reports 7/7 allocation regressions, 0/8 failed-create output mutations, 11/11 selected runtime tests, and a full Windows build. Product jobs are path-skipped in hosted CI, so those runtime results remain author-reported here.

I am withdrawing my earlier #5239 intake hold as a reason to defer this PR. The remaining repository requirement is a current-main base: this targets d6eafa5, while main is 15 commits ahead at 8b3a6ba. The intervening commits do not touch the HTTP files changed here, so the update should be low risk; please rebase and refresh conventions/overlap checks before merge. The ordering with #8357 is already agreed by both authors: this fix first, then #8357 rebases and carries rollback through its pool registry.

Amoenus and others added 4 commits October 22, 2026 16:03
…e-clear-handoff

fix(agc): settle queued HTILE clears before native depth use
…ic-tessellation-strips

feat(agc): draw triangle strips through the barycentric tessellation stages
@Ondra97
Ondra97 requested a review from MarcZX10 October 22, 2026 14:34
@DoubleAAGuy

Copy link
Copy Markdown

I am working on a separate libSceHttp request-transport PR for PPSA90210. It needs to register per-context net/SSL ids in sceHttpInit and remove them in sceHttpTerm, so those two exports overlap #9630. I can sequence my PR after yours and rebase/integrate the allocation rollback, or we can coordinate the combined hunk if you prefer. Do you expect #9630 to merge soon?

@Ondra97

Ondra97 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@DoubleAAGuy I prefer #9630 first, then your transport PR with Depends on: #9630. I do not have a reliable merge ETA; that decision belongs to the maintainers. You can prepare and test the transport work against the current #9630 head f66f955da716869c32b74a0d822e4ddea71ca43e, then rebase onto main after it lands.

For the combined initialization/termination hunk:

  • Treat all context registrations as one transaction. If any net/SSL registry insertion fails, roll back every entry already created, including the context/options state, while leaving existing contexts usable.
  • Keep transport teardown per-context, retain shared epolls until the last context terminates, and avoid allocating a temporary cleanup list after removing the context.
  • Extend the existing guest_http_state_* fault-injection coverage to the added registries so partial initialization and failed-output publication cannot return.

Please also account for #8357: its pool-size registry changes the same exports and already declares #9630 as its prerequisite. I would compare the transport and pool-stat hunks before choosing their relative order, preserving both registries.

I have recorded the proposed transport sequencing in this PR's description. Please link your branch or PR when available so I can compare the actual changes and help settle the remaining order with #8357.

AI-assisted: yes.

@DoubleAAGuy

Copy link
Copy Markdown

Agreed. I have the transport context registration inside the sceHttpInit rollback and remove it per context in sceHttpTerm; I will extend the allocation-failure tests to include Request.cpp registry insertions and verify existing contexts survive. I will declare this PR as depending on #9630 and #8357, then rebase onto both after they merge. I will share the branch/PR link once the integration test and overlap review are ready.

@DoubleAAGuy

Copy link
Copy Markdown

The transport work is now in draft #9834. It declares Depends on: #9630, #8357, #8620; I will integrate your rollback and the pool-size registry in one init/term transaction after both land. I have not run the combined allocation-fault test yet; I plan to extend the guest_http_state_* coverage before marking #9834 ready.

@MarcZX10

Copy link
Copy Markdown
Contributor

I’ve noticed that my earlier request to hold this PR based on #5239 was unnecessary. It’s fine to keep submitting PRs; maintainers will handle them in due course. Please disregard that request.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants