Skip to content

feat(libressl3): retain X509 store references - #12580

Closed
PrivacyOnion wants to merge 2 commits into
boykopovar:mainfrom
PrivacyOnion:feat/libressl3-x509-store-up-ref
Closed

PrivacyOnion wants to merge 2 commits into
boykopovar:mainfrom
PrivacyOnion:feat/libressl3-x509-store-up-ref

Conversation

@PrivacyOnion

@PrivacyOnion PrivacyOnion commented Oct 28, 2026 •

Copy link
Copy Markdown

What

Implement X509_STORE_up_ref_nid_postfix in libSceLibreSsl3.

  • In core/libs/prx/libSceLibreSsl3/src/Key.cpp, implement X509_STORE_up_ref_nid_postfix(LibreSslX509Store* store) to atomically increment store->references via Retain_nid_no_patch, following LibreSSL 4.0.0 x509_lu.c#L240-L244.
  • Reject null store pointers by throwing std::invalid_argument("X509_STORE_up_ref requires a store").
  • Verify LibreSslX509Store size (136 bytes) and references offset (128 bytes) from libSceLibreSsl with static_assert in Key.cpp.
  • Declare X509_STORE_up_ref_nid_postfix(LibreSslX509Store* store) in core/libs/prx/libSceLibreSsl3/include/Api.hpp and remove the stub from core/libs/prx/libSceLibreSsl3/Export.cpp.
  • Document technical debt in docs/dev/debt/libSceLibreSsl3.md.
  • Expand core/libs/tests/GuestLibreSsl3KeyReference.cpp to cover LibreSslX509Store with CheckKey<LibreSslX509Store>(X509_STORE_up_ref_nid_postfix, 128).

Tested

Windows 11, verified reference count increments across initial values, sequential retain, and concurrent multithreaded retain via GuestLibreSsl3KeyReference.

Ran locally on 71e4f22169:

$ ctest --test-dir build -R '^guest_libressl3_key_reference$' --output-on-failure
    Start 2225: guest_libressl3_key_reference
1/1 Test #2225: guest_libressl3_key_reference ....   Passed    0.05 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.20 sec

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: feat(libressl3): retain SSL_CTX references #12578
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Progress report (f7f03da → 71e4f22)

📈 System libraries: 43.43% (+0.01%, +2 implemented functions)

✅ 2 implemented
Library Name
libSceLibreSsl3 SSL_CTX_up_ref_nid_postfix
libSceLibreSsl3 X509_STORE_up_ref_nid_postfix

Changed libraries (implemented exports)

Library Before After Change
libSceLibreSsl3 5/270 (1.85%) 7/270 (2.59%) +0.74%

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation and focused reference-count test look internally consistent, but the PR needs an explicit landing order with the other open libSceLibreSsl3 reference-count changes before approval. I fetched the exact current heads and checked pairwise merge previews: #12580 conflicts with #12578 and #12582 in Key.cpp, GuestLibreSsl3KeyReference.cpp, and docs/dev/debt/libSceLibreSsl3.md. All three PR descriptions currently say Depends on: none.

Please agree which PR lands first, restack the later changes as needed, and record the resulting order in each affected Depends on checklist. This is a repository coordination requirement; I am not reporting a functional defect in the accessor here. Hosted Windows is still running; Ubuntu/macOS and product test jobs are not complete or are skipped.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — changes requested

The ref-count change and its focused regression test look internally consistent. This PR still needs an explicit order with #12578 and #12582: exact-head pairwise merge previews conflict in the files both features edit, while the PR currently says Depends on: none.

What I ran

  • Compared #12580 head e5f35a2407c01477085fc0d9aa78ddbba669fda7 with #12578 head 00b24c75b4be4b3ee7ed97013e4ea4a7a21b3afe and #12582 head dd47c78d6d72d3d88b2d21ccac72ffd8adb193c3.
  • Generated pairwise merge previews against current main 319621ba2e578bd053b0baba2877fe70520e0737.
  • Refreshed hosted checks.

Results

Check Result
#12580 + #12578 conflict in shared implementation, test, and debt files
#12580 + #12582 conflict in shared implementation, test, and debt files
Current #12580 hosted policy checks pass
Current #12580 Windows workflow pass
Linux/product and specialized jobs skipped

Findings

Blocking

Please record an agreed dependency/landing order or restack this PR so its relationship to the two open sibling changes is explicit. The merge previews conflict in:

  • core/libs/prx/libSceLibreSsl3/src/Key.cpp
  • core/libs/tests/GuestLibreSsl3KeyReference.cpp
  • docs/dev/debt/libSceLibreSsl3.md

Non-blocking

I found no separate functional blocker in the implementation or focused test. This review is requesting changes for integration clarity only.

Sources

Not verified

I did not independently rerun the focused C++ tests. Hosted Linux/product checks were skipped.

To get this to a good state

Set the intended order in the PR body/checklist and declare the predecessor dependency, or restack and update the affected diffs so the pairwise merge previews are clean.

@PrivacyOnion
PrivacyOnion force-pushed the feat/libressl3-x509-store-up-ref branch from e5f35a2 to e499110 Compare October 28, 2026 21:50
@PrivacyOnion

Copy link
Copy Markdown
Author

Restacked on #12578, lands second.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 9535 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 9536 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 9571 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 9970 core/libs/prx/libSceLibreSsl3/Export.cpp
docs/dev/debt/libSceLibreSsl3.md
PR 10395 core/libs/prx/libSceLibreSsl3/include/Api.hpp
core/libs/prx/libSceLibreSsl3/include/Key.hpp
core/libs/prx/libSceLibreSsl3/src/Key.cpp
core/libs/tests/GuestLibreSsl3KeyReference.cpp
docs/dev/debt/libSceLibreSsl3.md
PR 10452 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10461 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10490 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10491 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10493 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10557 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10631 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 10960 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 11146 core/libs/prx/libSceLibreSsl3/include/Api.hpp
core/libs/prx/libSceLibreSsl3/include/Key.hpp
core/libs/prx/libSceLibreSsl3/src/Key.cpp
core/libs/tests/GuestLibreSsl3KeyReference.cpp
docs/dev/debt/libSceLibreSsl3.md
PR 12315 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 12754 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
core/libs/prx/libSceLibreSsl3/include/Key.hpp
core/libs/prx/libSceLibreSsl3/src/Key.cpp
core/libs/tests/GuestLibreSsl3KeyReference.cpp
docs/dev/debt/libSceLibreSsl3.md
PR 12928 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 12945 core/libs/prx/libSceLibreSsl3/Export.cpp
core/libs/prx/libSceLibreSsl3/include/Api.hpp
core/libs/prx/libSceLibreSsl3/include/Key.hpp
core/libs/prx/libSceLibreSsl3/src/Key.cpp
core/libs/tests/GuestLibreSsl3KeyReference.cpp
docs/dev/debt/libSceLibreSsl3.md
PR 13659 core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 13662 core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 13663 core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md
PR 13831 core/libs/prx/libSceLibreSsl3/include/Api.hpp
docs/dev/debt/libSceLibreSsl3.md

@PrivacyOnion
PrivacyOnion force-pushed the feat/libressl3-x509-store-up-ref branch from e499110 to b1a7980 Compare October 30, 2026 07:10
@PrivacyOnion

Copy link
Copy Markdown
Author

@MarcZX10 Thanks for the review.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 30, 2026
@MarcZX10

Copy link
Copy Markdown
Contributor

Re-checked — dependency stack still conflicts

I rechecked the live heads after your update; #12580 still does not descend from #12578, and combining the two heads conflicts in three files.

What I ran

The live heads are #12578 ced92ec4eda54f95ccf42ef08408bb4aaec363a6 and #12580 b1a798019b0be957a42df6f45cb62913244f2896, both targeting main at f7f03daf52f11191174b1be089a56e268390a707.

git merge-base --is-ancestor refs/review/pr-12578 refs/review/pr-12580
git merge-tree --write-tree --name-only --no-messages refs/review/pr-12578 refs/review/pr-12580

Results

Check Result
#12578 head is an ancestor of #12580 No; the ancestry check exits 1.
Combined merge preview Conflicts in Key.cpp, GuestLibreSsl3KeyReference.cpp, and docs/dev/debt/libSceLibreSsl3.md.
Unedited command output
$ git merge-base --is-ancestor refs/review/pr-12578 refs/review/pr-12580
(no output; exit status 1)

$ git merge-tree --write-tree --name-only --no-messages refs/review/pr-12578 refs/review/pr-12580
a8d8be2934b1441829bd9c6d781f48bc890881db
core/libs/prx/libSceLibreSsl3/src/Key.cpp
core/libs/tests/GuestLibreSsl3KeyReference.cpp
docs/dev/debt/libSceLibreSsl3.md
(exit status 1)

Findings

Blocking

Not verified

I did not build or run CTest locally; I have not independently verified the author-reported focused test.

To get this to a good state

Please update the branch and dependency metadata together. I will recheck the new head, the combined merge preview, and the focused test evidence.

@PrivacyOnion
PrivacyOnion force-pushed the feat/libressl3-x509-store-up-ref branch from b1a7980 to 71e4f22 Compare October 30, 2026 07:41
@PrivacyOnion

Copy link
Copy Markdown
Author

@MarcZX10 Thanks for the review.

@maskym

maskym commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

X509_STORE_up_ref increments the wrong field: it uses LibreSslX509Store from libSceLibreSsl, which is the older LibreSSL layout, not the 4.0.0 layout libSceLibreSsl3 follows.

libSceLibreSsl/include/Api.hpp has int cache plus 11 callbacks, which puts references at 128 in a 136-byte struct. LibreSSL 4.0.0 x509_local.h has no cache field and only three callbacks:

struct x509_store_st {
	STACK_OF(X509_OBJECT) *objs;
	STACK_OF(X509_LOOKUP) *get_cert_methods;
	X509_VERIFY_PARAM *param;
	int (*verify)(X509_STORE_CTX *ctx);
	int (*verify_cb)(int ok, X509_STORE_CTX *ctx);
	int (*check_issued)(X509_STORE_CTX *ctx, X509 *x, X509 *issuer);
	CRYPTO_EX_DATA ex_data;	/* { void *sk; } */
	int references;
};

On LP64 that is 6 pointers plus ex_data (8 bytes), so references is at 56 and the object is 64 bytes. On a store the guest's libSceLibreSsl3 allocates, this PR increments 4 bytes at offset 128, which is past the end of the object. That is heap corruption, and the real count never moves.

The test doesn't catch it, because it builds the buffer from the same struct and checks offset 128: CheckKey<LibreSslX509Store>(X509_STORE_up_ref_nid_postfix, 128). A dedicated libSceLibreSsl3 struct with static_assert(offsetof(..., references) == 56) and sizeof == 64 fixes the code, and the test should expect 56.

SSL_CTX at 40 matches what I get from the same source, and so do the SSL, SSL_SESSION, X509 and BIO offsets in #12754 and #12945.

AI-assisted: yes (Claude Code), the layout was read from the pinned LibreSSL 4.0.0 source above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs needs rebase Conflicts with main, rebase needed prx

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants