Skip to content

feat(libSceLibreSsl): implement EC points, key generation, ECDSA and ECDH - #10407

Closed
MatteoSid wants to merge 2 commits into
boykopovar:mainfrom
MatteoSid:feat/libressl-ec-points-ecdsa-ecdh
Closed

MatteoSid wants to merge 2 commits into
boykopovar:mainfrom
MatteoSid:feat/libressl-ec-points-ecdsa-ecdh

Conversation

@MatteoSid

Copy link
Copy Markdown

What

Implements eleven libSceLibreSsl exports that were throwing NotImplemented: EC_POINT_new, EC_POINT_free, EC_POINT_point2oct, EC_POINT_oct2point, EC_KEY_generate_key, EC_KEY_check_key, EC_KEY_get0_public_key, EC_KEY_set_public_key, ECDSA_sign, ECDSA_verify and ECDH_compute_key. Builds on #10395, whose commit is the first one here; this diff is the second commit.

  • Signatures and results follow LibreSSL 2.5.4 at 2e2c612: ec/ec_lib.c, ec/ec_key.c, ec/ec_oct.c, ec/ecp_oct.c, ecdsa/ecs_sign.c, ecdsa/ecs_vrf.c, ecdsa/ecs_ossl.c and ecdh/ech_key.c.
  • EC_POINT is a host object holding an mbedtls_ecp_point and its curve; an EC_KEY now also holds its public point and private scalar. The three exports that take or return a BIGNUM (EC_POINT_mul, EC_KEY_get0_private_key, EC_KEY_set_private_key) still throw: BIGNUM is feat(libSceLibreSsl): implement the BN_ BIGNUM, BN_CTX, conversion, word, BN_mul and BN_mod_inverse functions #9446.
  • EC_POINT_point2oct and EC_POINT_oct2point handle the compressed, uncompressed and hybrid forms and the single zero octet for infinity, with LibreSSL's checks: form and y bit, exact length, coordinates below the prime, hybrid parity and the X9.62 on-curve test. A point that fails only the on-curve test keeps the decoded coordinates, as in LibreSSL, so EC_KEY_check_key can reject it later.
  • EC_KEY_generate_key draws the scalar in [1, n-1] from host entropy and reuses the key's existing point object, as LibreSSL does. EC_KEY_check_key checks infinity, the curve equation and, with a private key, that it gives the public key; the order test always passes because all 11 curves have cofactor 1.
  • ECDSA_sign signs through mbedtls with a random nonce and writes LibreSSL's DER (unsigned INTEGERs, 0x00 pad when the top bit is set); a null buffer only returns the length. ECDSA_verify returns -1 unless the signature is canonical DER with nothing after it (what LibreSSL's decode and re-encode comparison accepts), 0 for r or s outside [1, n-1] or a mismatch, and -1 when u1·G + u2·Q is at infinity. Digests longer than the order are truncated to its bit length, like ecdsa_do_sign and ecdsa_do_verify.
  • ECDH_compute_key returns the x coordinate padded to the field size: -1 without a private key, group or peer key, for a peer at infinity, for a length above INT_MAX or a buffer shorter than the secret without a KDF; a larger buffer is zero-filled. A KDF is called through the guest ABI and its output length is returned.

Gaps, in the debt file: no error is queued; a point used with a group of another curve throws, since LibreSSL's result depends on the EC methods its build picks; so does EC_KEY_set_group to another curve on a key that already has a public or private key; compressed points on secp224k1 and secp224r1 throw (mbedtls only decompresses for primes that are 3 mod 4); ECDSA_verify with a public key at infinity or off the curve and ECDH_compute_key with a peer key off the curve throw, where LibreSSL computes with them. The EC debt entry is split in two to stay under 1000 characters. The pending-exports count goes from 356 to 345.

Tested

Linux (Ubuntu 24.04 container, GCC 13.3), full configure with -DBUILD_TESTING=ON, Release: built libSceLibreSsl, libSceLibreSsl3, their patched .prx files and the eight LibreSSL tests; ctest -R 'libressl|crypto_memcmp' passes 8 of 8 (the new guest_libressl_ec_keys takes 0.6 s). It checks:

  • the P-256 generator in all three forms, its decompression with both parities (the even root is checked against the negated generator), hybrid parity, 11 malformed encodings that leave the point unchanged, a point off the curve that keeps its coordinates, infinity, other curves and the 224-bit compressed throws;
  • on all 11 curves: key generation, EC_KEY_check_key, encoded sizes, sign and verify (another key, a changed signature, another digest, an empty digest), ECDH agreement on both sides and regeneration keeping the point object;
  • the RFC 6979 A.2.5 P-256/SHA-256 "sample" signature, checked independently with OpenSSL through cryptography 46, and a secp224k1 signature from a Python reference implementation whose 225-bit order makes the truncation drop seven bits; ten non-DER signatures return -1 and five out-of-range ones return 0;
  • ECDH with the generator as peer returns the x coordinate of the own public key, plus padding, short buffers, KDF output, failing KDF, infinity, off-curve and other-curve peers;
  • EC_KEY_set_public_key copies, own key, null, infinity, off-curve and other-curve points, and EC_KEY_set_group with and without keys.

The test fails when the truncation shift, the hybrid y bit, the DER minimal-integer check or the ECDH zero fill is removed. nm -D on the patched libSceLibreSsl.prx lists the eleven NIDs. tools/check_conventions.py --base origin/main passes; tools/pr_overlap.py --branch feat/libressl-ec-points-ecdsa-ecdh finds no open PR with the same exports or new files. Its line overlaps with #9033, #9448, #9529, #10071, #10078, #10373, #10376 and #10398 are the add_library/test block in CMakeLists.txt and the pending count in the debt file; #9446, #9571, #9745, #9870, #9949, #10005 and #10248 conflict with main and could not be compared, and their diffs touch none of these exports. No title tested.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: feat(libSceLibreSsl): implement EC groups and keys for named curves #10395
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

MatteoSid and others added 2 commits October 23, 2026 23:05
EC_GROUP_new_by_curve_name, EC_GROUP_free, EC_GROUP_get_degree, EC_KEY_new,
EC_KEY_new_by_curve_name, EC_KEY_free, EC_KEY_set_group, EC_KEY_get0_group
and ECDSA_size follow LibreSSL 2.5.4 on top of mbedtls's named curves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ECDH

EC_POINT_new, EC_POINT_free, EC_POINT_point2oct, EC_POINT_oct2point,
EC_KEY_generate_key, EC_KEY_check_key, EC_KEY_get0_public_key,
EC_KEY_set_public_key, ECDSA_sign, ECDSA_verify and ECDH_compute_key
follow LibreSSL 2.5.4 on top of mbedtls's named curves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the open PR limit Closed over the open PR limit: 1 without merged PRs, 20 with at least 1, none from 20 label Oct 23, 2026
@github-actions

Copy link
Copy Markdown

Thank you for your contribution! Authors with no pull requests merged into main can have 1 open pull request at a time, authors with at least 1 merged can have 20, and from 20 merged there is no limit. This PR has been automatically closed and labeled "open PR limit" because that limit was reached. This is not a judgment of the quality of your changes: reopen it once one of your open pull requests is merged or closed.

@github-actions github-actions Bot closed this Oct 23, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head dcb4a21a2ec7b9e154760f6102e5f0c099bb3adc against refreshed upstream main c8d34aa8f4d8058d57169cc1e423ba6db00faaf0.

The two-commit stack is explicitly based on #10395, and the description gives unusually detailed coverage for point encodings, key generation, ECDSA DER/range behavior and ECDH buffer/KDF cases. Current-main conventions and diff-check pass. However, #10395 is still open, and the current-main merge-tree for this stack has content conflicts in core/libs/prx/libSceLibreSsl/CMakeLists.txt and docs/dev/debt/libSceLibreSsl.md. The exact reported Linux test suite therefore does not yet verify the final stack on current main; hosted product tests are skipped.

Please refresh the #10395 dependency and this follow-up to current main, resolve those conflicts while retaining both test registrations and debt entries, and rerun the eight focused LibreSSL tests against the composed head. I have not locally reproduced the crypto test suite or reviewed the final merged dependency composition, so this is an integration request rather than a claim that the cryptographic operations are incorrect.

@MatteoSid

Copy link
Copy Markdown
Author

Thanks, done. #10395 is refreshed to current main (98b611f6da on c8d34aa8f4). This PR was closed by the open-PR limit, and GitHub does not reopen a closed PR whose branch was force-pushed, so the composed stack is on MatteoSid:feat/libressl-ec-points-ecdsa-ecdh-main (4c0edc4543, parent 98b611f6da). CMakeLists.txt merged cleanly and keeps the AES, EC and EC keys test registrations. The debt file keeps the AES entry and both EC entries; the pending count is 337 (348 after #10395, minus the 11 exports here), matching the NotImplemented_nid_no_patch stubs left in Export.cpp.

On 4c0edc4543 (Ubuntu 24.04 container, GCC 13.3, Release, -DBUILD_TESTING=ON): built libSceLibreSsl, libSceLibreSsl3 and their patched .prx files; ctest -R 'libressl|crypto_memcmp' passes 9 of 9 (the eight reported plus guest_libressl_aes from #9033); nm -D on the patched libSceLibreSsl.prx lists the 20 NIDs of both PRs; tools/check_conventions.py --base origin/main passes. When #10395 is merged I will reopen this PR and move its branch to that stack rebased on main.

@MatteoSid

Copy link
Copy Markdown
Author

Refreshed again after #9529 and #10706 landed: #10395 is now f1f59e8408 on dc15ed6537, and the stack on MatteoSid:feat/libressl-ec-points-ecdsa-ecdh-main is 75e5fbf241. Following #10706, the 20 EC exports are declared once in include/Api.hpp with their signatures, over opaque EcGroup, EcKey and EcPoint; their layouts are in the internal include/Ec.hpp, which only Ec.cpp includes, and the tests no longer declare the functions. The debt file keeps the AES, error-queue and both EC entries; the pending count is 333, matching the stubs left in Export.cpp.

On 75e5fbf241: ctest -R 'libressl|crypto_memcmp|public_headers' passes 75 of 75 (the ten LibreSSL tests, public_headers and its 64 compile checks), the EC sources build without warnings, nm -D on the patched libSceLibreSsl.prx lists the 20 NIDs, and tools/check_conventions.py --base origin/main passes.

@MatteoSid

Copy link
Copy Markdown
Author

Applied the test rule from the #10395 review here too: GuestLibreSslEcKeys.cpp no longer requires throws from valid but unimplemented paths (another curve's group or point, 224-bit compressed points, verify and ECDH with an off-curve or infinite key, EC_KEY_set_group to another curve). Those stay as production throws with their debt entries; the remaining throw checks are null arguments. The stack on MatteoSid:feat/libressl-ec-points-ecdsa-ecdh-main is now a58f124dbd on #10395 b0ae5d98f6; ctest -R 'libressl|crypto_memcmp|public_headers' passes 75 of 75 and check_conventions passes.

ahmadmuj added a commit to ahmadmuj/AnyPS5 that referenced this pull request Oct 26, 2026
ahmadmuj added a commit to ahmadmuj/AnyPS5 that referenced this pull request Oct 26, 2026
ahmadmuj added a commit to ahmadmuj/AnyPS5 that referenced this pull request Oct 26, 2026
…var#10407) into fix/anyps5-lssl-ec-privkey

# Conflicts:
#	core/libs/prx/libSceLibreSsl/CMakeLists.txt
#	core/libs/prx/libSceLibreSsl/Export.cpp
#	core/libs/prx/libSceLibreSsl/include/Api.hpp
#	core/libs/prx/libSceLibreSsl/include/Ec.hpp
#	core/libs/prx/libSceLibreSsl/src/Ec.cpp
#	core/libs/tests/GuestLibreSslEcKeys.cpp
#	docs/dev/debt/libSceLibreSsl.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open PR limit Closed over the open PR limit: 1 without merged PRs, 20 with at least 1, none from 20

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants