Skip to content

fix(libkernel): fail an unknown setsockopt or getsockopt level with EINVAL like FreeBSD - #8448

Open
xXPBSXx wants to merge 3 commits into
boykopovar:mainfrom
xXPBSXx:fix/sockopt-unknown-level-einval
Open

xXPBSXx wants to merge 3 commits into
boykopovar:mainfrom
xXPBSXx:fix/sockopt-unknown-level-einval

Conversation

@xXPBSXx

@xXPBSXx xXPBSXx commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

What

setsockopt_nid_postfix and getsockopt_nid_postfix (Udp.cpp) answered ENOPROTOOPT for every level except SOL_SOCKET, IPPROTO_TCP and IPPROTO_IPV6. FreeBSD 11.4 hands such a level to the protocol, and ip_ctloutput (ip_output.c#L943-L944), ip6_ctloutput (ip6_output.c#L1459) and uipc_ctloutput (uipc_usrreq.c#L1228-L1229) fail every level they do not own with EINVAL. Now an unknown level fails with EINVAL. The levels a protocol owns keep ENOPROTOOPT for the options nothing here implements: IPPROTO_IP on an AF_INET socket and IPPROTO_UDP on a datagram socket (udp_ctloutput); level 0 on an AF_UNIX socket fails with EOPNOTSUPP for the options uipc_ctloutput does not handle; LOCAL_PEERCRED (get), LOCAL_CREDS and LOCAL_CONNWAIT, which it handles, throw as unimplemented. The Technical debt entry is not touched: main has since split the getsockopt, setsockopt entry into measured per-option entries, and its unmeasured sentence on the IPPROTO_IP level is now only exact for AF_INET sockets (this change answers EINVAL on AF_INET6 and EOPNOTSUPP/EINVAL on AF_UNIX); that is left for a separate note.
Fixes #8447
Overlap: git merge-tree of this head with each of #7490, #7475, #7485, #7090, #9515 and #9578 (every one on the merge base git finds for the pair) is clean, and so is this head against main 160f6592b5. #7511, which changed the same function bodies, was closed on 25 October; #8392, #7850 and #8312 are merged. pr_overlap was not run. #10320 (merged) changed the same two lines and routes level 0 to SetIpLevelOption/GetIpLevelOption; here level 0 reaches UnsupportedLevelError only on an AF_UNIX socket, so the PS5-measured AF_INET and AF_INET6 answers of #10320 are unchanged.

Tested

First measurement, Windows 10 (19045), MinGW-w64 GCC 15.2.0, Release, on upstream/main 02b5a8e + the first commit:

  • guest_socket_options (the only test registered under that name): fails without the change to Udp.cpp (an unknown protocol level on a stream socket must fail with EINVAL (22), ip_ctloutput (errno 42), exit 0xc0000409) and passes with it.

  • 40 socket and network tests (guest_socket_*, guest_sce_net, guest_udp, guest_tcp, guest_poll, guest_select*, guest_datagram_*, guest_recvmsg_memory) pass.

  • python tools/check_conventions.py --base upstream/main: no finding.

  • Rebased onto main b607bfdc8b (head 425ada08af; conflict only in Udp.cpp, one line in each of setsockopt_nid_postfix and getsockopt_nid_postfix, from fix(libkernel): keep IP_TTL, IP_TOS and the multicast options as measured on a PS5 #10320 routing level 0): Windows 10 19045, MinGW-w64 GCC 15.2.0, Release, both arms on the same base and rebuilt: guest_socket_options and guest_socket_stream_options pass 4 of 4 runs; with Udp.cpp restored to main, guest_socket_options fails 4 of 4 (exit code 0xc0000409) and guest_socket_stream_options passes 4 of 4. Linux (WSL2 Ubuntu 24.04, virtualised kernel, not bare-metal), GCC 13.3.0, Release, one run per arm: both pass; with Udp.cpp restored to main, guest_socket_options aborts and guest_socket_stream_options passes.

  • Not tested: bare-metal Linux, macOS, a console, a title run. Not covered by the new test: level 17 on an AF_INET6 datagram socket and any console behaviour of the throwing options.

Checklist

AI-assisted (Claude Code).

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 7088 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 7201 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 7475 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 8605 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 9509 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 9713 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 9948 core/libs/prx/libkernel/Socket/src/Udp.cpp
PR 10621 core/libs/prx/libkernel/Socket/src/Udp.cpp

@xXPBSXx

xXPBSXx commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 1896871ba7 (head 01b3b7e9a9, same commit). The conflict was only in Udp.cpp: main added ReceiveAnsweredHere next to where UnsupportedLevelError goes; both functions are kept. Windows 10 19045, WinLibs GCC 15.2.0 r7, Release: ctest -R '^guest_socket_options$' passes 3 of 3; the socket options, udp, tcp, socket_pair, poll and socket_address_length tests pass (6 of 6). check_conventions passes (one existing length warning on the long getsockopt debt entry); git merge-tree against main is clean. I did not re-run the test without the change on this head.

AI-assisted (Claude Code).

@xXPBSXx
xXPBSXx force-pushed the fix/sockopt-unknown-level-einval branch from a91225e to e33b6a9 Compare October 24, 2026 03:00
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 24, 2026

@harisibrar1 harisibrar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked against main 522e32e and FreeBSD 11.4 sys/kern/uipc_usrreq.c, sys/netinet/udp_usrreq.c and sys/netinet/ip_output.c:

  • Main still answers every level other than 0xffff/6/41 with ENOPROTOOPT (Udp.cpp:1148, 1160), and git merge-tree against main is clean. orezraey's #10320 changes the same two lines with PS5 13.60 measurements (gist d9efe0da…, pinet4): level 99 gives EINVAL, IPPROTO_IP on AF_INET6 gives EINVAL, and an unknown IP option gives ENOPROTOOPT. Those agree with this PR, so they may be worth citing. One difference: #10320's blanket EINVAL would also cover level 17 on a datagram socket, where udp_ctloutput (udp_usrreq.c:982) keeps ENOPROTOOPT as this PR does, so whichever lands second should keep that.
  • AF_UNIX, level 0: uipc_ctloutput (uipc_usrreq.c:1237-1309) supports LOCAL_PEERCRED (1, get), LOCAL_CREDS (2) and LOCAL_CONNWAIT (4); only other options get EOPNOTSUPP. With this change (UnsupportedLevelError, Udp.cpp:729), setsockopt(unixfd, 0, 2, &one, 4) fails with EOPNOTSUPP where 11.4 returns 0. Could those three options throw (NotImplemented_nid_no_patch) and the rest return EOPNOTSUPP?
  • Small test note: option 1 at IPPROTO_UDP is UDP_ENCAP (netinet/udp.h), which 11.4 handles on an IPSEC kernel (udp_usrreq.c:1004-1012). An option number with no case would test ENOPROTOOPT without depending on the kernel config.

Reviewed with AI assistance (Claude Code); the checks above were run locally on my machine.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Progress report (acb900c → 77c7232)

📉 System libraries: 50.17% (-0.02%, -2 reverted functions)

⚠️ 2 went back to stubs
Library Name
libkernel getsockopt_nid_postfix
libkernel setsockopt_nid_postfix

Changed libraries (implemented exports)

Library Before After Change
libkernel 683/721 (94.73%) 681/721 (94.45%) -0.28 pp

@xXPBSXx

xXPBSXx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

@harisibrar1 thanks for the review. Pushed b6ca1f3660 on top (no force):

  • AF_UNIX level 0: getsockopt of LOCAL_PEERCRED, LOCAL_CREDS and LOCAL_CONNWAIT and setsockopt of LOCAL_CREDS and LOCAL_CONNWAIT now throw as unimplemented, as uipc_ctloutput answers them; every other option at level 0 fails with EOPNOTSUPP (45), setsockopt of LOCAL_PEERCRED included (11.4 has no set case for it). Any other level on an AF_UNIX socket fails with EINVAL.
  • The IPPROTO_UDP test uses option 77, which has no case in udp_ctloutput, instead of UDP_ENCAP. New AF_UNIX cases cover the level, the unknown option and the throwing options.
  • Level 17 on a datagram socket stays ENOPROTOOPT, as udp_ctloutput does; whichever of this PR and fix(libkernel): keep IP_TTL, IP_TOS and the multicast options as measured on a PS5 #10320 lands second should keep that. Thanks for pointing at fix(libkernel): keep IP_TTL, IP_TOS and the multicast options as measured on a PS5 #10320: its PS5 13.60 measurements (level 99 and IPPROTO_IP on AF_INET6 give EINVAL, an unknown IP option gives ENOPROTOOPT) agree with this PR; I have not measured on a console myself.
  • Both runs on the same base main 379b1dca32, rebuilt each time. Windows 10 19045, WinLibs GCC 15.2.0: guest_socket_options and guest_socket_stream_options pass 3 of 3; with Udp.cpp as before this commit and the new tests, guest_socket_options aborts 3 of 3. Linux (WSL2 Ubuntu 24.04, virtualised kernel, not bare-metal): both pass 2 of 2; with the old Udp.cpp, guest_socket_options aborts.

AI-assisted (Claude Code).

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 24, 2026

@harisibrar1 harisibrar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked head b6ca1f3 against main b607bfd:

  • The AF_UNIX change matches uipc_ctloutput: on get, LOCAL_PEERCRED/LOCAL_CREDS/LOCAL_CONNWAIT now throw; on set, only LOCAL_CREDS/LOCAL_CONNWAIT throw (11.4's SOPT_SET switch has no LOCAL_PEERCRED case, so EOPNOTSUPP there is right), and every other level-0 option gets EOPNOTSUPP. Option 77 at IPPROTO_UDP avoids the IPSEC dependency. Thanks.
  • #10320 has merged since, and it changed the same two lines: main now has if (level != 0xffff && level != 0 && level != 6 && level != 41) return Fail(22); in both setsockopt and getsockopt (3-way git merge-tree shows two conflicts in Udp.cpp, plus GuestSocketOptions.cpp changed on both sides). Two things to keep in mind when resolving:
    • Level 0 is now routed to SetIpLevelOption/GetIpLevelOption, which return EINVAL for anything that isn't IPv4 (SocketOptions.cpp:283, :320). So for the AF_UNIX handling to take effect, the socket.family == GuestUnix check has to run before that dispatch; otherwise every AF_UNIX level-0 call becomes EINVAL.
    • Main's blanket Fail(22) also covers IPPROTO_UDP (17) on a datagram socket, where your helper keeps udp_ctloutput's ENOPROTOOPT, so keeping UnsupportedLevelError for the non-zero levels preserves that.
  • With #10320's PS5 13.60 measurements now on main (level 99 and IPPROTO_IP on AF_INET6 → EINVAL), the remaining parts of this PR are the AF_UNIX and IPPROTO_UDP cases.

Reviewed with AI assistance (Claude Code); the checks above were run locally on my machine.

xXPBSXx and others added 2 commits October 24, 2026 17:26
…INVAL like FreeBSD

sosetopt and sogetopt hand a level other than SOL_SOCKET to the protocol.
ip_ctloutput, ip6_ctloutput and tcp_ctloutput fail every level they do not
own with EINVAL, and uipc_ctloutput does so for any level but 0. The call
failed with ENOPROTOOPT for them; it still does for the IP level on an
AF_INET socket and the UDP level on a datagram socket, which are handled
levels with no option implemented.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d with an error

uipc_ctloutput handles LOCAL_PEERCRED (get), LOCAL_CREDS and LOCAL_CONNWAIT, so answering them with EOPNOTSUPP was wrong; they throw as unimplemented, every other option at level 0 still fails with EOPNOTSUPP. The UDP-level test uses an option number with no case in udp_ctloutput instead of UDP_ENCAP, which an IPSEC kernel handles.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xXPBSXx
xXPBSXx force-pushed the fix/sockopt-unknown-level-einval branch from b6ca1f3 to 425ada0 Compare October 24, 2026 15:34
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 24, 2026
@xXPBSXx

xXPBSXx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main b607bfdc8b (head 425ada08af). The conflict was the level check in setsockopt_nid_postfix and getsockopt_nid_postfix from #10320, as you described. I kept level 0 on AF_INET and AF_INET6 going to SetIpLevelOption/GetIpLevelOption and let only AF_UNIX reach UnsupportedLevelError for level 0, so UnsupportedLevelError and the tests are unchanged. guest_socket_options and guest_socket_stream_options pass 4 of 4 on Windows and once on Linux (WSL2); with Udp.cpp restored to main on the same base guest_socket_options fails 4 of 4 on Windows and aborts on Linux, guest_socket_stream_options passes. The ownFamily term in UnsupportedLevelError (AF_INET, level 0) can no longer be reached, because #10320 answers that case; I left it so the patch stays as reviewed, and can remove it in a follow-up commit if you prefer.

AI-assisted (Claude Code).

@wachi35676

Copy link
Copy Markdown
Contributor

Tested head 425ada08af merged onto main 49d1f9b07e, Windows 11 (WinLibs GCC 15.2.0, Release) and WSL2 Ubuntu 24.04 (GCC 13.3), BUILD_TESTING=ON.

  • guest_socket_options, guest_socket_stream_options, guest_socket_error, guest_tcp, guest_udp, guest_socket_pair: 6/6 pass, 2 runs, on both hosts. With Udp.cpp from main, guest_socket_options fails on both and the other five pass.
  • A probe of the cross-protocol levels gives EINVAL on both hosts, as FreeBSD 11.4's ip_ctloutput/ip6_ctloutput do: level 6 on a UDP/IPv4 socket, level 41 on TCP and UDP IPv4 sockets, level 17 on a TCP/IPv6 socket, and level 0 on a TCP/IPv6 socket.

One test note. SetFails(stream, 0, 1, &one, sizeof(one), ErrorNoProtocolOption) uses option 1, which is IP_OPTIONS. FreeBSD 11.4 handles it: ip_ctloutput passes the 4 bytes 01 00 00 00 to ip_pcbopts (ip_options.c:600), which reads a NOP, then an EOL, and returns 0. So the assertion pins an answer FreeBSD does not give, as harisibrar1 pointed out for UDP_ENCAP. The unknown IP option measured on the PS5 in #10320's pinet4.log is 0x7777 (ip_unknown ret=-1 errno=42). Using that number would test the same path and match the console.

AI-assisted: yes (Claude Code); results above were run on this machine.

Option 1 at level 0 is IP_OPTIONS, which FreeBSD 11.4 accepts: ip_ctloutput passes the four bytes to ip_pcbopts (sys/netinet/ip_options.c:600), which reads a NOP and then an EOL and returns 0. The check asserted an answer FreeBSD does not give. Use 0x7777, the number the IPPROTO_IP getsockopt check in the same file already uses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@xXPBSXx

xXPBSXx commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the runs and for the note on the test. You are right about option 1: at level 0 it is IP_OPTIONS, and in FreeBSD 11.4 ip_ctloutput hands the four bytes to ip_pcbopts, which reads a NOP, then an EOL, and returns 0 (sys/netinet/ip_output.c, case IP_OPTIONS; sys/netinet/ip_options.c:600). The check therefore asserted an answer FreeBSD does not give. The new commit 77c7232681 changes that line to option 0x7777, the number the IPPROTO_IP getsockopt check in the same file already uses, so it still tests the unhandled-option path. On Windows 10 (MinGW-w64 GCC 15.2, Release), the six socket tests guest_socket_error, guest_socket_options, guest_socket_stream_options, guest_udp, guest_tcp and guest_socket_pair pass in 3 of 3 runs with the new commit (ninja exit 0); I did not run Linux, and the change has no failing arm because it only moves the check to a number that returns ENOPROTOOPT either way.

AI-assisted (Claude Code).

@wachi35676

Copy link
Copy Markdown
Contributor

Re-checked head 77c7232681 merged onto main 0f34ebb85 (clean). Windows 11 with WinLibs GCC 15.2.0, and WSL Ubuntu 24.04 with GCC 13.3.

  • guest_socket_options, guest_socket_stream_options, guest_socket_error, guest_tcp, guest_udp and guest_socket_pair pass 6/6 on both hosts, 3 runs. This covers the Linux run you didn't do.
  • With Udp.cpp from main, guest_socket_options still fails on Windows. Option 0x7777 at level 0 reaches ip_ctloutput's default ENOPROTOOPT in FreeBSD 11.4, so the check now matches both the source and the console log.

AI-assisted: yes (Claude Code); results above were run on this machine.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs fix error Closes an issue (Fixes #N) libkernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libkernel: setsockopt/getsockopt with an unknown level fails with ENOPROTOOPT where FreeBSD 11.4 fails with EINVAL

4 participants