Repository navigation
docs(debt): say that sceNetResolverCreate fails non-zero flags and what the titles pass - #14990
Merged
oneandonlydean merged 1 commit intoNov 2, 2026
Conversation
…at the titles pass sceNetResolverCreate returns EINVAL for any non-zero flags, as the module is read in prosper. The new entry says so and that no direct call in the titles checked passes a non-zero value. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
Author
|
Per-title results and the counting script for the table in the description: one line per call site of Script (the title folders and the objdump path are local arguments): #!/usr/bin/env python3
"""Count the call sites of imported functions in title executables and read argument registers.
Input: a directory ROOT with one run folder per title, PPSA<id>_r*/, each holding
app.registry.json (import table written by the relinker; every import lists the addresses
of its call sites) and in/eboot.elf (the title's executable as ELF). Per title the registry
with the most imports that have call sites is used (ties: the newest file).
Only the title's own executable is read: libraries / extra module files, function pointers and
code that is only reached through data are not seen.
What is counted per title and function (NAME = import name, NID computed from the name):
* call sites: the registry addresses whose first bytes are FF 25 (PLT stub, jmp [rip+got]).
* direct calls: E8 rel32 instructions in the executable segments whose target is such a stub.
* tail jumps: E9 rel32 instructions whose target is such a stub. They are listed and counted
separately and are NOT direct calls.
* calls to a jump stub: E8 rel32 whose target is one of those tail-jump instructions
(a thunk that forwards to the import). Listed and counted separately, not direct calls.
* non-stub registry call sites: registry addresses that are not a stub start (FF 15 call
[rip+got], 48 8B / 4C 8B mov reg,[rip+got], anything else, or an address outside the file).
Per title and function with address and instruction kind. They are not scanned for calls.
Matches are byte patterns in the executable segments, no control-flow recovery: an E8 inside an
instruction or data can only be a false hit if its target happens to be a stub (not checked).
Argument value (optional, --arg NAME=REG, REG one of rdi rsi rdx rcx r8 r9; the 32/16/8-bit
names are accepted and mean the same register): for every direct call the 64..48 bytes before it
are decoded with objdump (Intel syntax), once for every start offset in that range.
* a decoding is used only if the call address is decoded as an instruction start; if no start
offset gives that, the site is reported as 'desync';
* going backwards from the call, the last instruction that writes REG decides: the first operand
is REG or one of its sub-registers (mov, lea, movzx, pop, adc, rol, bswap, cmov, ...),
an implicit writer (rep/movs/lods/stos/cmps/scas/ins/outs, cdq/cqo/cwd, mul/div/idiv and the
one-operand imul for rdx, cpuid, rdtsc(p), rdmsr, rdpmc, xgetbv, syscall, cmpxchg8b/16b),
xchg/xadd/cmpxchg that name REG, or a 'call' (clobbers all argument registers);
cmp, test, push, bt and nop only read;
* 'xor R,R' (R the 32- or 64-bit register) = 0; 'mov R,imm' (32- or 64-bit register) = imm;
every other writer, and any write to a 16/8-bit part, = 'computed' (not constant);
* a branch whose target lies between the last writer and the call, or an unconditional
jmp / ret between them, = 'computed' (the writer may not be on the path to the call);
* no writer in the window = 'unreadable' (not constant: set before the window or by the caller);
* if the start offsets give different answers the site is 'ambiguous' (the answers are listed in
the site line). With --reach-neutral an 'unreadable' answer of a shorter window is ignored when
a longer window reads a value (it only means that the writer lies beyond that window's reach);
all other disagreements stay 'ambiguous'.
Both 'computed' and 'unreadable' and 'ambiguous' mean "not a constant" in the sums; they are kept
apart so that a reader can see why. Window limits: a writer more than ~48 bytes back is not seen,
values that flow through memory or a callee-saved register are not followed, conditional moves
and branches taking different values on different paths are reported as 'computed'.
Nothing is executed; only counts, addresses and classifications are printed.
usage: anhang_zaehlen.py ROOT OBJDUMP NAME[,NAME...] OUT [--arg NAME=REG ...] [--titles FILE]
OBJDUMP is the objdump command (GNU binutils, x86-64); 'wsl objdump' is accepted, paths are then
converted to /mnt/<drive>/... . --titles FILE restricts the titles (PPSA<digits> found in FILE).
Output (OUT and stdout summary): one line per site "title | function | address | kind | value",
the non-stub list, and one summary line per function.
"""
import argparse
import bisect
import collections
import glob
import hashlib
import json
import os
import re
import struct
import subprocess
import sys
import tempfile
import numpy as np
SUFFIX = bytes([0x51, 0x8D, 0x64, 0xA6, 0x35, 0xDE, 0xD8, 0xC1, 0xE6, 0xB0, 0x39, 0xB1, 0xC3, 0xE5, 0x52, 0x30])
B64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-"
def nid(name):
d = hashlib.sha1(name.encode() + SUFFIX).digest()
r = bytes(d[7 - i] for i in range(8))
out = []
for i in range(0, 6, 3):
t = (r[i] << 16) | (r[i + 1] << 8) | r[i + 2]
out += [B64[(t >> 18) & 63], B64[(t >> 12) & 63], B64[(t >> 6) & 63], B64[t & 63]]
t = (r[6] << 16) | (r[7] << 8)
return ''.join(out + [B64[(t >> 18) & 63], B64[(t >> 12) & 63], B64[(t >> 6) & 63]])
# argument registers: all names of the register, and the two names a constant may be written to
PARTS = {
'rdi': ('rdi', 'edi', 'di', 'dil'),
'rsi': ('rsi', 'esi', 'si', 'sil'),
'rdx': ('rdx', 'edx', 'dx', 'dl', 'dh'),
'rcx': ('rcx', 'ecx', 'cx', 'cl', 'ch'),
'r8': ('r8', 'r8d', 'r8w', 'r8b'),
'r9': ('r9', 'r9d', 'r9w', 'r9b'),
}
FULL = {'rdi': ('rdi', 'edi'), 'rsi': ('rsi', 'esi'), 'rdx': ('rdx', 'edx'), 'rcx': ('rcx', 'ecx'),
'r8': ('r8', 'r8d'), 'r9': ('r9', 'r9d')}
ALIAS = {n: r for r, names in PARTS.items() for n in names}
PREFIX = {'lock', 'rep', 'repz', 'repe', 'repnz', 'repne', 'bnd', 'notrack', 'data16', 'cs', 'ds', 'es', 'fs', 'gs', 'addr32'}
READ_ONLY = {'cmp', 'test', 'push', 'bt', 'nop', 'ptest', 'ucomiss', 'ucomisd', 'comiss', 'comisd'}
# registers written by instructions that do not name them
STRING_OPS = {'movs': ('rsi', 'rdi'), 'cmps': ('rsi', 'rdi'), 'lods': ('rsi',), 'stos': ('rdi',),
'scas': ('rdi',), 'ins': ('rdi',), 'outs': ('rsi',)}
IMPLICIT = {'rdx': ('cdq', 'cqo', 'cwd', 'mul', 'div', 'idiv', 'rdtsc', 'rdtscp', 'cpuid', 'rdmsr', 'rdpmc',
'xgetbv', 'cmpxchg8b', 'cmpxchg16b'),
'rcx': ('loop', 'cpuid', 'rdtscp', 'xgetbv', 'syscall', 'sysenter', 'rdmsr', 'rdpmc'),
'rdi': (), 'rsi': (), 'r8': (), 'r9': ()}
BRANCH = re.compile(r'^(j\w+|loop\w*|jrcxz|jecxz)$')
LINE = re.compile(r'\s+([0-9a-f]+):\s+(?:[0-9a-f]{2} )+\s*(.*)')
def split_ins(text):
toks = text.split()
pfx = []
while toks and toks[0] in PREFIX:
pfx.append(toks.pop(0))
if not toks:
return pfx, '', ''
return pfx, toks[0], ' '.join(toks[1:])
def writes_reg(pfx, op, rest, reg):
"""True if the instruction (Intel syntax, prefixes split off) writes register reg."""
names = PARTS[reg]
anyname = re.compile(r'\b(?:' + '|'.join(names) + r')\b')
if op == 'call' or op == 'syscall':
return True
base = op.rstrip('bwlq') if op.rstrip('bwlq') in STRING_OPS else op
if base in STRING_OPS:
if reg in STRING_OPS[base]:
return True
if reg == 'rcx' and any(p.startswith('rep') for p in pfx):
return True
return False
if any(p.startswith('rep') for p in pfx) and reg == 'rcx':
return True
if op in IMPLICIT[reg]:
return True
if reg == 'rdx' and op in ('imul',) and ',' not in rest:
return True
if op in ('xchg', 'xadd', 'cmpxchg') and anyname.search(rest):
return True
if op in READ_ONLY:
return False
first = rest.split(',')[0].strip()
return bool(anyname.fullmatch(first))
def const_of(op, rest, reg):
"""Value if the instruction sets the full register to a constant, else None."""
f64, f32 = FULL[reg]
ops = [x.strip() for x in rest.split(',')]
if op == 'xor' and len(ops) == 2 and ops[0] == ops[1] and ops[0] in (f64, f32):
return 0
if op == 'mov' and len(ops) == 2 and ops[0] in (f64, f32) and re.fullmatch(r'-?(0x[0-9a-f]+|\d+)', ops[1]):
v = int(ops[1], 0)
return v & 0xffffffff if ops[0] == f32 else v & 0xffffffffffffffff
return None
def classify(ins, callpos, reg):
"""ins: sorted list of (pos, text) of one decoded window. Returns a value string or None (desync)."""
if not any(p == callpos for p, _ in ins):
return None
last = None
lastpos = -1
branches = []
for pos, text in ins:
if pos >= callpos:
break
pfx, op, rest = split_ins(text)
if not op:
continue
if BRANCH.match(op) or op in ('jmp', 'ret', 'retn'):
m = re.fullmatch(r'(?:0x)?([0-9a-f]+)', rest.split()[0]) if rest else None
branches.append((pos, op, int(m.group(1), 16) if m else None))
continue
if writes_reg(pfx, op, rest, reg):
last, lastpos = (op, rest), pos
if last is None:
return 'unreadable'
for pos, op, tgt in branches:
if pos > lastpos and op in ('jmp', 'ret', 'retn'):
return 'computed'
if tgt is not None and lastpos < tgt <= callpos:
return 'computed'
v = const_of(last[0], last[1], reg)
if v is None:
return 'computed'
return str(v) if v < 10 else hex(v)
def to_tool_path(objdump, p):
if objdump.split()[0] == 'wsl':
p = os.path.abspath(p).replace('\\', '/')
return '/mnt/' + p[0].lower() + p[2:] if len(p) > 1 and p[1] == ':' else p
return p
def load_elf(path):
data = open(path, 'rb').read()
if data[:4] != b'\x7fELF':
return None, None
phoff = struct.unpack_from('<Q', data, 0x20)[0]
phentsize, phnum = struct.unpack_from('<HH', data, 0x36)
segs = []
for i in range(phnum):
ptype, flags, off, vaddr, paddr, filesz, memsz, align = struct.unpack_from('<IIQQQQQQ', data, phoff + i * phentsize)
if ptype == 1:
segs.append((vaddr, off, filesz, flags))
return data, segs
def file_offset(segs, va):
for vaddr, off, filesz, flags in segs:
if vaddr <= va < vaddr + filesz:
return off + (va - vaddr)
return None
def scan(arr, segs, targets):
"""E8 (call) and E9 (jmp) rel32 instructions in executable segments whose target is in targets."""
res = {'call': [], 'jmp': []}
if not targets:
return res
tarr = np.array(sorted(targets), dtype=np.int64)
for vaddr, off, filesz, flags in segs:
if not flags & 1:
continue
seg = arr[off:off + filesz]
for op, name in ((0xE8, 'call'), (0xE9, 'jmp')):
idx = np.nonzero(seg[:-4] == op)[0]
if idx.size == 0:
continue
rel = (seg[idx + 1].astype(np.int64) | (seg[idx + 2].astype(np.int64) << 8) |
(seg[idx + 3].astype(np.int64) << 16) | (seg[idx + 4].astype(np.int64) << 24))
rel = np.where(rel >= 0x80000000, rel - 0x100000000, rel)
hit = np.isin(vaddr + idx + 5 + rel, tarr)
res[name] += [int(vaddr + i) for i in idx[hit]]
return res
def site_kind(data, segs, va):
fo = file_offset(segs, va)
if fo is None:
return 'outside file'
b = data[fo:fo + 3]
if b[:2] == b'\xff\x25':
return 'FF 25 stub'
if b[:2] == b'\xff\x15':
return 'FF 15 call [rip+got]'
if b[:2] in (b'\x48\x8b', b'\x4c\x8b'):
return '%02X 8B mov reg,[rip+got]' % b[0]
return 'other ' + b.hex(' ')
STARTS = range(64, 47, -1)
PAD = b'\xcc' * 16
def arg_values(objdump, data, segs, jobs, tmpdir, reach_neutral=False):
"""jobs: list of (va, reg). Returns {(va, reg): value string}. One objdump run for all windows."""
blob = bytearray()
wins = [] # (job index, back, start in blob, callpos in blob)
for ji, (va, reg) in enumerate(jobs):
fo = file_offset(segs, va)
for back in STARTS:
w0 = max(0, fo - back)
s = len(blob)
blob += data[w0:fo + 5]
wins.append((ji, s, s + (fo - w0)))
blob += PAD
out = {}
if not jobs:
return out
tmp = os.path.join(tmpdir, 'windows.bin')
open(tmp, 'wb').write(bytes(blob))
cmd = objdump.split() + ['-D', '-b', 'binary', '-mi386:x86-64', '-M', 'intel', to_tool_path(objdump, tmp)]
txt = subprocess.run(cmd, capture_output=True, text=True).stdout
pos_list, text_list = [], []
for line in txt.split('\n'):
m = LINE.match(line)
if m and m.group(2).strip():
pos_list.append(int(m.group(1), 16))
text_list.append(m.group(2).strip())
per = collections.defaultdict(collections.Counter)
for ji, s, c in wins:
lo = bisect.bisect_left(pos_list, s)
hi = bisect.bisect_right(pos_list, c)
v = classify(list(zip(pos_list[lo:hi], text_list[lo:hi])), c, ALIAS_REG[jobs[ji][1]])
per[ji][v] += 1
for ji, (va, reg) in enumerate(jobs):
vals = {k: n for k, n in per[ji].items() if k is not None}
if reach_neutral and len(vals) > 1 and 'unreadable' in vals:
del vals['unreadable'] # a shorter window simply does not reach the writer
if not vals:
out[(va, reg)] = 'desync'
elif len(vals) == 1:
out[(va, reg)] = next(iter(vals))
else:
out[(va, reg)] = 'ambiguous'
DETAIL[(va, reg)] = ', '.join('%s x%d' % (k, n) for k, n in sorted(vals.items(), key=lambda kv: -kv[1]))
return out
ALIAS_REG = ALIAS
DETAIL = {} # (address, register) -> answers of the start offsets for 'ambiguous' sites
def main():
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
ap.add_argument('root')
ap.add_argument('objdump')
ap.add_argument('names')
ap.add_argument('out')
ap.add_argument('--arg', action='append', default=[], help='NAME=REG (rdi rsi rdx rcx r8 r9)')
ap.add_argument('--titles')
ap.add_argument('--reach-neutral', action='store_true',
help="ignore 'unreadable' answers of short windows when a longer window reads a value "
"(default: any disagreement between start offsets is 'ambiguous')")
a = ap.parse_args()
names = a.names.split(',')
nids = {n: nid(n) for n in names}
argreg = {}
for x in a.arg:
n, r = x.split('=')
argreg[n] = ALIAS[r.lower()]
only = None
if a.titles:
only = set(re.findall(r'PPSA\d+', open(a.titles, encoding='utf-8', errors='replace').read()))
rows = {}
for f in glob.glob(os.path.join(a.root, 'PPSA*_r*', 'app.registry.json')):
folder = os.path.dirname(f)
m = re.match(r'(PPSA\d+)_r\w*$', os.path.basename(folder))
if not m or (only is not None and m.group(1) not in only):
continue
try:
d = json.load(open(f, encoding='utf-8'))
except Exception:
continue
key = (sum(1 for e in d if e.get('callSites')), os.path.getmtime(f))
if m.group(1) not in rows or key > rows[m.group(1)][0]:
rows[m.group(1)] = (key, d, folder)
lines = ['names: ' + ', '.join('%s=%s' % (n, nids[n]) for n in names),
'titles with a registry: %d' % len(rows), '',
'title | function | address | kind | value']
nonstub_lines = []
summ = {n: {'import': set(), 'calling': set(), 'call': 0, 'jmp': 0, 'thunk': 0, 'val': collections.Counter(),
'nonstub': collections.Counter(), 'noelf': set()} for n in names}
with tempfile.TemporaryDirectory() as tmpdir:
for t in sorted(rows):
key, d, folder = rows[t]
elf = os.path.join(folder, 'in', 'eboot.elf')
data = segs = arr = None
for n in names:
sites = sorted({int(x, 16) if isinstance(x, str) else int(x)
for e in d if str(e.get('nid', '')).split('#')[0] == nids[n]
for x in (e.get('callSites') or [])})
if not sites:
continue
summ[n]['import'].add(t)
if data is None and os.path.exists(elf):
data, segs = load_elf(elf)
arr = np.frombuffer(data, dtype=np.uint8) if data is not None else None
if data is None:
summ[n]['noelf'].add(t)
continue
stubs = []
for va in sites:
k = site_kind(data, segs, va)
if k == 'FF 25 stub':
stubs.append(va)
else:
summ[n]['nonstub'][k.split()[0] + k.split()[1] if k.startswith('FF') else 'other'] += 1
nonstub_lines.append('%s | %s | 0x%x | %s' % (t, n, va, k))
r = scan(arr, segs, stubs)
via = scan(arr, segs, r['jmp'])['call'] if r['jmp'] else []
vals = {}
if n in argreg and r['call']:
vals = arg_values(a.objdump, data, segs, [(va, argreg[n]) for va in r['call']], tmpdir, a.reach_neutral)
for va in sorted(r['call']):
v = vals.get((va, argreg.get(n)), '')
if n in argreg:
summ[n]['val'][v] += 1
if v == 'ambiguous':
v += ' [' + DETAIL.get((va, argreg[n]), '') + ' of %d start offsets]' % len(STARTS)
lines.append('%s | %s | 0x%x | call | %s' % (t, n, va, v))
for va in sorted(r['jmp']):
lines.append('%s | %s | 0x%x | tail jmp (not counted as call) |' % (t, n, va))
for va in sorted(via):
lines.append('%s | %s | 0x%x | call to jump stub (not counted as direct call) |' % (t, n, va))
if r['call']:
summ[n]['calling'].add(t)
summ[n]['call'] += len(r['call'])
summ[n]['jmp'] += len(r['jmp'])
summ[n]['thunk'] += len(via)
lines += ['', 'non-stub registry call sites: title | function | address | instruction kind']
lines += nonstub_lines or ['(none)']
lines += ['', 'summary: function | titles that import | titles with direct calls | direct calls | tail jmp | '
'calls to jump stub | non-stub registry sites | titles without readable ELF | argument values']
for n in names:
s = summ[n]
lines.append('%s | %d | %d | %d | %d | %d | %s | %d | %s' % (
n, len(s['import']), len(s['calling']), s['call'], s['jmp'], s['thunk'],
json.dumps(dict(s['nonstub']), sort_keys=True) if s['nonstub'] else '0',
len(s['noelf']), json.dumps(dict(s['val']), sort_keys=True)))
open(a.out, 'w', encoding='utf-8').write('\n'.join(lines) + '\n')
print('\n'.join(lines[-(len(names) + 1):]))
if __name__ == '__main__':
main()AI-assisted (Claude Code). |
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
sceNetResolverCreate(const char* name, int memid, int flags)(Export.cppL2336-L2344) returnsEINVALfor anyflagsother than 0 (L2339) and ignores the name and the memory id. With SysV argument registersflagsisrdx. The check was added in #9727 on the strength of prosper's reading of the libSceNet module ("Non-zero flags are EINVAL (+0xbae0)"), andcore/libs/tests/GuestSceNet.cpppins it for the flags 1, 0x10000 and -1. No entry ofdocs/dev/debt/libSceNet.mdnamed the function:ResolverCreatedoes not occur in the file; thesceNetResolverStartNtoaentry added in #14938 covers the flags of that call only. The new entry says what the code does, where the behaviour comes from, and what the titles checked pass:The entry is 476 characters; the counts stay out of it and are here. Changed:
docs/dev/debt/libSceNet.md, one new entry placed after thesceNetResolverStartNtoaentry. No code change.Count (attached script and output, one line per call site with title, address and value; the script is the one of #14236, sha256
77e3d27f…): 33 titles importsceNetResolverCreateand all 33 call it directly, 74 direct calls in total, andflags(rdx) is the constant 0 in all 74. There is one tail jump to the function, in PPSA10261 at0x3e42a47; that title also has two direct calls, both 0. There are no calls to a jump stub. A tail jump passes on the caller's registers, so itsflagsis not read.rdx)Method: for every direct call to the import stub the bytes before the call are decoded and the last write to the argument register decides:
xorof the register with itself or amovof an immediate is a constant, any other write is "computed", no write in the window is "unreadable", and decodings that disagree are "ambiguous". The run used--reach-neutral: an "unreadable" answer from a shorter decoding window is ignored when a longer window reads a value, because it only means that the write lies beyond the shorter window; every other disagreement stays "ambiguous". A tail jump is anE9jump to the stub; it passes on the caller's registers, so its arguments would not be read. Only the main executable of each title is read, from the 53-title registry.Overlap
Open pull requests that change
docs/dev/debt/libSceNet.mdorcore/libs/prx/libSceNet/Export.cpp(file lists from the GraphQL files of every open pull request, 1909 as of 2026-11-02 06:51 UTC,main4569fe654a; the 20 pull requests with more than 100 files were read through the REST files API): 39 touch at least one of them. None changessceNetResolverCreateor an entry on it, and none gives a new conflict with this change: each gives the same merge result with and without it. Three of them (#5836, #12104, #14212) conflict withmainwith or without it. #12683, #9932 and #9948 (socket info) merge cleanly and do not mention the function in their diffs. #9727 and #14938 are merged. None of the 14 pull requests opened since 07:10 UTC (#14975 to #14989, listed at 08:22 UTC) touches either file.Tested
Documentation only.
tools/check_conventions.py --base 4569fe654apasses; the entry is 476 characters; based onmain4569fe654a, and it merges withmain88e804b0a7without a conflict (checked 2026-11-02 08:22 UTC). Not built, not run.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)AI-assisted (Claude Code).
🤖 Generated with Claude Code