Skip to content

feat(fork): validate AIO, H.264 video and glyph rendering - #8

Draft
R3Dzf wants to merge 26 commits into
integration/multilib-usbd-all-fixesfrom
integration/io-video-font-completion
Draft

R3Dzf wants to merge 26 commits into
integration/multilib-usbd-all-fixesfrom
integration/io-video-font-completion

Conversation

@R3Dzf

@R3Dzf R3Dzf commented Oct 9, 2026 •

Copy link
Copy Markdown
Owner

What

Fork-only integration of the existing AIO, H.264 software-decoder and generated font-glyph implementations, layered on integration/multilib-usbd-all-fixes. This is an integration review in the fork, not an upstream implementation submission.

Original contributions retain their commit history and authorship:

The additional decoder fix rejects access units larger than the underlying decoder's signed integer size limit before pointer arithmetic or copying. Regression coverage verifies both oversized inputs and successful decoding in the same session after rejection.

Adds fork CI that builds the affected targets, runs their tests, builds the whole project and guest libraries, and runs the full registered suite on Linux and Windows.

The source progress scanner reports 2718/3044 exports (89.29%), compared with 2702/3044 on the integration base. This count does not establish game compatibility, complete audio output, shader correctness or support for encrypted SELF inputs.

Tested

Linux, GCC 13.3:

  • All six affected tests passed: guest_aio, guest_vdecsw, guest_videodec2, guest_font, guest_system_font and guest_font_writing_line.
  • The new oversized-input regression failed against the pre-fix implementation and passed with the fix.
  • The software decoder test passed with AddressSanitizer and UndefinedBehaviorSanitizer. LeakSanitizer was disabled because process inspection is unavailable in the execution environment.
  • Convention and test-registration checks passed locally.
  • Full project and guest-library builds passed on Linux and Windows in run 37987608948; all six affected tests passed on both hosts.
  • The full Linux suite ran 518 tests, with only global_vcc_base and flat_store exceeding the 30-second limit. The full Windows suite ran 509 tests, with only patched_libraries failing because a mismatched C++ runtime was selected.

Combined follow-up validation

The combined review branch integration/project-completion at commit 76f71f8 includes #8, #9, #10 and #11 plus the separately developed Windows runtime fixes. Results below apply to that combined commit; the individual PR heads retain the baseline results above.

  • Full build and test run 38033998905: Linux built the full project and all guest libraries, then completed 523 registered tests with zero failures and 27 explicit skips. Windows built the full project and all guest libraries, then completed 514 registered tests with zero failures and 211 explicit skips. The patched_libraries loader test passed in 0.06 seconds.
  • Cold Linux affected-test timings were 25.05 seconds for flat_store and 15.91 seconds for global_vcc_base. The subsequent full-suite timings were 4.20 and 2.89 seconds and can benefit from the driver cache. The 30-second limits remain in effect.
  • The main-based Windows exception topic fixes future-result destruction using the wrong exception-header layout, native primary-exception initialization and make_exception_ptr reference ownership. Native future helpers and RTTI remain exported under their GNU names after NID patching. Failed video initialization owns its startup promise and completes libc shutdown.
  • Standalone source verification 38033956388 reproduced a crash after restoring the old runtime, then passed 20 repeated exception tests and 400 fresh video-initialization processes with the fix. All 400 video runs reported unavailable video/Vulkan facilities with exit 77; this verifies failure cleanup rather than successful rendering. A separate real NID-patch probe retained all eight native future exports and passed another 200 processes, each with 100 cross-thread exception transfers.
  • New-delta convention checks and the standalone exception topic pass. The combined main-based history still has eight inherited integration(...) commit-subject findings, which remain unresolved for review.

Commercial games, a physical discrete GPU, PCIe performance and console execution have not been tested. The scanner's export-body counts do not establish semantic completeness: encrypted SELF conversion, complete Tempest audio, predictive GPU memory management and full Mesh/Ray Tracing support remain outside the implemented changes.

Checklist

Owner-requested integration scope: this batch is based on the existing main-derived integration branch and intentionally reuses the three upstream requests listed above, rather than claiming new implementations.
Follow-up development uses separate topic branches and requests.

awake-devel and others added 26 commits October 8, 2026 10:47
…nity

sceKernelGetCurrentCpu returned the raw host CPU index. Guest affinities
are bookkeeping only and sceKernelGetAvailableCpumask advertises CPUs 0
to 12, so on a host with more than 13 logical CPUs a title could get 13
or higher, and a thread pinned to one CPU could get another one.

The host index now selects a CPU of the calling thread's affinity
clipped to the available mask, so the result is always an advertised
CPU and a thread pinned to one CPU reports that CPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeC9bA3n74TzJRYv9AgBE8
A thread that sceKernelRaiseException redirects resumes through
RtlRestoreContext with a CONTEXT that has no XSTATE, so the upper halves
of ymm0-ymm15 came back as the handler and the delivery path left them.
A handler or host routine that runs vzeroupper or any VEX instruction
(Zen 2 guest code, the CRT's memcpy) cleared the upper lanes of a thread
interrupted in the middle of AVX code. The entry stub now stores
ymm0-ymm15 next to the delivery when the CPU and the OS enable AVX, and
the exit loads them back right before RtlRestoreContext, which restores
only the low halves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeC9bA3n74TzJRYv9AgBE8
…d wait

A condition wait on a recursive mutex locked more than once unlocked the
native mutex only once, so the thread that should signal blocked forever
on it. Unlock the remaining levels before the wait and lock them again
after it, as FreeBSD libthr does in _mutex_cv_unlock and _mutex_cv_lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeC9bA3n74TzJRYv9AgBE8
Capturing a shader threw "null or misaligned address" when the flat
SRT walk read through a pointer loaded from memory that is null or
unmapped, even when the program skips that s_load. Such a read now
yields zero and a record with its pc and address, as does a slot whose
address needs it. The registered artifact checks one flag word per
guarded slot at each ReadConst; the capture appends the flags and the
records to the flattened SRT, and a set flag raises an Unmapped BDA
fault through the fault buffer and stops the invocation as a BDA read
does. Nothing per capture enters a key: poisoned and clean captures
share one variant and specialization. Descriptors, branch conditions,
uniform fills, table bases and user-data pointers that need such a
value still throw, and poisoned results stay out of the dispatch and
draw caches. The shader disk cache format becomes 21.
Implemented with AI assistance.
An address-based build maps each imported registered range through its
host import. UploadPrepare merges a descriptor range or snapshot
overlapping such a range into the range's region, and the region kept
the import even when the merge took it past the import's end, as a V#
crossing into the next registered range does. Descriptor then bound the
import's buffer beyond its size, and the BDA table mapped the whole
region, the next range included, through the import's device address.

A merge that extends a region past its end now drops the import. The
region then takes the import lookup and, with no import covering it,
the copy path, as a range crossing an image mirror's end already does.
Like every copied range, it is read-only to pointer stores.
libSceVdecsw is the software H.264 decoder interface PPSA05684 uses for
its movies. Decoder creation, compute queues, memory queries, decode
input and output, AVC picture info, reset and finalize are implemented
with the layouts of shadPS4's vdecsw.h, checked against the title's
movie decoder, and decode through libSceVideodec2. The H.264 stream the
Videodec2 test uses moves to a shared fixture, and a guest test decodes
it through Vdecsw.

Uncharted: Legacy of Thieves Collection (PPSA05684) plays its movies through libSceVdecsw and stopped at sceVdecswQueryComputeMemoryInfo.
…address-based builds

A build served by the cached address space walked every region of the
space for its mirrors, sorted the heap mirrors, compared each heap
mirror's blocks with the write tracker under its own lock and swept the
whole mirror registry. In an offline replay with 37,000 heap mirrors
that took about 10 ms per build, every build.

The space now keeps its writable mirrors, its sorted heap mirrors and
their adjacent runs from when it is published. Each run is still
flushed and collected; a run with no block stamped since the generation
its last complete check collected skips the per-mirror compare, and that
generation is stored only once the build's blocks were copied. The
registry sweep is skipped while every live mirror's range is held by the
cached address space's lease and no mirror was added, replaced or
dropped since.
An address-based build maps every readable registered range: through a
host import when one can be made, else through a heap mirror, a
host-visible copy of the range. A range mirrored while the import budget
was full kept its mirror after a later pass imported it, until the guest
freed the range. The pass then read the range through the import, and the
mirror only held host-visible memory and a second copy of the bytes.

The pass now releases the heap mirror of every range it serves through
an import. Recorded work that bound the mirror keeps it until it
completes. If the import goes away while the range stays registered, the
next pass makes a new mirror from the guest bytes. The [buffers] image
mirrors profile line counts the released mirrors.
An address-based build whose heap mirror the device refused, or one past
APS5_HEAP_MIRROR_MIB, stopped the process inside acquireMirror. PPSA26344
hit it at 478 s with 6.8 GiB of heap mirrors beside the host imports.

- a mirror that would go past the budget first sweeps the mirrors whose
  registered range is gone, before its allocation; if it still does not
  fit, the build throws "past the budget"
- the mirror is allocated once: a refusal throws with the range, the
  Vulkan result and the mirror totals, so the draw or dispatch fails like
  any other allocation; nothing is registered for the refused range
- the BDA resource tests run with a 4 MiB mirror budget and a mock memory
  limit: a mirror past the budget is made in the swept mirror's memory,
  and a refused one fails after a single allocation attempt
The device-local tier of BufferPool only reused a buffer of the exact
size and usage it was asked for. Partial storage-image detiles and
retiles ask for a new size and the other transfer direction almost
every time, so the tier sat at its budget and missed and evicted
dozens of buffers per frame, and the vkFreeMemory calls on the release
thread held up submits and allocations on the recording threads.

Device-tier requests of 1 MiB and more now round up to an eighth of
their octave, the tier's buffers are created for storage and both
transfer directions, and a request takes the smallest retained buffer
from its own class up to twice its size. Host tiers, the budget and
eviction are unchanged. agc_buffer_pool covers the classes, the fit
and its bound, reuse only after release, the budget and the unchanged
lists.
GitHub refreshes merge_commit_sha only when the pull request's branch
changes, so a manual build could test the pull request against a main
from days earlier. Each job now checks out main as it is when the build
starts and merges the pull request head into it.
…rrent-main

ci(build): build pull requests merged into current main
…n-guest-mask

fix(libkernel): report the current CPU within the thread's guest affinity
…delivery

fix(libkernel): keep ymm upper halves across a redirected raise
…ursive-mutex

fix(libkernel): release every level of a recursive mutex during a cond wait
…ge-past-import

fix(agc): copy an imported range once a merge extends it past its import
…budget

fix(agc): throw instead of aborting when a heap mirror cannot be made
…heap-mirrors

perf(agc): release a heap mirror once a host import serves its range
…or-runs

perf(agc): skip unchanged heap mirror runs and idle mirror sweeps in address-based builds
…ool-reuse

perf(agc): pool device staging buffers by size class
…ll-pointers

feat(agc): implement deferred faults for reads through null SRT pointers
Integrate upstream boykopovar#2118 (Ruzi), boykopovar#2203 (XairRrR), and boykopovar#2150 (erick-dsnk), preserving their original commits as merge parents. Resolve font and technical-debt conflicts against the existing fork integration.

Six targeted Linux CTests passed. Source-based implementation coverage is 2718/3044 (89.29%), not game compatibility. Native console behavior and complete Windows/full project validation remain unverified.

AI-assisted integration: yes.
Validate the delegated decoder's INT_MAX input limit before forming an end pointer or allocating packet storage. The regression fails before the guard and passes with it; it checks unchanged queues and correct decoding after rejection.

Six targeted Linux CTests passed. ASan/UBSan passed for Vdecsw code; LeakSanitizer is unavailable in the local environment. AI-assisted: yes.
Run six affected CTests before the full build and test suite. The workflow is limited to this experimental integration branch and uses the project's supported Windows toolchain.

AI-assisted: yes.
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Progress report (86a4c74 → d6a1f6a)

📈 System libraries: 89.29% (+0.53%, +16 implemented functions)

✅ 16 implemented
Library Name
libSceFont sceFontGlyphRenderImageHorizontal
libSceFont sceFontGlyphRenderImageVertical
libSceVdecsw sceVdecswAllocateComputeQueue
libSceVdecsw sceVdecswCreateDecoder
libSceVdecsw sceVdecswDeleteDecoder
libSceVdecsw sceVdecswFinalizeDecodeSequence
libSceVdecsw sceVdecswGetAvcPictureInfo
libSceVdecsw sceVdecswQueryComputeMemoryInfo
libSceVdecsw sceVdecswQueryDecoderMemoryInfo
libSceVdecsw sceVdecswReleaseComputeQueue
libSceVdecsw sceVdecswResetDecoder
libSceVdecsw sceVdecswSetDecodeInput
libSceVdecsw sceVdecswSetDecodeOutput
libSceVdecsw sceVdecswTrySyncDecodeInput
libSceVdecsw sceVdecswTrySyncDecodeOutput
libkernel sceKernelAioPollRequests

@R3Dzf R3Dzf changed the title integration(fork): validate AIO, H.264 video and glyph rendering feat(fork): validate AIO, H.264 video and glyph rendering Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants