Skip to content

feat(libSceAvPlayer): read progressive media over HTTP - #10957

Open
AZLisme wants to merge 7 commits into
boykopovar:mainfrom
AZLisme:feat/avplayer-http-source
Open

AZLisme wants to merge 7 commits into
boykopovar:mainfrom
AZLisme:feat/avplayer-http-source

Conversation

@AZLisme

@AZLisme AZLisme commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What

Add progressive HTTP media support to AvPlayer through a reusable libSceHttp client, as requested by the maintainer. Both libSceAvPlayer variants dynamically link the shared provider. Eight canonical extern "C" / APS5_VABI helpers use _nid_postfix names so the real NID patcher can patch imports from our libraries. The provider owns handle allocation and destruction; its implementation stays private.

AvPlayer keeps the FFmpeg AVIO adapter and catches both read and seek exceptions before returning through FFmpeg. The shared client preserves redirects, range and framing checks, representation validators, caching, independent handles, reopen and start/stop behavior. Existing Sony sceHttp* request/context APIs remain unchanged. No third-party code is added.

Debt records the remaining plaintext-HTTP, eager initial-response buffering, blocking DNS and console-equivalence limits.

The Windows build follow-up b27b2a83311db49b72e9ffaa9d1b0360c5f16711 adds a MinGW-only native-unwind compile option to the host-only module ownership probe. Production sources, the eight helper signatures, provider/adapter behavior and fixture assertions are unchanged; no guest-runtime unwind setting is changed.

Follow-up dbdc41e773010bc23ad743e793731149d3c71a50 gives the six existing private ForTest_nid_no_patch host helpers canonical C linkage. Their actual exported names now end in _nid_no_patch, so normal provider patching and the fixture’s preserved imports agree. This is a general linkage correction, with no per-name patcher exception. Namespace call sites, production function bodies, fixture assertions and the eight canonical HTTP helper signatures remain unchanged.

Tested

Current-head upstream Windows job 122459254778, run 40580762434 attempt 1, succeeds for dbdc41e773010bc23ad743e793731149d3c71a50 merged onto main 4a6005955ff169db8d66fc43ede7ac0cc426ee4a. It links 27 selected host test targets, including avplayer_http_patched_modules_tests.exe; HTTP/AvPlayer libraries also compile, link and patch as dependencies. The full build, explicit library build, CTest and skip-validator steps were skipped. This automatic job supplies build/link evidence, with no runtime-test result.

Corrected source, qualified Linux composition: private product c13e5db7a961eb29d313ff88591cecd09101bbde on frozen main e0cd1259a95e6dfaf97f8f73e74410bbc5d8d199 contains the nine owned topics, still-open #9448, this private HTTP fix and the exact proposed #11985 fixture projection. The paired control b4506efc625689282c290ac3dc53c8805d52edce is main plus that same fixture projection, not bare main. These are private QA sources, not actual prerequisite merges or standalone dbdc validation.

The normal-root Linux Release lane passes 76 selected CTests: 10 HTTP/AvPlayer, 65 public-header checks and one projected debt-fixture test, with zero failures or skips. Seven objects recompiled and three outputs relinked; 392 unchanged QA objects were qualified for reuse. Actual normal/native providers, before and after real NID patching, export all six plain helper names; normal and patched player executables import those same names. The patched HTTP path passes all 16 owner bindings, client body checks and regular player network/decode. The repository-pinned prebuilt FFmpeg package remains outside any new source-build or instrumentation claim. This Linux ELF proof does not identify the old Windows loader’s first missing import or establish a native Windows fix.

Native Windows, corrected controlled compositions: run 40566638846, attempt 1 uses harness 57804c82f6ea and frozen source inputs. Union job 122409555324 tested product c13e5db7a961, containing the exact dbdc41e77301 correction. The full normal build, patched libraries, source-before/after checks, full CTest, expected-skip validator and final job gate passed. Full CTest: 1,853 passed, zero failures, 832 expected skips (2,685 cases). All 112 required focused tests passed without skips, comprising 47 functional tests and 65 public-header checks. All ten HTTP/AvPlayer tests passed. The patched-module fixture verified all 16 HTTP provider-owner bindings, both AvPlayer consumers, the patched client body and regular patched player HTTP/decode behavior; the prior player launch failure is resolved in this composition.

Control job 122409556161 tested b4506efc6256, frozen main plus the exact proposed #11985 test-fixture change: 1,837 passed, zero failures, 832 expected skips (2,669 cases), including all 65 header checks. This is a fixture-applied control, not bare main. All 2,669 common case outcomes agree; the union's sixteen additional cases pass. Compared with the prior failed run, test names and expected-skip names are unchanged; the shared debt-fixture failure passes in both cases and the patched HTTP failure passes in the union. Artifact ZIP digests and sealed evidence manifests were verified. These results establish the controlled compositions above, without claiming standalone PR/current-main execution, actual prerequisite merges, console validation or a new ASan run.

Earlier Windows QA found a failing patched-player regression. Run 40549933171, attempt 1, union job 122350610019, tested the private composition 54c6a452cc948273a9de6d9d9a12089547672c6b on main e0cd1259a95e6dfaf97f8f73e74410bbc5d8d199, containing this exact b27b2a83311d head, still-open #9448 and the other eight owned topics (frozen inputs). This is composition-specific evidence.

Normal build, patched libraries, source verification and all 65 public-header tests passed. HTTP coverage passed 9/10: avplayer_http_patched_modules verified all 16 provider-owner rows and the patched client, then its patched player exited 0xc0000139. The full suite reported 1,851 passes, two failures and 832 expected skips; skip validation passed. The second failure, conventions_debt_length, also failed on the main baseline and is addressed by open #11985.

Captured PE exports and source rules show six private C++ test-helper names being renamed to NIDs while the executable's imports are preserved. The exact first failing executable import was not captured. The C-linkage correction above passes the controlled Linux and native Windows validation described above. The failed run remains historical evidence, with its original source and scope preserved.

Windows run 40485913793, attempt 1, job 122127573914, tested 3043f029726e with main 3fea2a937ba9fbab2f06c51691fe0c5763659aa3 and failed while linking the host module probe with unresolved __gxx_personality_v0. CTest did not run in that Windows job. The three-line MinGW override above selects the host compiler's native unwinding for this pure-host executable. Windows run 40491012301, attempt 1 passes configure, registration and all selected host-only compile/link targets for b27b2a83311d on the same main 3fea2a937ba9 and WinLibs GCC 15.2 toolchain. The previously failing module probe compiles and links successfully. CTest was skipped, so this verifies the link fix without adding Windows runtime or header-test results.

For b27b2a83311d, normal-root Linux reconfiguration and registration checks passed; the selected target build had no work. Selected compile/link commands, the complete compile database, 234 objects, 435 compiler inputs, 671 system/generated inputs and 22 artifacts match the qualified 3043 build. One current avplayer_http_patched_modules CTest passed without skips or failures, including 16 provider-owner bindings and real NID-patched client and regular AvPlayer HTTP decode. The earlier Release/header and ASan suites below were not rerun; their execution remains on 3043, with the Linux-inactive build change and explicit equivalence checks recorded separately.

Linux, source 3043f029726e on base aebeaa53f54d, normal-root GCC 16.2.1 / CMake 4.4.4 builds: all 10 focused Release tests and all 65 public-header checks passed with zero failures or skips. All 10 focused ASan tests also passed with leak detection enabled. Changed backend/header consumers were rebuilt; retained objects were checked against unchanged compiler inputs. The repository-pinned FFmpeg package is prebuilt and is outside the ASan instrumentation claim.

Tests exercise payloads, fixed/chunked/close framing, ranges, cache and seek behavior, validators, reopen, handle isolation and synchronized cancellation during open/read. The module fixture stages six normally built modules and applies the normal export-list/NID-patcher commands. Both AvPlayer variants import all eight helpers from the HTTP provider; both provider-ownership checks pass. Patched direct-client and regular AvPlayer loopback/decode scenarios pass. Coverage is the selected six-module closure.

Earlier maintainer Linux/Windows CI applies to fd13526ec6a6 with main 0a570f617d85; it remains historical. The controlled native Windows validation above passes; applicable full maintainer CI and re-review remain pending. Manual overlap review found independent AvPlayer changes in #1601/#7412/#7622 and guest HTTP transport work in #9834/#11134; preserve their distinct CMake/debt changes if they land first.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7622 core/libs/prx/libSceAvPlayer/src/Source.cpp L4

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1601 core/libs/prx/libSceAvPlayer/src/Source.cpp
core/libs/tests/GuestAvPlayer.cpp
PR 9834 core/libs/prx/libSceHttp/CMakeLists.txt
docs/dev/debt/libSceHttp.md
PR 11134 core/libs/prx/libSceHttp/CMakeLists.txt
docs/dev/debt/libSceHttp.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head d4a68788b98c1bb6a734e655d51adc8cb5c5cff0. I traced request framing, range and ignored-range handling, content-range validation, validators, redirects, chunk decoding, cache offsets, AVIO seek/read and stop propagation. The 200 response path correctly keeps cacheStart = 0 and validates the full representation length before serving bytes requested from a nonzero offset; the 206 path requires exact requested start and body/range-length agreement. The new tests cover both cases plus chunking, redirects, cancellation/restart and callback priority. Local git diff --check and python3 tools/check_conventions.py --base upstream/main --head refs/review/10957 passed. Hosted eligibility, intake, conventions, labels and report checks pass; product build jobs were skipped. The 5/5 Linux runtime results are author-reported; I have not independently run them, and Windows/macOS/PS5 execution remains unverified.

@wachi35676

Copy link
Copy Markdown
Contributor

Windows run, which the description lists as not performed. Head d4a68788b98c merged onto main 90f918fa7, WinLibs GCC 15.2.0, Release, Python 3.13 test server.

  • guest_avplayer_http and avplayer_http_transport pass against AvPlayerHttpServer.py on loopback, 2 runs. That exercises the Winsock side of HttpStream.cpp: range reads and seeks, chunked framing, redirects, cancel and restart.
  • guest_avplayer, guest_avplayer_callback_close and guest_avplayer_paused_first_video still pass.

AI-assisted: yes (Claude Code); results above were run on this machine.

@AZLisme

AZLisme commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Thank you @wachi35676 for testing exact head d4a68788b98c through Winsock and the loopback HTTP server, plus the three AvPlayer regressions. I’ve credited your report in Tested, preserving its two-run scope for the HTTP targets and separating it from my Linux checks. No hosted product CI or console validation is inferred.

@oneandonlydean oneandonlydean self-assigned this Oct 26, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@AZLisme One change in the tests: please remove the loop at GuestAvPlayer.cpp:2074-2079 that requires the HTTPS, FTP and HLS (.m3u8) sources to throw. Those are valid source forms that just aren't supported yet, so the test would fail as soon as one of them is implemented. The explicit unsupported paths and the debt entry can stay as they are.

Please keep the positive HTTP MP4/WebM and callback-priority checks, the invalid framing/CRLF checks at 2071-2072, and the late-response failure check at 2080-2083.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@boykopovar before we review this further: it adds an HTTP client inside libSceAvPlayer so titles can stream progressive media from a URL. Is that a feature you want in AnyPS5, or should AvPlayer stay limited to local files for now? @AZLisme thanks for the work; we'll hold the review until that's settled.

@AZLisme

AZLisme commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

@oneandonlydean Removed the six-line HTTPS/FTP/HLS required-throw loop in fd13526ec6a6ef536ee8a639e58da5a876b25946. The positive HTTP MP4/WebM and callback-priority checks, invalid framing/CRLF checks and late-response failure checks are unchanged. Production unsupported paths and debt are unchanged.

Both existing HTTP tests pass with the real Python server and a fresh source-aligned core/AvPlayer build; exact source/artifact bindings and independent review pass. Conventions and a static merge pass against 4e43321bd6ceccdc1ae28568bbbc35d9a6446464. This is Linux evidence for the revised fixture, separate from the older D4 Windows report.

I understand that further feature review and merge are on hold pending the maintainer decision you requested. This test-only update leaves that decision open.

@AZLisme

AZLisme commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Run 40452627814/a1 passed Linux and Windows Release CI for the single #10957 fd13526e + main 0a570f61 composition. Both HTTP/transport tests and the other three AVPlayer regressions passed, alongside headers and skip validation. The owner feature-decision hold remains unresolved. If the owner approves the feature direction, re-review and eventual merge can be considered after integration and review requirements are satisfied. CI success does not resolve that decision.

@boykopovar

Copy link
Copy Markdown
Owner

@boykopovar before we review this further: it adds an HTTP client inside libSceAvPlayer so titles can stream progressive media from a URL. Is that a feature you want in AnyPS5, or should AvPlayer stay limited to local files for now? @AZLisme thanks for the work; we'll hold the review until that's settled.

That's good. It's just better to place the HTTP client somewhere it can be reused. For example, we could add it to libSceHttp and dynamically link libSceHttp to libSceAvPlayer. Then we could link libSceHttp to other libraries whenever they need an HTTP client, without writing a new one. In order for nid_patcher to patch imports from our libraries, need to add _nid_postfix to the function names.

@oneandonlydean

Copy link
Copy Markdown
Collaborator

Thanks @boykopovar, that settles the direction. @AZLisme: could you move the HTTP client into libSceHttp, as reusable exports there, and have libSceAvPlayer link libSceHttp dynamically, so other libraries can use the same client later? The AvPlayer side then calls into libSceHttp instead of carrying its own. We'll re-review once that restructure is up; the current CI run and tests are a good base for it.

@AZLisme
AZLisme force-pushed the feat/avplayer-http-source branch from fd13526 to 3043f02 Compare October 27, 2026 10:06
@AZLisme

AZLisme commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Moved the reusable client into libSceHttp and linked both AvPlayer variants to it in 3043f029726e. The eight canonical C helpers use _nid_postfix; the real patcher and both provider-ownership checks pass. AvPlayer retains the read/seek exception firewall.

Normal-root Linux Release passes 10 focused tests plus 65 header checks; ASan passes all 10 focused tests with leak detection enabled. Both patched-client behavior and regular AvPlayer HTTP decode pass. The canonical handle now lives in its header, with private provider storage.

Please rerun current-head maintainer CI and re-review the shared-client arrangement; merge when the integration and review checks are satisfied.

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Progress report (4a60059 → dbdc41e)

📈 System libraries: 48.81% (+0.03%, +8 implemented functions)

✅ 8 implemented
Library Name
libSceHttp Aps5HttpClient_Create_nid_postfix
libSceHttp Aps5HttpClient_Destroy_nid_postfix
libSceHttp Aps5HttpClient_GetSize_nid_postfix
libSceHttp Aps5HttpClient_Open_nid_postfix
libSceHttp Aps5HttpClient_Read_nid_postfix
libSceHttp Aps5HttpClient_Seek_nid_postfix
libSceHttp Aps5HttpClient_Start_nid_postfix
libSceHttp Aps5HttpClient_Stop_nid_postfix

Changed libraries (implemented exports)

Library Before After Change
libSceHttp 86/115 (74.78%) 94/123 (76.42%) +1.64 pp

@AZLisme

AZLisme commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the Windows host-probe link failure in b27b2a83311db49b72e9ffaa9d1b0360c5f16711 with a three-line MinGW-only CMake override selecting native unwinding for that pure-host executable. Production libraries, the HTTP client/adapter, all eight helper signatures and test assertions are unchanged.

The 3043f029726e / main 3fea2a937ba9 Windows job in run 40485913793 stopped at __gxx_personality_v0 while linking the probe; it did not execute CTest.

Normal-root Linux configure/registration passed with a no-work build. The selected compile/link commands, complete compile database, objects, inputs and artifacts remain equal to 3043; one current patched-module CTest passed, including both provider-owner checks and real patched client/player network behavior. Prior 3043 Release 10+65 and ASan 10 remain historical executions, with the Linux-equivalence limits recorded explicitly.

Please recheck the new Windows job when it runs, then re-review and merge once the integration and review checks are satisfied. Windows success is not claimed yet.

@AZLisme

AZLisme commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Windows job 122144698466 now passes for b27b2a83311d on the same main 3fea2a937ba9 where 3043 failed. The HTTP module probe compiles and links successfully with the target-local MinGW unwind fix.

This automatic job ran configure, registration and host-only build/link checks; CTest was skipped. Windows runtime and full-suite validation remain pending. Please re-review and merge when the remaining integration and review checks are satisfied.

@AZLisme

AZLisme commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Windows QA found a failure in this exact b27b2a83311d head's new patched-module regression, on private union 54c6a452cc (run). Builds and 65 header checks passed; 9/10 HTTP tests passed. The fixture verified all 16 HTTP provider rows and its patched client, then the patched player exited 0xc0000139.

The other full-suite failure is the inherited debt-length fixture, also reproduced on baseline; #11985 covers it. Static PE/source review found six private C++ test-helper exports renamed to NIDs while their incoming names are preserved. The first failing executable import itself was not captured. A private C-linkage fix is being validated on Linux and then Windows. The body records the failed run and its scope. Please hold merge until the correction is validated.

@AZLisme

AZLisme commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Fixed the patched Windows player regression in dbdc41e773010bc23ad743e793731149d3c71a50. The six existing private test helpers now have C linkage, preserving their terminal _nid_no_patch names through normal patching. Production function bodies, all eight shared HTTP helper signatures and test assertions are unchanged.

Native Windows run 40566638846/a1 passes the full suite on controlled union c13e5db7a961: 1,853 passed, zero failed, 832 expected skips; all 112 required focused tests pass, including all ten HTTP/AvPlayer tests and 65 public-header checks. The previously failing fixture now completes both provider checks, patched client and regular patched player HTTP/decode. The paired fixture-applied control b4506efc6256 passes 1,837 tests with the same 832 expected skips. Both compositions include the exact proposed #11985 test-only fixture correction; this control is not bare main and no prerequisite merge is inferred. Linux on the corrected composition passes 10 HTTP tests, 65 header checks and the projected fixture test.

Additional upstream Windows validation: run 40746237530/a1, Windows job passed for #10957 dbdc41e773010bc23ad743e793731149d3c71a50 merged alone onto pinned main 319621ba2e578bd053b0baba2877fe70520e0737. The normal Release build, production libraries and registration audit passed. CTest recorded 2,003 passed, 862 expected skips, zero failures, including all ten owned HTTP/AvPlayer/client fixtures and all 65 public-header tests registered in this job; avplayer_http_patched_modules passed in 10.72 seconds. This was cached CI: 3,499/3,929 compiler-cache hits, 430 misses, with configure-seed and dependency restores. The 47 diagnostics remain recorded, including two UTF-8 path deprecations and the check_main_sync timing warning. These results cover this Windows job only; whole-workflow, other-platform and existing review/integration requirements remain separate.

Paired upstream Linux validation: run 40746237530/a1, Linux job passed for the same #10957 dbdc41e773010bc23ad743e793731149d3c71a50 and pinned main 319621ba2e578bd053b0baba2877fe70520e0737. Normal Release configuration, registration, build and libraries passed. CTest recorded 2,660 passed, 100 expected skips, zero failures, including all ten owned HTTP/AvPlayer/client fixtures and all 65 actual public-header tests; skipped device paths were not executed. This was cached CI: 8,084/8,108 compiler-cache hits, 24 misses, 98 uncacheable calls and persistent Mesa cache. The 69 warning headers include 67 compiler warnings, including two UTF-8 path deprecations, plus checkout and Opus notices; no GitHub warning annotations. Later normal CI metadata records overall workflow SUCCESS; four optional jobs were skipped. Other platforms, GPU validation, runtime internals, final tree attestation and existing source overlap and re-review requirements remain separate.

The body records exact sources, prior failures and validation limits. The earlier patched-player hold is resolved by this correction and validation. Please run applicable current-head maintainer CI, re-review, and merge once the remaining integration and review requirements are satisfied.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants