[babyhuey]

What

sceVideoOutOpen on a bus that is already open now returns SCE_VIDEO_OUT_ERROR_RESOURCE_BUSY (0x80290009) instead of throwing VideoOut: port already open.

• VideoOutDriver::Open returns -1 for an open bus, and sceVideoOutOpen turns that into the error. It already mapped a negative handle to VIDEO_OUT_ERROR_RESOURCE_BUSY, but threw it.
• This follows shadPS4's PS4 driver, which returns ORBIS_VIDEO_OUT_ERROR_RESOURCE_BUSY when its main port is already open. The PS5 result is unverified, and that's recorded under Functional in docs/dev/debt/libSceVideoOut.md.
• guest_video_out_output_mode checks that a second open of the main bus returns the error while the first handle stays open.

Where a title reaches it

Porpoise (PPSA99764, Porpoise 3.0) opens the main bus twice. On main 1c5e628 + #4690 (/data at start-up, which it needs to get past its jailbreak check), Windows 11 (WinLibs GCC 15.2, RTX 3080):

• Without this PR it brings up its Vulkan stack (PlayStation 5 GPU (RADV NAVI21)) and stops at VideoOut: port already open.
• With this PR (fork test/verify-busystack 52ec5e4) the second open returns the error and Porpoise carries on. It logs vk: the console outputs 0x0 (handle 0x80290009, status 0xffffffff) for its own display probe, then creates its swapchain (1920x1080, 3 images), presents through the first handle (vk: VideoOut output 1 for V-Sync) and renders. In two 60 s runs it kept up about 60 GPU submissions per second (radv/ps5 submissions: window count=600), with no throw.

On Linux the title doesn't get this far in a headless run.

Tested

On the PR head c15e9f2, Release:

• Windows 11 (MinGW-w64 GCC 15.2 from WinLibs): guest_video_out_output_mode 5/5 (--repeat until-fail:5) with ANYPS5_REQUIRE_DISPLAY=1, so the open and the second open ran with a real window.
• Linux (Ubuntu 24.04, GCC 13, headless): guest_video_out_output_mode 5/5. The display part is skipped there, as before.
• No warnings in Output.cpp, VideoOutDriver.cpp or GuestVideoOutOutputMode.cpp.

check_conventions.py passes.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none (fix(libkernel): create /data at start-up #4690 has merged; Porpoise stops here on main)
• Unblocks: PPSA99764 Porpoise (DolphinPS5) (PPSA99764) #4854
• AI-assisted: yes

[babyhuey]

Ran the rest of the VideoOut tests on c15e9f2 against a real display (Windows 11, RTX 3080, WinLibs GCC 15.2), since headless they mostly skip:

• With ANYPS5_REQUIRE_DISPLAY=1 under ctest, these pass: agc_driver_flip, agc_driver_flip_compute, video_out_output_mode_policy, guest_video_out_high_frame_rate, video_out_flip_decode, guest_video_out_wqhd_detection, guest_video_out_resolution_status, guest_video_out_output_mode.
• ctest gives the other flip tests SDL_VIDEODRIVER=${ANYPS5_VIDEO_OUT_TEST_DRIVER} (offscreen in this build), so I ran video_out_flip_tests.exe directly with SDL_VIDEODRIVER=windows, one process per mode. lifetime, reopen, controls, overlay, unopened and decode all printed VideoOut flip tests passed and exited 0, each with a real 1920x1080 window.

reopen is the one closest to this change, and it's unaffected: a bus can still be opened again after it's closed.

[babyhuey]

#4690 has merged, so Porpoise now reaches this on plain main. On main c13dc7e, Windows 11 (RTX 3080, WinLibs GCC 15.2), it gets past its jailbreak check, brings up its Vulkan stack and stops at:

PlayStation 5 GPU (RADV NAVI21)
[libc] unhandled exception 'St13runtime_error' thrown from ...
  what():  VideoOut: port already open

With this PR on the same base it carries on and renders, as described above. I've updated "Depends on" to none.

[LeandroLP23]

Porpoise (PPSA99764, DolphinPS5) on Windows. Windows 11 (26300), Radeon 860M, WinLibs GCC 15.2.0, Release. Head c15e9f2a3 merged onto main 88107b7de, relinked with --windows --trap-sdk-syscall-stubs.

• main: VideoOut: port already open, exit 0xc0000409 after 5 s.
• With this PR: the window (Porpoise (DolphinPS5)) stays up for the full 60-second run.

AI-assisted: yes (Claude Code)

[nicolasfalesy]

Linux test with a display (Arch, Intel Arc B390 on Mesa 26.2.2 ANV, Xwayland, GCC 16.2.1, Release), this PR merged onto main 7e181fc together with #4863, #4040 and #4930 (clean merges): guest_video_out_output_mode passes 3 of 3 with ANYPS5_REQUIRE_DISPLAY=1 on a real X display, so the open and the second open ran with a window on Linux too (the Linux run above was headless). The five video_out_flip_tests modes also pass on the display with the three PRs merged.

[p3n1cillin]

Title run with this PR: Windows 11, MinGW GCC 15.2.0 (WinLibs), Release. Built on main 3faa9cb40 merged with this PR and the other open PRs that the catalog titles need (#4377, #4379, #4374, #4383, #3798, #4863, #4105, my #5013/#5061/#5078/#5098/#5009 and the carries #5183/#5186, plus screamm's chroot branch from #3632); every title converted from its catalog release, eboot.bin kept in app0, 60 s runs.

• Porpoise (PPSA99764): on main (with its chroot import from libkernel: getpriority, chflags, chroot and clock_settime are absent (PS5MC, Porpoise); branch ready #3632's branch) it stops with VideoOut: port already open about 2 s in, on Windows and on Linux. With this PR it runs the whole 60 s with no exception.

AI-assisted (Claude Code).