Skip to content

feat(libSceFios2): implement file, directory and operation exports - #12269

Merged
oneandonlydean merged 8 commits into
boykopovar:mainfrom
MarcZX10:feat/libscefios2-file-io
Nov 2, 2026
Merged

oneandonlydean merged 8 commits into
boykopovar:mainfrom
MarcZX10:feat/libscefios2-file-io

Conversation

@MarcZX10

@MarcZX10 MarcZX10 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What

Implements 106 of the 168 libSceFios2 exports that only threw NotImplemented_nid_no_patch through parameterless declarations, on top of the kernel file exports (sceKernelOpen, sceKernelPread, sceKernelGetdirentries, ...), so guest path mapping and archive handling stay in libkernel. The library gets a CMakeLists.txt, its own types header and a shared runtime.

  • Runtime and control: sceFiosInitialize, sceFiosTerminate, sceFiosIsInitialized, sceFiosUpdateParameters, default operation attributes (global and per thread), suspend and resume with queued operations, sceFiosCancelAllOps, sceFiosShutdownAndCancelOps, sceFiosCloseAllFiles, sceFiosIsIdle, sceFiosGetAllFHs/DHs/Ops, sceFiosIsValidHandle and the sceFiosOp* lifecycle (wait, sync wait, error, counts, attr, path, buffer, offset, cancel, delete, callbacks).
  • Path queries: sceFiosExists, sceFiosFileExists, sceFiosDirectoryExists, sceFiosFileGetSize, sceFiosStat (plus Sync forms).
  • Path mutations and whole-file transfers: sceFiosDelete, sceFiosFileDelete, sceFiosDirectoryCreate, sceFiosDirectoryDelete, sceFiosRename, sceFiosFileRead, sceFiosFileWrite, sceFiosFileTruncate (plus Sync forms).
  • File handles: sceFiosFHOpen, sceFiosFHOpenWithMode, sceFiosFHClose, sceFiosFHStat, sceFiosFHTruncate, sceFiosFHSync, the read/write/pread/pwrite families including the vectored forms (plus Sync forms), sceFiosFHSeek, sceFiosFHTell, sceFiosFHGetSize, sceFiosFHGetPath, sceFiosFHGetOpenParams.
  • Directory handles: sceFiosDHOpen, sceFiosDHRead, sceFiosDHClose (plus Sync forms), sceFiosDHGetPath.
  • Dates: sceFiosDateGetCurrent, sceFiosDateFromComponents, sceFiosDateToComponents, sceFiosTimeIntervalFromNanoseconds, sceFiosTimeIntervalToNanoseconds.

Sources (references for prototypes and behaviour only, code written independently, all cited with commits in docs/dev/debt/libSceFios2.md): RPCS3's Vita FIOS2 module (jacquesjk/RPCS-DX12 mirror of Emu/ARMv7/Modules/sceFios.*), kalaposfos13/fios2, sonny-tel/libSceFios2, Inori/GPCS4, SonicMastr/SceLibcPosix and vascon-es (a PC port of a PS4 title, CUSA01176 and CUSA00935 version 01.24; call-site evidence for the stat, directory entry and directory-open sizing behaviour).

Not implemented, still throwing, because no credible public prototype or behaviour was found: archive mount/unmount and buffer-size exports, overlays, sceFiosResolve*, IO filters, the cache prefetch/flush/contains family, sceFiosDirectoryCreateWithMode, sceFiosFHToFileno, sceFiosFilenoToFH, sceFiosPathcmp/ncmp/Normalize, sceFiosOpWaitUntil, sceFiosOpReschedule*, sceFiosOpSetBuffer, the debug dump, printf and timestamp exports, sceFiosTimeGetCurrent, sceFiosDLL*, sceFiosDeallocatePassthruFH. No open PR touches libSceFios2 (searched libSceFios2; the only hit is an unrelated draft).

Known gaps are in the debt file: operations run on the calling thread (no IO threads, deadlines or priorities), error mapping from kernel errors is chosen here, and the dir-open required size is not the console's value.

Tested

Linux (host build, Release + tests). guest_fios2 (new, core/libs/tests/GuestFios2.cpp) exercises the error paths, callbacks, suspension and cancellation, whole-file and handle I/O including vectored and positioned transfers, directory listing with the sizing probe, and date conversion against a known epoch value. check_conventions.py and the public header audit pass. Not tested on a console or with a title; the prototypes and behaviour are reasoned from the public sources above.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

Add the shared FIOS2 runtime: one handle number space for operations,
files and directories, operation records with completion callbacks,
suspend and resume with queued operations, cancellation, default
operation attributes and the stored initialization parameters.

Implement the control exports (sceFiosInitialize, sceFiosTerminate,
suspend and resume, the sceFiosGetAll* enumerations, sceFiosOp*) and
the path queries sceFiosExists, sceFiosFileExists, sceFiosDirectoryExists,
sceFiosFileGetSize and sceFiosStat with their Sync forms on top of
sceKernelStat. Failures return SCE_FIOS_ERROR_* codes.

The prototypes and layouts come from public emulator sources and are
listed in docs/dev/debt/libSceFios2.md.
Implement sceFiosDelete, sceFiosFileDelete, sceFiosDirectoryCreate,
sceFiosDirectoryDelete, sceFiosRename, sceFiosFileRead,
sceFiosFileWrite and sceFiosFileTruncate with their Sync forms on top
of the kernel file exports, so guest path mapping and archive handling
stay in one place. Reads and writes loop until the request is
satisfied or the file ends, argument and kernel failures map to
SCE_FIOS_ERROR_* codes, and the test covers the error paths and the
operation counters.
Implement sceFiosFHOpen, sceFiosFHOpenWithMode, sceFiosFHClose,
sceFiosFHStat, sceFiosFHTruncate, sceFiosFHSync, the read, write,
pread and pwrite families including the vectored forms with their Sync
variants, and sceFiosFHSeek, sceFiosFHTell, sceFiosFHGetSize,
sceFiosFHGetPath and sceFiosFHGetOpenParams.

A handle owns a kernel descriptor and serializes its transfers.
Transfers the open flags exclude, bad vector counts, negative sizes and
offsets and closed handles return SCE_FIOS_ERROR_* codes.
Implement sceFiosDHOpen, sceFiosDHRead, sceFiosDHClose, their Sync
forms and sceFiosDHGetPath. Opening lists the directory through
sceKernelGetdirentries and reports SCE_FIOS_ERROR_BAD_SIZE with the
required size through sceFiosOpGetActualCount when the supplied buffer
is null or too small, the sizing probe title code performs. Reading
fills SceFiosDirEntry and ends with SCE_FIOS_ERROR_EOF.
Implement sceFiosDateGetCurrent, sceFiosDateFromComponents,
sceFiosDateToComponents, sceFiosTimeIntervalFromNanoseconds and
sceFiosTimeIntervalToNanoseconds. Dates are nanoseconds since the Unix
epoch in UTC, converted with a proleptic Gregorian calendar that does
not depend on the host C library, and out-of-range components are
normalized.
Keep each entry under the length the conventions check expects.
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Progress report (b99bbeb → dddc4fd)

📈 System libraries: 47.46% (+0.75%, +106 implemented functions)

✅ 106 implemented
Library Name
libSceFios2 sceFiosCancelAllOps
libSceFios2 sceFiosCloseAllFiles
libSceFios2 sceFiosDHClose
libSceFios2 sceFiosDHCloseSync
libSceFios2 sceFiosDHGetPath
libSceFios2 sceFiosDHOpen
libSceFios2 sceFiosDHOpenSync
libSceFios2 sceFiosDHRead
libSceFios2 sceFiosDHReadSync
libSceFios2 sceFiosDateFromComponents
libSceFios2 sceFiosDateGetCurrent
libSceFios2 sceFiosDateToComponents
libSceFios2 sceFiosDelete
libSceFios2 sceFiosDeleteSync
libSceFios2 sceFiosDirectoryCreate
libSceFios2 sceFiosDirectoryCreateSync
libSceFios2 sceFiosDirectoryDelete
libSceFios2 sceFiosDirectoryDeleteSync
libSceFios2 sceFiosDirectoryExists
libSceFios2 sceFiosDirectoryExistsSync
libSceFios2 sceFiosExists
libSceFios2 sceFiosExistsSync
libSceFios2 sceFiosFHClose
libSceFios2 sceFiosFHCloseSync
libSceFios2 sceFiosFHGetOpenParams
libSceFios2 sceFiosFHGetPath
libSceFios2 sceFiosFHGetSize
libSceFios2 sceFiosFHOpen
libSceFios2 sceFiosFHOpenSync
libSceFios2 sceFiosFHOpenWithMode
libSceFios2 sceFiosFHOpenWithModeSync
libSceFios2 sceFiosFHPread
libSceFios2 sceFiosFHPreadSync
libSceFios2 sceFiosFHPreadv
libSceFios2 sceFiosFHPreadvSync
libSceFios2 sceFiosFHPwrite
libSceFios2 sceFiosFHPwriteSync
libSceFios2 sceFiosFHPwritev
libSceFios2 sceFiosFHPwritevSync
libSceFios2 sceFiosFHRead
libSceFios2 sceFiosFHReadSync
libSceFios2 sceFiosFHReadv
libSceFios2 sceFiosFHReadvSync
libSceFios2 sceFiosFHSeek
libSceFios2 sceFiosFHStat
libSceFios2 sceFiosFHStatSync
libSceFios2 sceFiosFHSync
libSceFios2 sceFiosFHSyncSync
libSceFios2 sceFiosFHTell
libSceFios2 sceFiosFHTruncate
libSceFios2 sceFiosFHTruncateSync
libSceFios2 sceFiosFHWrite
libSceFios2 sceFiosFHWriteSync
libSceFios2 sceFiosFHWritev
libSceFios2 sceFiosFHWritevSync
libSceFios2 sceFiosFileDelete
libSceFios2 sceFiosFileDeleteSync
libSceFios2 sceFiosFileExists
libSceFios2 sceFiosFileExistsSync
libSceFios2 sceFiosFileGetSize
libSceFios2 sceFiosFileGetSizeSync
libSceFios2 sceFiosFileRead
libSceFios2 sceFiosFileReadSync
libSceFios2 sceFiosFileTruncate
libSceFios2 sceFiosFileTruncateSync
libSceFios2 sceFiosFileWrite
libSceFios2 sceFiosFileWriteSync
libSceFios2 sceFiosGetAllDHs
libSceFios2 sceFiosGetAllFHs
libSceFios2 sceFiosGetAllOps
libSceFios2 sceFiosGetDefaultOpAttr
libSceFios2 sceFiosGetGlobalDefaultOpAttr
libSceFios2 sceFiosGetSuspendCount
libSceFios2 sceFiosGetThreadDefaultOpAttr
libSceFios2 sceFiosInitialize
libSceFios2 sceFiosIsIdle
libSceFios2 sceFiosIsInitialized
libSceFios2 sceFiosIsSuspended
libSceFios2 sceFiosIsValidHandle
libSceFios2 sceFiosOpCancel
libSceFios2 sceFiosOpDelete
libSceFios2 sceFiosOpGetActualCount
libSceFios2 sceFiosOpGetAttr
libSceFios2 sceFiosOpGetBuffer
libSceFios2 sceFiosOpGetError
libSceFios2 sceFiosOpGetOffset
libSceFios2 sceFiosOpGetPath
libSceFios2 sceFiosOpGetRequestCount
libSceFios2 sceFiosOpIsCancelled
libSceFios2 sceFiosOpIsDone
libSceFios2 sceFiosOpSyncWait
libSceFios2 sceFiosOpSyncWaitForIO
libSceFios2 sceFiosOpWait
libSceFios2 sceFiosRename
libSceFios2 sceFiosRenameSync
libSceFios2 sceFiosResume
libSceFios2 sceFiosSetGlobalDefaultOpAttr
libSceFios2 sceFiosSetThreadDefaultOpAttr
libSceFios2 sceFiosShutdownAndCancelOps
libSceFios2 sceFiosStat
... 6 more
⚪ 2 behaviourally empty (constant return or throw)
Library Name
libSceFios2 sceFiosTimeIntervalFromNanoseconds
libSceFios2 sceFiosTimeIntervalToNanoseconds

Changed libraries (implemented exports)

Library Before After Change
libSceFios2 0/168 (0.00%) 106/168 (63.10%) +63.10 pp

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@MarcZX10 the structure reads well: every path goes through the kernel exports, so app0/archive mapping stays in libkernel, callbacks fire outside the runtime lock (deleting an op from its own completion callback works), and the 62 exports without a source still throw.

One design point on operation completion: every operation runs and fires SCE_FIOS_OPEVENT_COMPLETE inside the call that issues it. The debt entry covers the callback seeing an op id the caller hasn't received yet, but there's a second consequence: a title that issues an operation while holding a lock that its own completion callback takes now deadlocks on its own thread, where on the console the callback runs on the FIOS I/O thread. Would you consider running the work and callbacks on one worker thread (sceFiosOpWait and sceFiosOpSyncWait already wait on a condition variable)? If not, could the debt entry mention the deadlock case too?

@oneandonlydean

Copy link
Copy Markdown
Collaborator

@MarcZX10 two small things on the debt file, and one optional:

  1. docs/dev/debt/libSceFios2.md cites the call-site source by its repository name (lines 7, 18 and 20), and that name is a game title. The debt rules allow title IDs but not game names. Please cite it by owner, commit and the title ID instead (for example "vascon-es, PC port of a PS4 title (CUSAxxxxx), at ").
  2. check_conventions.py still warns on two debt entries over 1000 characters (lines 15 and 25, at 1060 and 1997 characters). Line 25 is the long list of unknown functions, so it may be fine as it is. Line 15 just needs a split.
  3. Optional: outputs go through GuestCopyOut, but guest inputs are read directly: SceFiosParams in StoreParameters, *attr, *openParams, the iov array in StartVectors and SceFiosTm in sceFiosDateFromComponents. With GuestCopyIn_nid_no_patch, a bad pointer would return SCE_FIOS_ERROR_BAD_PTR instead of faulting in the host.

@JJenkx

JJenkx commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

A Windows run, since the description lists none. Head 1126d89811 was merged onto main 0cd07dc4ab with no conflicts. Setup: Windows 11 Enterprise 10.0.22621, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON, on NTFS.

libs builds, and guest_fios2, with this PR's new GuestFios2.cpp checks for the file, directory and path exports, passes in each of 3 runs. I didn't run it under a title.

AI-assisted: yes (Claude Code).

@fogydes

fogydes commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

verdict: should-fix

  1. [verified] FileHandles.cpp directly dereferences any non-null openParams pointer (params = *openParams) before the operation is submitted. An invalid guest pointer can fault the host instead of returning SCE_FIOS_ERROR_BAD_PTR. Similar unchecked reads occur for the op attributes, FIOS params, date components, and vector descriptors. Use the guest-copy helper and map copy failures to the FIOS bad-pointer error.
  2. [verified] The conventions checker exits 0 but reports two debt-length warnings in the new debt file: 1,060 and 1,997 characters. The latest commit says it splits long entries, but both remaining entries still exceed the checker limit.
  3. [read-only] The PR title differs from its latest commit subject: docs(libSceFios2): split the long technical debt entries.
  4. [author-reported] The Linux Release build, guest_fios2 test, and header audit are reported as passing; I did not rerun them.

Freshness envelope: head 1126d89; merge-base 9fc181d; behind current main: 73; git merge-tree --write-tree origin/main pull/12269: exit 0, clean.

Cross-PR ledger: pr_overlap.py checked 1,372 other open PRs and found no shared files or exports.

Copy SceFiosOpAttr, SceFiosOpenParams, SceFiosParams, the vector arrays
and struct tm out of guest memory with GuestCopyIn instead of
dereferencing them, and return SCE_FIOS_ERROR_BAD_PTR when a copy
fails. The *Sync forms return the error of an operation that was never
created, and the test covers unreadable attributes, open parameters and
vector arrays.
Cite the PC port that supplied PS4 call-site evidence by owner, commit
and title IDs instead of its repository name, split the file handle
entry under the length the conventions check expects, and record the
operation threading and guest input handling in the technical debt.
@MarcZX10

Copy link
Copy Markdown
Contributor Author

Pushed two follow-up commits.

  1. The call-site source is now cited by owner, commit and title IDs (vascon-es, PC port of a PS4 title, CUSA01176 and CUSA00935 version 01.24, commit 57c3c202975ea48e574df2d1e4a620b4e18b7701) in the debt file and the description; no game name is left.
  2. The file handle debt entry is split under 1000 characters. check_conventions.py now warns only on the pre-existing unknown-functions list.
  3. SceFiosOpAttr, SceFiosOpenParams, SceFiosParams, the vector arrays and struct tm are read with GuestCopyIn_nid_no_patch; a failed copy returns SCE_FIOS_ERROR_BAD_PTR. guest_fios2 now covers unreadable attr, open parameters and iov arrays. It passes on Linux.
  4. A worker thread was not added. Guest callbacks need a guest thread (scePthreadCreate with TLS and stack, as libSceJobManager uses), the Sync forms must stay synchronous on the caller, and moving completion off the issuing call changes the ordering of output-pointer writes, callbacks and teardown that the library and its tests rely on. That could not be validated against a title or a console here, so it is recorded as an open gap ("Operation threading") in docs/dev/debt/libSceFios2.md.

@maskym

maskym commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Windows result for head dddc4fd030, merged into main 87b810a1db (the PR's Tested section names no Windows run).

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

The tree also carried the open PRs #12290, #12288, #12285, #12265, #12232, #12206, #12201, #12199, #12197, #12165, #12146, all merged cleanly, so the run is a joint one; the tests of this PR are guest_fios2 (0.13 s), all passed (every test registered from the tree's changed test files: 38/38). Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 14869 docs/dev/debt/libSceFios2.md L5, L7

@oneandonlydean
oneandonlydean merged commit 1b3c540 into boykopovar:main Nov 2, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants