Skip to content

feat(tools): keep technical debt in one file per library - #4411

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
adriadam10:docs/debt-per-library
Oct 12, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
adriadam10:docs/debt-per-library

Conversation

@adriadam10

@adriadam10 adriadam10 commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

What

docs/dev/TechnicalDebt.md had 684 entries in four sections, and most pull requests append to the same section ends, so it was the most common conflict: 176 open pull requests touch it. This keeps one file per library or component under docs/dev/debt/, with TechnicalDebt.md as the index.

  • docs/dev/debt/<name>.md: an entry goes to the file named after the first core/ path it links: <name> for core/libs/prx/<name>/..., otherwise core/<name>/... (shader, relinker, libs), so a recompiler entry that also cites a driver test stays in shader.md, under the same ### sections and order as before. Entries that link neither (Build, the 4 cross-library "Unknown function info" lines, 3 Functional lines) and the Silent stubs preamble stay in the index; #silent-stubs and #functional anchors still exist there. Links are rewritten to ../../../.
  • The second commit is generated only by python3 tools/debt_split.py split, nothing edited by hand; on a rebase it is regenerated rather than rebased.
  • tools/debt_split.py:
    • split moves every library or component entry in the index to its file, appending to existing files (idempotent: also fixes an entry a clean merge left in the index).
    • resolve, run during a rebase stopped on TechnicalDebt.md, takes the stopped commit's change between the merge base and its version (stages 1 and 3), applies its added, edited and removed entries to the debt files (an addition goes after the entry it followed), keeps main's index and stages everything. It refuses, leaving the conflict as it was, when the change also edits lines that stay in the index or when an edited/removed entry was also changed on main.
  • Conventions (check_conventions.py):
    • Silent stubs are looked up in every debt file, each link resolved from its own file.
    • Comments are accepted when the pull request changes the index or any debt file.
    • New debt-file rule: an added entry whose name is not the file it is in (including the index) fails, naming the right file. The naming rule lives once, in check_conventions.debt_name, which debt_split.py imports.
  • stubgen.py default debt link is relative to docs/dev/debt/.
  • CONTRIBUTING (Documentation and Code) and the PR template point at the per-library files and explain resolve.

Tested

  • No entry lost: the multiset of (section, entry with links resolved) is identical before and after, 684 = 684, 0 missing, 0 extra. Every new file passes doc-link.
  • tools/tests/test_debt_split.py (registered as debt_split): split by first core link, split appending to existing files without duplicating the index note, and resolve in real rebases: added + edited + removed entries (order kept, emptied file deleted, index equal to main's), refusal on index lines, refusal on an entry main also changed, refusal outside a conflict. Mutations (drop the delete, drop the index-line guard, always insert the note) each fail a test.
  • test_check_conventions_comments.py: silent-stub cases moved to a library file; new cases for debt-file (library entry in the index, in another library's file, component file, entry without a core link stays, entry filed by its first core link). Mutations (rule disabled; links resolved from the index instead of their own file) fail 2 and 4 tests.
  • All tools/tests suites pass; check_conventions.py --base upstream/main passes.
  • Rebased the commits of all 176 open pull requests that touch TechnicalDebt.md onto this branch, running resolve at every stop:
Today on main With this branch PRs
merges cleanly resolve handles every stop 74
merges cleanly clean (1), or leaves an entry in the index that debt-file reports and split moves (2) 3
merges cleanly refused: edits a line that stays in the index 1
merges cleanly conflicts in code files: the same commit-by-commit conflicts happen rebasing onto main (checked for all 6) 6
conflicts resolve handles the TechnicalDebt.md part 32
conflicts refused (index lines) 3
conflicts conflicts in other files 58

So no pull request gets a new conflict it cannot resolve with resolve except the 4 that edit index lines (the same hand merge as today).

@boykopovar this changes where technical debt lives (layout call). @oneandonlydean after merge, authors rebase and run python3 tools/debt_split.py resolve when TechnicalDebt.md conflicts; I can run it on the open pull requests we have edit access to.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

This was referenced Oct 12, 2026
@CnRJay

CnRJay commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

+1

@Kikoarl

Kikoarl commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Strong +1.

Keeping TechnicalDebt.md in sync across orthogonal PRs (libSceAgcDriver, shader, libSceSysmodule) was consistently the primary source of rebase churn. Modularizing into docs/dev/debt/<component>.md and providing debt_split.py resolve cleanly eliminates cross-subsystem collisions while making rebases effortless for open branches.

@fma-rc

fma-rc commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

LGTM.

@blawesom

Copy link
Copy Markdown
Contributor

Rebase handoff — rebased onto current main, authorship unchanged. This one needed a real semantic docs resolution, so here's what it involves:

This PR restructures the technical-debt ledger into one file per library (docs/dev/debt/<name>.md), while main kept adding new entries to the old single file — 10 entries landed since this PR's base. A plain union would have re-broken the PR's own convention (and the debt-file check enforces it). The rebased branch instead routes each of main's 10 new entries to its per-library file by the first core/ path it links (libSceHttp2, libSceRtc, libkernel, libSceNgs2.native, shader, libkernel_sys, libc ×2, libSceAgcDriver, shader), with the relative links adjusted for the deeper directory.

One consequence worth knowing for future PRs: any PR that adds a technical-debt entry while this one is open should target docs/dev/debt/<library>.md directly, or it will conflict here again.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

  • builds; tools/check_conventions.py --base origin/main passes
  • full suite: 958/958 passed

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/4411
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is 63851a36c.

@Mouad-Scriptz-0

Copy link
Copy Markdown

Thanks, the first-core/ link rule reads right, and the five recompiler entries belong in shader.md.

On the two conflicts: #4448 is rebased on current main (f82f49d, mergeable again; the BdaResources.cpp overlap was the failed-import test from #1357 next to mine). #4048's conflict is the agc_shader_replay usage line against #4325's --prepare; I'll rebase it today.

adriadam10 and others added 2 commits October 12, 2026 20:38
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@adriadam10
adriadam10 force-pushed the docs/debt-per-library branch from 8b30984 to 342a8c4 Compare October 12, 2026 18:38
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 12, 2026
@adriadam10
adriadam10 merged commit 1acfa7f into boykopovar:main Oct 12, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants