fix(agc): keep at most 4 GiB of the host heap from the host import budget by Mouhand-Kaddo · Pull Request #7844 · boykopovar/AnyPS5

Mouhand-Kaddo · 2026-10-19T10:24:16Z

What

Demon's Souls (PPSA01342) stops on main while loading into the tutorial level, right after the intro cinematic, on a 32 GiB Linux machine (RX 6900 XT, RADV):

[gpu] host import budget: 7893 MiB live of 7955 MiB (APS5_HOST_IMPORT_MIB); 294335 imports (615986 MiB) refused so far, last 0x2a2b00000+0x38170000
FATAL: an address-based build copies 359 MiB of 1 registered ranges, past 64 MiB (APS5_ADDRESS_COPY_MAX_MIB); the largest: 0x260100000+0x16730000 (359.2 MiB committed, outside the write-watched arena)

#6472 keeps the larger of 2 GiB and half the largest host heap out of the import budget. That fixed the 16 GiB machine it was measured on (heap 8128 MiB, about 4 GiB kept back). The same half also applies to larger heaps, so a 32 GiB machine (RADV heap 15911 MiB) went from 13863 MiB to 7955 MiB. With APS5_TRACE_HOST_IMPORT, this title holds up to 10466 MiB of live imports at once while it loads the level, so the address-based build's imports are refused and the copy limit aborts.

The reserve is now half the heap kept between 2 and 4 GiB:

-reserve = max(2 GiB, hostHeap / 2)
+reserve = max(2 GiB, min(hostHeap / 2, 4 GiB))
 budget  = min(max(6 GiB, physical / 2), hostHeap - reserve)

host heap	`main`	this PR
3 GiB	1 GiB	1 GiB
8128 MiB (16 GiB machine, #6472's case)	4064 MiB	4064 MiB
15911 MiB (32 GiB, RADV here)	7955 MiB	11815 MiB
16 GiB	8 GiB	12 GiB
32 GiB (64 GiB machine)	16 GiB	28 GiB

Heaps up to 8 GiB keep main's budget. APS5_HOST_IMPORT_MIB still overrides it, and #6479 (lowering the budget after vkAllocateMemory failures) composes with this: it caps the effective budget, this sets the default it starts from. #7808 rewrites the APS5_HOST_IMPORT_MIB row of ENVIRONMENT.md; I left that row alone to avoid the conflict. The debt entry is updated.

Risk: two-way door, one constant. A machine with a host heap above 8 GiB pins more host memory for imports before the least-recently-used ones are retired. On the run below, total GTT use peaked at 12.6 of 15.9 GiB.

Tested

Linux 7.2.11 (EndeavourOS), i7-13700K, 32 GiB, RX 6900 XT with Mesa 26.2.5 RADV, GCC 16.2.1, Release, BUILD_TESTING=ON, main 36f60df; rebased onto main 98db779 on 2026-10-20 (only the debt entry conflicted: main had extended the same paragraph, so its text is kept with this PR's reserve sentence applied).

agc_driver_graphics (hostImportBudgetTests): with only the test change on main it fails with a 32 GiB machine does not keep 4 GiB of its 16 GiB host heap from imports; with the change it passes. The 8128 MiB and 3 GiB cases are unchanged and still pass. Rerun on 98db779: agc_driver_graphics_tests with main's GuestBufferMemory.cpp exits 1 with the same message, and with this PR it prints Graphics validation tests passed.
agc_host_import_unmap, agc_host_import_retirement, agc_host_import_retire, agc_host_import_sparse, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads pass. agc_driver_bda_device fails the same way on main and on this branch on this GPU (an anchored BDA read differs from the same read through the lookups at anchor ...); it does not use the budget.
python3 tools/check_conventions.py --base upstream/main passes.
Demon's Souls PPSA01342 01.004.000, relinked with --to-intel, continuing a saved character (main menu → Continue → Continue Offline), 1080p (+r_Wants4K = WantsPerf in a copy of the renderconfig_ps5_*.txt files), no environment overrides:
- main e12c913 (same HostImportBudget): the stop above while loading the level, exit 134 (one run).
- The same main with APS5_HOST_IMPORT_MIB=13862 (the old default): reaches the level in both runs, peak 10466 MiB live imports, no refusal.
- This PR, no override: no import refused, peak 10490 MiB live, the level loads into Outpost Passage with the HUD, and the left stick moves the character (level, after holding the stick right). The level runs at well under 1 fps; that is separate.
A main run with APS5_HOST_IMPORT_MIB=11815 (this PR's value) crashed in title code while loading: a guest heap pointer had been overwritten with RGBA16F texel data. No import was refused in it, so it is a separate bug, which I'm investigating.

Checklist

github-actions · 2026-10-19T10:35:12Z

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on `main`)	Same exports	Same new files
PR 7978	`docs/dev/debt/libSceAgcDriver.md` L205-206
PR 8104	`docs/dev/debt/libSceAgcDriver.md` L206

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1191	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 1395	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 2434	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 4040	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 4448	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 4449	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 5021	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 5058	`docs/dev/debt/libSceAgcDriver.md`
PR 5372	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 5441	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 5499	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 5925	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6186	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 6194	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6197	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 6223	`docs/dev/debt/libSceAgcDriver.md`
PR 6235	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 6280	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6300	`docs/dev/debt/libSceAgcDriver.md`
PR 6412	`docs/dev/debt/libSceAgcDriver.md`
PR 6481	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6548	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6808	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6867	`docs/dev/debt/libSceAgcDriver.md`
PR 6868	`docs/dev/debt/libSceAgcDriver.md`
PR 6918	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 6946	`docs/dev/debt/libSceAgcDriver.md`
PR 6947	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6977	`docs/dev/debt/libSceAgcDriver.md`
PR 7201	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 7256	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7410	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 7449	`docs/dev/debt/libSceAgcDriver.md`
PR 7505	`docs/dev/debt/libSceAgcDriver.md`
PR 7514	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp`
PR 7604	`docs/dev/debt/libSceAgcDriver.md`
PR 7642	`core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7800	`docs/dev/debt/libSceAgcDriver.md`
PR 7831	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 8014	`docs/dev/debt/libSceAgcDriver.md`
PR 8246	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 8564	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 8592	`docs/dev/debt/libSceAgcDriver.md`

LeandroLP23 · 2026-10-19T10:44:06Z

Windows and AMD's own driver. AMD Radeon 860M (RDNA 3.5, 1002:1114), driver 32.0.13058.8001, Windows 11, WinLibs GCC 15.2.0, Release, ANYPS5_NO_SHADER_CACHE=1. Head merged into main 118266a (clean merge). "Main's sources" means this PR's test files over main's other files.

Test	This head	`main`'s sources
`agc_driver_graphics`	pass, 3 of 3	fails: `a 32 GiB machine does not keep 4 GiB of its 16 GiB host heap from imports`
`agc_host_import_unmap`, `agc_host_import_retirement`, `agc_host_import_retire`	pass, 3 of 3	pass
`agc_host_import_sparse`	skips	skips

AI-assisted: yes (Claude Code)

fogydes

Reviewed the change:

Capping the host heap reserve at 4 GiB (std::min<std::uint64_t>(hostHeap / 2, 4096ull << 20u)) prevents 32 GiB hosts with 16 GiB host heaps from clamping the import budget below the 10,466 MiB working set required by memory-intensive titles (PPSA24191), while preserving the 2 GiB minimum floor for smaller 16 GiB configurations.
The unit test cases in Graphics.cpp explicitly guard both 16 GiB, 32 GiB, and 15,911 MiB heap scenarios against regression.
Technical debt in docs/dev/debt/libSceAgcDriver.md updated accurately.
Confirmed independent hardware test results posted by @LeandroLP23 (AMD Radeon 860M) verifying agc_driver_graphics passes 3/3 on this head.
Zero comments added in code; Conventional Commits title; all CI checks pass.

Ready to merge.

AI-assisted: yes

savvaniss · 2026-10-19T14:32:27Z

AMD and Intel Windows data: with this PR's test changes, agc_driver_graphics fails on main and passes with the PR, on an AMD RX 9070 XT and an Intel UHD 770.

Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, every execution test built, ANYPS5_NO_SHADER_CACHE=1, serial CTest. Intel UHD Graphics 770 (8086:a780), driver 32.0.101.7088. AMD Radeon RX 9070 XT (1002:7550), driver 32.0.31041.1004, with VkCache emptied first. This head 096de7f53 merged into main 85029a00f. "Without" means only this PR's diff to tests/Graphics.cpp, applied to main 85029a00f.

Tests, ×3	UHD 770	RX 9070 XT
This head: the 29 tests built from `tests/Graphics.cpp`	29 of 29 pass	29 of 29 pass
Without	`agc_driver_graphics` fails: `a 32 GiB machine does not keep 4 GiB of its 16 GiB host heap from imports`; the other 28 pass	same

AI-assisted: yes (Claude Code)

Celegans12 · 2026-10-19T20:12:42Z

I ran into the same thing with Ghost of Yōtei (PPSA26344) on a 32 GiB Windows laptop with an NVIDIA GPU, so here's a second data point from another title and vendor. With #6472's rule the budget there comes out at 8,117 MiB. In a warm run from a save into the open world, imports started being refused as soon as the world began loading, 171,828 of them by the end of the run, and the environment only appeared in the frame dump at 492 s. The same build with the old 2 GiB reserve put back (about 14 GiB here) refused no import in a 1,408 s run, and the same frame showed the environment at 412 s.

I haven't run Yōtei with this PR's 4 GiB cap yet; it would give about 12 GiB on this machine. Our older notes put Yōtei's import working set at around 13 GiB, so it may still refuse some here, though far fewer than at 8 GiB. If it's useful, I can run your branch and post the refusal count.

One caveat: these runs are from our local tree (a main from this morning plus our Yōtei changes), not plain main. The two runs differ only in the reserve.

Details

Machine: Windows 11, i9-11900H, 32 GiB, NVIDIA GeForce RTX 3080 Laptop GPU, driver 610.62; host heap about 16 GiB.
With fix(agc): keep half the host heap out of the host import budget #6472's reserve: the budget (8,117 MiB) is full and refusals begin between 264 and 279 s, when the world starts loading; 9,654 refused imports (1.2 GiB) by frame 84 and 171,828 (about 150 GB of requests) by the end. Frame 88, the first with the environment, was read back at 492.3 s.
With a 2 GiB reserve, same save, settings and pad script: 0 host import budget / refused for the host heap lines in 1,408 s; frame 88 (environment complete) at 411.6 s.

EdouardSence · 2026-10-19T20:47:00Z

Performance data from another title: on a 32 GiB Linux machine, #6472's budget makes ASTRO BOT about 2.5× slower, and this PR brings it back.

Setup: ASTRO BOT PPSA21567 01.019.000, -lvl hub_crashsite, no input. Linux 7.2.10 (CachyOS), Ryzen 7 5700X, 31 GiB RAM, RX 9070 XT, Mesa 26.2.4 RADV (host heap 16304 MiB). Headless gamescope at 3840×2160. APS5_NO_SNAPSHOT_CHECK=1, official trace APS5_TRACE_SECONDS=90:50, 180 s runs. Each build got a warm-up run first, and the main and override runs were interleaved.

Build	Import budget	presents (50 s)	frame p50 / p90 / p99 ms	GPU busy / present
`main` `12ee57a39`, run 1	7807 MiB	210	234.5 / 298.3 / 370.4	92.2 ms
`main` `12ee57a39`, run 2	7807 MiB	207	237.0 / 311.7 / 361.8	93.9 ms
`main` + `APS5_HOST_IMPORT_MIB=11815`, run 1	11815 MiB	524	94.5 / 102.7 / 107.1	56.3 ms
`main` + `APS5_HOST_IMPORT_MIB=11815`, run 2	11815 MiB	516	96.2 / 103.0 / 107.8	56.8 ms
`main` + this PR (`ac03b41d4`, clean merge), run 1	11518 MiB	521	95.2 / 103.0 / 108.8	56.6 ms
`main` + this PR, run 2	11518 MiB	509	97.3 / 106.9 / 110.0	61.1 ms

What happens on main: the scene keeps about 10.4 GiB of live imports, and the budget is 7807 MiB. On main the log ends with 7797 MiB live of 7807 MiB; 32 imports (51260 MiB) refused. The refused ranges then take device-local memory. VRAM use climbs to ~12.2 GiB of the 14.4 GiB budget, and the sampled texture cache drops to its 2048 MiB floor while the level holds ~2.5 GiB of sampled textures. So textures are evicted and recreated continuously, the same thrash @mrfelek measured in Sky Garden on #357. With this PR, VRAM use stays around 9.7 GiB, and the texture cache keeps 4.2 GiB under a 5.6 GiB budget.

How I found it: Crash Site went from p50 124 ms on aff725bf6 (2026-10-16) to 251 ms on 1e774a3b2. Rerunning the old build today still gives 126 ms, so the host is not the cause. A first-parent bisect narrowed it to the 25 merges between fd2ef7e28 (114 ms) and 8687da6c0 (249 ms), which include #6472. I stopped the bisect there because of the override result above: the budget alone accounts for the whole slowdown.

AI-assisted: yes (Claude Code)

…dget

EdouardSence · 2026-10-20T19:59:12Z

Follow-up on the first level, Sky Garden (-lvl underwater_aerial_garden), same machine and builds as my comment above (Linux, RX 9070 XT 16 GB / RADV Mesa 26.2.4, 31 GiB RAM, ASTRO BOT PPSA21567 01.019.000). Headless, no input, level start, official trace APS5_TRACE_SECONDS=90:50, warm caches.

Build	Frame p50 ms	p90 ms	GPU busy ms/present	Texture cache budget
`main` `12ee57a`	828.8	1110.7	255.2	2048 MiB (floor)
`main` + this PR, run 1	254.1	274.8	136.0	~3.8–4.3 GiB
`main` + this PR, run 2	254.0	272.7	136.0	~3.8–4.3 GiB
`main` + this PR + `APS5_TEXTURE_CACHE_MIB=4096` (2 runs)	253.8 / 254.1	273.8 / 275.4	137.9 / 144.4	—
`main` + this PR + `APS5_HOST_IMPORT_MIB=15000`	255.1	275.5	145.6	~4.4–5.0 GiB

On this level main is about 3.3× slower than with this PR (~1.2 FPS against ~3.9 FPS).
With the PR, neither a larger import budget nor a larger texture cache changes the frame time here, so on a 16 GB card the import budget is the only memory limit involved. The rest of the frame time is the existing draw cost.
One run for main, two for the PR.

Tested and written by Claude (AI assistant) on a contributor's machine.
AI-assisted: yes

xXPBSXx · 2026-10-21T04:47:34Z

Windows data point for 2af7b9bfb merged onto main 9f4ddf7cf (clean, git merge-tree rc 0): Windows 10.0.19045, MinGW-w64 GCC 15.2.0, Release. ctest -N -R '^agc_driver_graphics$' lists exactly that one test; ctest --no-tests=error -R '^agc_driver_graphics$' was run three times per tree.

With the PR: passes 3 of 3 (9.9-11.2 s).
Same tree with core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp restored to main (the new test cases kept): fails 3 of 3 (ordinary test failure, not a crash).
Not tested: other platforms, a title run.
AI-assisted (Claude Code).

github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 19, 2026

github-actions Bot added docs agc libSceAgc* labels Oct 19, 2026

Mouhand-Kaddo mentioned this pull request Oct 19, 2026

perf(agc): drop the guest byte snapshot of a large cached texture the write tracker covers #6982

Merged

9 tasks done

fogydes approved these changes Oct 19, 2026

View reviewed changes

github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 19, 2026

Mouhand-Kaddo force-pushed the fix/agc-host-import-reserve-cap branch from 096de7f to ac03b41 Compare October 19, 2026 16:31

github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 19, 2026

adriadam10 self-assigned this Oct 19, 2026

fisherjoey mentioned this pull request Oct 19, 2026

fix(agc): wait for the idle flush's unit-shadow publish before returning #7984

Open

10 tasks done

github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 19, 2026

JJenkx mentioned this pull request Oct 19, 2026

fix(agc): lower import budgets only on memory exhaustion #8091

Draft

12 tasks done

mrfelek mentioned this pull request Oct 19, 2026

docs(dev): name the memory budgets and the sleeping-display stop in the game testing guide #8135

Open

12 tasks done

fix(agc): keep at most 4 GiB of the host heap from the host import bu…

2af7b9b

…dget

Mouhand-Kaddo force-pushed the fix/agc-host-import-reserve-cap branch from ac03b41 to 2af7b9b Compare October 20, 2026 06:43

github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 20, 2026

Zenodeon mentioned this pull request Oct 20, 2026

fix(agc): mirror only unimported tails of registered heaps #8380

Open

12 tasks done

EdouardSence mentioned this pull request Oct 20, 2026

perf(agc): data-only draw hits compare only the user words a stage reads #5656

Merged

11 tasks done

EdouardSence mentioned this pull request Oct 21, 2026

perf(agc): skip the queued-draw read check when a storage flush stores nothing #8911

Open

12 tasks done

Navigation Menu

Repository navigation

fix(agc): keep at most 4 GiB of the host heap from the host import budget - #7844

fix(agc): keep at most 4 GiB of the host heap from the host import budget#7844
Mouhand-Kaddo wants to merge 1 commit into
boykopovar:mainfrom
Mouhand-Kaddo:fix/agc-host-import-reserve-cap

Mouhand-Kaddo commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

LeandroLP23 commented Oct 19, 2026

Uh oh!

fogydes left a comment

Uh oh!

savvaniss commented Oct 19, 2026

Uh oh!

Celegans12 commented Oct 19, 2026

Uh oh!

EdouardSence commented Oct 19, 2026 •

edited

Loading

Uh oh!

EdouardSence commented Oct 20, 2026

Uh oh!

xXPBSXx commented Oct 21, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

Mouhand-Kaddo commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Tested

Checklist

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeandroLP23 commented Oct 19, 2026

Uh oh!

fogydes left a comment

Choose a reason for hiding this comment

Uh oh!

savvaniss commented Oct 19, 2026

Uh oh!

Celegans12 commented Oct 19, 2026

Uh oh!

EdouardSence commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EdouardSence commented Oct 20, 2026

Uh oh!

xXPBSXx commented Oct 21, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Mouhand-Kaddo commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

EdouardSence commented Oct 19, 2026 •

edited

Loading