Skip to content

fix(agc): accept integer tessellation partitioning - #5234

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
AnthonyClemens:fix/agc-integer-tessellation-partitioning
Oct 14, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
AnthonyClemens:fix/agc-integer-tessellation-partitioning

Conversation

@AnthonyClemens

@AnthonyClemens AnthonyClemens commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Resident Evil 2, PPSA04288 (and presumably other RE engine titles) tessellates with integer partitioning, and right now any draw that does is thrown out. The state decoder stops with only triangle and quad domains with fractional-odd partitioning and triangle output are supported by the reference path, and the hull and domain programs stop with ShaderInputInfoBuilder: unsupported tessellation configuration.

This lets VGT_TF_PARAM.PARTITIONING = 0 through for triangle and quad patches and draws it with Vulkan's equal spacing.

Where the mapping comes from: PARTITIONING is bits 2-4 of VGT_TF_PARAM and 0 is PART_INTEGER in Mesa's gfx10.json, and radeonsi writes PART_INTEGER for TESS_SPACING_EQUAL in si_state_shaders.cpp. I have not measured anything on a console.

What changed:

  • The three places that checked for fractional-odd now also take integer.
  • The domain module gets SpacingEqual instead of SpacingFractionalOdd, and the pipeline's module check expects the same.
  • The hull's factor clamp tops out at 64 for integer. It stays at 63 for fractional-odd, which is what Vulkan allows each mode when the maximum level is 64.

Fractional-odd draws go through the same code as before. Partitioning was already in the shader cache key, the request serializer and the draw and pipeline keys, so none of that needed touching.

Power-of-two (1) and fractional-even (3) still throw. Vulkan has no power-of-two mode, and fractional-even needs a different clamp range, so I left it for its own PR. I added a line to docs/dev/debt/shader.md saying the console's rounding of a clamped factor is unmeasured.

Tested

Windows 11, MinGW-w64 GCC 15.2.0, Radeon RX 9070 XT.

  • agc_driver_tessellation and agc_driver_graphics both fail if I revert the source change and pass with it. The new checks cover: integer triangle patches decode, compile to SpacingEqual with a bound of 64, and draw; fractional-odd still compiles to SpacingFractionalOdd with a bound of 63; power-of-two is still refused.
  • Full ctest: 1082 of 1083 pass, 8 expected skips. The one failure is agc_driver_layered_volume_vertex_no_gpl (slice 1 texel (0, 0) channel 0 is 64, expected 255). It fails the same way on this card without my change.
  • PPSA04288, with fix(agc): start the sample location registers at their clear-state value #4949 applied locally so it gets as far as its first tessellated draw (VGT_TF_PARAM=0x40042, quad domain, integer): the partitioning refusal is gone. It then stops a bit later in the hull lowering with the hull shader reads no LDS per control point, so the local stride is unknown, which is a separate problem this PR doesn't touch. So I don't have a screenshot of an integer-tessellated patch from a real game, only the test's draw.

I only have this one title to test with.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

A draw with VGT_TF_PARAM.PARTITIONING 0 (PART_INTEGER) was refused by the
state decoder and its hull and domain programs by the recompiler. It is now
drawn with Vulkan's equal spacing, the mode radeonsi programs as
PART_INTEGER, with tessellation factors bounded at 64 instead of
fractional-odd's 63. Power-of-two and fractional-even partitioning still
throw.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#4201 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L768-769
core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp L420
docs/dev/debt/shader.md L90
#4222 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L768-769
core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp L5, L420
docs/dev/debt/shader.md L90
#5457 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L768-769
core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp L5, L420
docs/dev/debt/shader.md L90

@AnthonyClemens

Copy link
Copy Markdown
Contributor Author

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
#4201 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L602-603
core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp L420
docs/dev/debt/shader.md L90
#4222 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp L602-603
core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp L5, L420
docs/dev/debt/shader.md L90

Note: The overlap with #4201 and #4222 is only in tests and docs: we each add a test in Tessellation.cpp, a line in shader.md, and touch neighboring lines in Graphics.cpp. None of the source files overlap and this doesn't build on them. Not depended on.

@coltken95

Copy link
Copy Markdown

Tested on Linux x86-64 (Release build, lavapipe / llvmpipe LLVM 20.1.2), with d7818c9 merged into main 9d0a41d:

  • With the PR: all six tests in the PR's targets pass (agc_driver_graphics, agc_driver_tessellation, agc_sparse_buffer_holes, agc_descriptor_pool_reset_failure, agc_compute_descriptor_layout_flags, agc_driver_covered_centroid_state).
  • With main's sources and the PR's tests: agc_driver_graphics and agc_driver_tessellation fail; the other four pass.

AI-assisted: yes (Claude Code)

@EagleBearerYS

Copy link
Copy Markdown
Contributor

Resident Evil Village (PPSA01556, 01.210.000), another RE Engine title, needs this too. It draws quad patches with integer partitioning (VGT_TF_PARAM 0x40042: quads of one control point, integer, CW, max level 15), and on main they stop with:

VGT_SHADER_STAGES_EN=0x200210d: only triangle and quad domains with fractional-odd partitioning and triangle output are supported by the reference path

I had the same change locally, line for line in the production files, so I won't open mine. I replayed 21 tessellation requests dumped from Village with agc_shader_control_flow_tests, built on #4222's head (3a72fa3). Without integer partitioning, 5 compile and 15 stop at ShaderInputInfoBuilder: unsupported tessellation configuration. With it, 15 compile. The rest need other hull changes. That was on Windows 11 with WinLibs GCC 15.2.0.

@oneandonlydean
oneandonlydean merged commit f265cd5 into boykopovar:main Oct 14, 2026
10 checks passed
@AnthonyClemens
AnthonyClemens deleted the fix/agc-integer-tessellation-partitioning branch October 14, 2026 02:59
oneandonlydean added a commit that referenced this pull request Oct 14, 2026
Since #5263 the Linux job restores a lavapipe shader cache keyed by the
runner's /proc/cpuinfo flags. llvmpipe compiles shaders with LLVM's host
CPU *name* (from vendor, family and model) plus the detected feature bits,
and keys its cache only by the feature bits, so two runners with the same
flags but different models share cached code. A hull shader JIT-compiled
on a znver4/znver5 or Ice Lake class host uses GFNI (vgf2p8affineqb);
loaded on a Zen 3 host with the same flags it raises SIGILL, which is what
agc_driver_tessellation has been hitting on main and on dispatched builds
since the first main build after #5234 saved that hull variant.

Hash vendor_id, cpu family, model and flags into the key, so a runner only
restores code compiled for its own CPU model, and print the model name so a
future mismatch can be read off the log.
RINNECODER added a commit to RINNECODER/AnyPS5 that referenced this pull request Oct 14, 2026
* fix(agc): accept integer tessellation partitioning

Ported from upstream PR boykopovar#5234, adapted to our triangle/fractional-odd
only tessellation path: VGT_TF_PARAM.PARTITIONING 0 decodes, emits equal
spacing and bounds factors at 64. Metal still refuses guest tessellation
draws in MetalRenderState.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(agc): accept fractional-even tessellation partitioning

Ported from upstream PR boykopovar#5402 (its integer-partitioning base commit is
the previous commit), adapted to our triangle-only tessellation path:
VGT_TF_PARAM.PARTITIONING 3 decodes, emits fractional-even spacing and
bounds factors at 64. Metal still refuses guest tessellation draws.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): place hull input components by their offset in the control-point slot

Ported from upstream PR boykopovar#4723 (shader hunk; its Vulkan execution test is
not ported).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): lower hulls with one input or one output control point (partial)

Ported from upstream PR boykopovar#5457, commits 1, 4 and 5 of 5 (hull exec masks
restored through phis, input stride from the patch term, one output
control point per patch). Vulkan execution-test hunks are not ported.

Skipped from boykopovar#5457, with reasons:
- relative patch index rebuilt as data: needs GE_CNTL.PRIM_GRP_SIZE and
  BREAK_WAVE_AT_EOI in TessellationConfiguration, VGT_LS_HS_CONFIG
  patchesPerGroup and the request key/serializer additions. None of
  these exist on main, which only lowers triangle fractional-odd hulls.
- recording BDA faults in hull programs: drops a refusal whose safety
  rests on upstream hull work we lack; the refusal stays.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): honour SPI_BARYC_CNTL.FRONT_FACE_ALL_BITS for the front-face input

Ported from upstream PR boykopovar#5314. Tracks SPI_BARYC_CNTL in the draw key and
shader request key, and bumps the request serialization to version 24
(shared with the following slice-offset port, which adds no second bump).

Not ported: the ShaderMemory serializer-version assertions of the PR,
which build on upstream test scaffolding absent from main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): propagate color view slice to pixel ancillary input

Ported from upstream PR boykopovar#5500, adapted to main: the draw decode sets the
color view slice as layerBase (the layer-export decode this PR also
touches does not exist on main, so its diagnostic and layers hunks are
skipped). layerBase is added to the pixel info, request key, shader info,
disk cache and pipeline specialization constants. The request
serialization stays at version 24, bumped once by the front-face commit.
The shader disk cache format is bumped to 36.

Not ported: unrelated hunks the upstream commit carried in Recompiler.cpp
(dot-product capability option) and its ShaderMemory serializer tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* test(shader): cover tessellation partitioning, hull slot placement, layerBase and serializer v24

Fix the stale serialization version expectation in the shader memory test,
add pixel ancillary rows with a non-zero layerBase, extract PixelLayerBase
for a derivation test, and point the FrontFace note at TechnicalDebt.md.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: XairRrR <dev910382@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants