Skip to content

feat(shader): emulate barycentrics through tessellation on devices without geometry shaders - #7308

Merged
oneandonlydean merged 7 commits into
boykopovar:mainfrom
DavidSamir:feat/shader-barycentric-tessellation
Oct 17, 2026
Merged

oneandonlydean merged 7 commits into
boykopovar:mainfrom
DavidSamir:feat/shader-barycentric-tessellation

Conversation

@DavidSamir

Copy link
Copy Markdown
Contributor

What

On a device without geometry shaders, a pixel shader that needs exact barycentrics or raw vertex values (and so uses the barycentric emulation) stops the draw with Barycentric geometry SPIR-V: geometry shaders are unavailable. MoltenVK on Apple GPUs has no geometry shaders, so on macOS this ends many catalog titles, most of them a few seconds after start-up. MoltenVK does have tessellation, which can do the same job:

  • New tessellation stages (BuildBarycentricTessellationShaders, next to the geometry stage in BarycentricGeometryShader.cpp). Each triangle of a triangle list becomes one three-point patch at tessellation level 1, so it comes out as the same single triangle. At each corner the evaluation stage reads gl_TessCoord to pick its vertex, then writes that corner as the weights and passes every vertex's parameters for raw reads. This is the same data the geometry stage provides. A flat parameter takes the first vertex's value, as the geometry stage's strip gives it.
  • Clip distances are forwarded. The vertex shader's gl_ClipDistance, which every position export has written since fix(shader): clip primitives with an all-zero clip position #3878, goes through both stages. Without it, MoltenVK laid out the control stage's input differently from the vertex output: it read every position from the wrong offset, and nothing was drawn. fix(shader): forward gl_ClipDistance through the generated rect-list stages #7124 found the same for rect lists; this PR uses its VertexClipDistanceCount and builds on it.
  • When this route is used. The geometry stage stays in use wherever the device has geometryShader. The tessellation stages are used only when the device has tessellationShader but no geometryShader, and only for a triangle list without primitive restart. Strips, fans and restart still throw, with a message saying so.
  • Pipeline. The draw uses a three-point patch list, stays out of graphics pipeline libraries (their vertex-input key holds the guest topology), and still throws for the last provoking vertex.
  • Validation. ShaderValidation accepts the generated stages and checks their execution modes and three-element interface arrays. A four-stage vertex pipeline whose pixel shader does not emulate barycentrics still fails with "incorrect graphics stage count".
  • Winding. The patch uses VertexOrderCw, which keeps the triangle's facing; the draw test checks this with face culling.
  • Other changes. First, a refactor so the geometry and tessellation builders gather the emulated interface the same way. Then the two shader.md debt entries are updated.

Commits: refactor, recompiler stages, driver wiring, tests, docs, then a merge of current main. In the merge, #7251's new last-vertex rect-list control stage also gets #7124's clip-distance argument, without which RectListShaders.cpp doesn't compile.

Related: #7201 handles MoltenVK per-vertex inputs differently, by rebuilding them from barycentric coordinates and derivatives; #6941 covers the mesh path. Neither generates tessellation stages for the vertex path.

Tested

Apple M2, macOS 26.7.1, MoltenVK 1.4.2 (geometryShader false, tessellationShader true), x86-64 under Rosetta, on the community macOS stack (mugurc's port branches over main from 15 Oct) with #7124 applied, Release build. The changed files of this branch, as merged with current main, also compile on this Mac with SPIR-V Tools on. main's own libc doesn't build on macOS, so the tests ran on the port stack.

  • agc_driver_pixel_barycentric_emulation (extended). It now runs the geometry route where the device has geometry shaders and the tessellation route where it has tessellation; before, it skipped any device without geometryShader.
    • On the M2, main's test reports skipped, the device lacks geometryShader.
    • With this PR it passes the tessellation route: 932 perspective and 932 linear I/J pixels match the reference, the raw vertex values (v_interp_mov P0/P10/P20) are exact, host interpolation is unchanged, and back- and front-face culling give the same result as the same draw without emulation.
  • How the M2 failures were narrowed down:
    • The first runs drew nothing. The Vulkan validation layer reported no error.
    • MVK_CONFIG_DEBUG showed MoltenVK's control-stage input struct as uint4, uint, float4 gl_Position against a vertex output of out_param_0, gl_Position, gl_ClipDistance[1], i.e. a mismatched layout. Forwarding the clip distances fixed it.
    • VertexOrderCcw then failed the culling check and VertexOrderCw passed.
  • agc_driver_graphics_tests (new unit tests) passes. It covers:
    • both stages' execution modes;
    • recognising the generated stages, and the stage count without emulation;
    • triangle strips, fans and point lists rejected;
    • missing tessellation, missing capability, device limits (tessellation and fragment), a missing vertex export;
    • clip distances forwarded when the vertex shader writes two and absent when it writes none;
    • clip distances refused without shaderClipDistance.
  • Unchanged on the M2 with this PR:
    • pass: agc_fragment_barycentric, agc_driver_interpolation_modes, agc_driver_draw_offset_indices, agc_rect_list_position_invariant, agc_driver_indirect_patched_stage;
    • skip: agc_driver_interpolation_f16 (no fragmentShaderBarycentric).
  • Same failure with and without the change (rebuilt without it to check): agc_driver_tessellation (hull relative patch index) and agc_driver_pixel_ancillary_word (its pixel shader needs the Geometry capability).

Titles (homebrew catalog, checksum-checked, 60 s each, relinked for macOS):

All of these stopped with Barycentric geometry SPIR-V: geometry shaders are unavailable before. For three of them I rebuilt the same tree without this change and re-ran them:

Title Without this PR (same tree) With this PR
EVO Player (PPSA99039) aborts at 32 s, geometry stage runs 60 s, 47 flips
Nuvio PS5 (PPSA99997) aborts at 11 s, geometry stage runs 60 s, 47 flips
Cloud Play (PPSA99600) aborts at 5 s, geometry stage gets past it, stops at the next missing feature (below)

From earlier runs of the same stack:

  • Jelly5 (PPSA99505), Twispero (PPSA99474) and Emby5 (PPSA99515) now run the full 60 s.
  • Stremio Plus (PPSA74126), CD Player (PPSA98427), ProsperoPuzzles (PPSA99006), OpenNOW PS5 (PPSA99082), ProsperoStore (PPSA99000) and Régie5 (PPSA99520) get past this stop. They then stop at last provoking vertex requires VK_EXT_provoking_vertex with provokingVertexLast enabled, which the M2 also lacks; the generated stages could handle that too, as fix(agc): draw rect lists in last provoking vertex mode #7251 does for rect lists, so it's a follow-up.

Device limits of the M2 that stop titles here, not tested past them: no geometryShader, shaderFloat64, VK_EXT_provoking_vertex or VK_EXT_depth_range_unrestricted.

Not tested yet: Windows and Linux. There the geometry route is unchanged and the tessellation route runs only from the extended test, which lavapipe and other drivers with tessellation will run in CI. vkQuake (PPSA99010) is not tested here; on this Mac it stops earlier, creating its Vulkan surface.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: fix(shader): forward gl_ClipDistance through the generated rect-list stages #7124
  • Fixes: none
  • Unblocks: PPSA99039, PPSA99997, PPSA99505, PPSA99474, PPSA99515 on macOS (macOS support #1); past this stop also PPSA74126, PPSA98427, PPSA99006, PPSA99082, PPSA99600, PPSA99000, PPSA99520
  • GPUs tested: Apple M2, MoltenVK 1.4.2, macOS 26.7.1 (x86-64 under Rosetta)
  • AI-assisted: yes

Alexius0815 and others added 7 commits October 17, 2026 14:36
…stages

Since boykopovar#3878 every vertex program that exports a position also writes
gl_ClipDistance for the all-zero position distance. The tessellation
stages generated for rect lists declared only gl_Position in gl_in, so
their input interface no longer matched the vertex outputs; on MoltenVK
the control stage then reads all-zero positions and every rect-list
draw stops with "rect-list requires finite nondegenerate axis-aligned
positions" (reported in boykopovar#1).

The control and evaluation stages now declare the vertex program's
gl_ClipDistance array in their per-vertex blocks and forward it. The
generated fourth corner gets the clip distances interpolated with the
same weights as its position, so the all-zero position clip keeps
working for rect lists. Validation accepts gl_ClipDistance as a
tessellation input and as a tessellation-control output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QXdkh9PHBCPqGcGQPeikAe
On a device without geometry shaders, each triangle becomes a three-point
patch at tessellation level 1. The evaluation stage takes the corner
gl_TessCoord points at as its vertex and writes it as the weights, and
passes every vertex's parameters for raw reads. The vertex clip distances
are forwarded, so MoltenVK's stage interface matches the vertex output.
… shaders

A triangle list without primitive restart whose pixel shader emulates
barycentrics draws through the generated tessellation stages when the
device has tessellation but no geometry shaders, as on MoltenVK. The
pipeline uses a three-point patch list for it, keeps it out of pipeline
libraries and validates the generated stages' modes and interface.
…ic-tessellation

# Conflicts:
#	core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
#	docs/dev/debt/shader.md
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6317 docs/dev/debt/shader.md L146
PR 6470 docs/dev/debt/shader.md L146-147
PR 6612 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp L607, L638

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1341 core/shader/recompiler/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 2434 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 3247 core/shader/recompiler/Recompiler.hpp
PR 4040 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4072 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
PR 4448 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
PR 4472 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4650 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 4891 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5021 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 5404 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
PR 5426 docs/dev/debt/shader.md
PR 5431 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5441 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5499 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5500 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
docs/dev/debt/shader.md
PR 5609 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/Recompiler.hpp
PR 5925 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/CMakeLists.txt
core/shader/recompiler/Recompiler.hpp
PR 5941 docs/dev/debt/shader.md
PR 5958 core/shader/recompiler/Recompiler.hpp
docs/dev/debt/shader.md
PR 5998 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6009 docs/dev/debt/shader.md
PR 6094 docs/dev/debt/shader.md
PR 6177 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/Recompiler.hpp
PR 6188 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 6194 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6222 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/libs/prx/libSceAgcDriver/tests/execution/PixelBarycentricEmulation.cpp
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/src/RectListShaders.cpp
docs/dev/debt/shader.md
PR 6235 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
docs/dev/debt/shader.md
PR 6259 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6292 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
PR 6297 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6320 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6412 core/shader/recompiler/Recompiler.hpp
docs/dev/debt/shader.md
PR 6414 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6454 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/CMakeLists.txt
docs/dev/debt/shader.md
PR 6457 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6479 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6481 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6486 core/shader/recompiler/Recompiler.hpp
PR 6535 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6539 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/Recompiler.hpp
PR 6548 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6563 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6615 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
PR 6635 core/shader/recompiler/Recompiler.hpp
PR 6647 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6730 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6783 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6786 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6788 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6804 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6808 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6931 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
PR 6946 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/shader/recompiler/CMakeLists.txt
core/shader/recompiler/Recompiler.hpp
PR 6947 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderValidation.cpp
core/shader/recompiler/CMakeLists.txt
core/shader/recompiler/Recompiler.hpp
PR 6973 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 6977 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/shader/recompiler/CMakeLists.txt
docs/dev/debt/shader.md
PR 6987 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 6989 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 6995 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 7002 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/include/Pipeline.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
core/shader/recompiler/Recompiler.hpp
PR 7089 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 7154 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7175 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 7201 core/libs/prx/libSceAgcDriver/Execution/include/VulkanDevice.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Draw/Draw.cpp
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
core/libs/prx/libSceAgcDriver/Graphics/src/Pipeline.cpp
core/shader/recompiler/CMakeLists.txt
core/shader/recompiler/Recompiler.hpp
core/shader/recompiler/SpirvBackend/src/RectListShaders.cpp
PR 7223 docs/dev/debt/shader.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants