Generate consistent bindings

[Nielsbishere]

Currently, when resources are declared without explicit register bindings, DXC assigns them after GlobalDCE runs. This can lead to inconsistent register assignments across entry points (depending on optimizations), which causes mismatched resource IDs.

A common case is bindless-style includes, e.g. declaring multiple textures without explicit bindings but with spaces. In this situation, the generated binaries may not match across different entry points or when compiling as a library.

Solution

This PR introduces a new flag:

-consistent-bindings

When enabled, DXC will generate stable bindings by letting GlobalDCE eliminate only unused resources' symbols. Then, the bindings are generated and finally the unused resources are stripped again.

This ensures consistent register assignments across all entry points, whether compiling as individual shaders or as a library.

• Resources are preserved until final binding assignment.
• After assignment, unused bindings are stripped to avoid breaking optimization assumptions.
• The behavior is similar to -flegacy-resource-reservation, but tailored for ensuring binding consistency.

Example

Command:

dxc -consistent-bindings -E main -T cs_6_0 myFile.hlsl

Shader:

Texture2D a;
Texture2D b;
RWTexture2D<float4> c;

[numthreads(1,1,1)]
void main() {
   c[0.xx] = b[0.xx];
}

[numthreads(1,1,1)]
void main2() {
   c[0.xx] = a[0.xx];
}

Generated bindings:

• a → t0
• b → t1
• c → u0

Both main and main2 will consistently use the same bindings (b at t1, c at u0), regardless of entry point or library compilation.

Notes

• Step 1 of 2 toward fixing Allow unused registers to be output to reflection #5105.
• Replaces custom preprocessing logic with compiler-side handling for stability and maintainability.

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[llvm-beanz]

@hekota & @bogner, please look over this. I'm not sure how this aligns with our work in Clang and how we want to handle bindings in the future.

[Nielsbishere]

Important information to add is that the spirv backend already handles this correctly, so no fix is needed there. Will add a unit test as well.

[hekota]

@Nielsbishere - Looking at this change and your proposed update to add -keep-all-resources flag, could you please help me understand what the benefit of the -consistent-bindings flag is compared to -keep-all-resources?

It seems to me that -keep-all-resources should already handle the consistent binding issue since it basically means “don’t strip unused resources.” The bindings in the example above end up inconsistent across different entry functions because the unused resources get stripped.

Also, as you’ve seen, implicit resource bindings tend to cause trouble since they make shader–application interfaces unpredictable and potentially compiler-dependent. It’d be great if you could find a way to stick with the explicit bindings to make sure your resource layouts stay stable and consistent.

[tex3d]

Above, after the call to LegalizeResources, it calls RemoveResourcesWithUnusedSymbols if bLocalChanged. Don't you want to predicate that with if (!DM.GetConsistentBindings())? Otherwise, resources may be removed before calling AllocateRegisters.

Also, it seems we should be tracking changes from this with bChanged. bChanged = bChanged || (numResources != newResources); is used earlier for this, and a similar thing could be done here after updating newResources. An alternative is to modify RemoveResourcesWithUnusedSymbols to return true if any changes were made.

Side note: Looking a little further up, I noticed when ApplyBindingTableFromMetadata is called, it doesn't update bChanged either, but that's an existing bug.

[tex3d]

We probably need a test where LegalizeResources eliminates a resource use to test that code path. This relies on DxilValueCache, which was added largely to handle known-constant values in -Od cases where these values aren't simplified in the IR, but certain transformations we need to perform require these known values. In other words, you may be able to hit this code path using -Od with something that uses one resource or another in two code paths, depending on some condition that can be simplified to a constant, but is not normally simplified for -Od.

[Nielsbishere]

Good catches, have adjusted RemoveResourcesWithUnusedSymbols to return isModified and passing it along to bChanged. Also fixed the issue you said with LegalizeResources.
Since the issue ApplyBindingTableFromMetadata was already in there, it might be worth tackling this in a further PR/issue.
As for the test case for LegalizeResources; I'm not entirely sure where this would happen, I tried to read the code of LegalizeResources but it wasn't completely clear to me. Is there a unit test that tests legalize resources that I can combine with -consistent-bindings as a test case?

[Nielsbishere]

@hekota excuse me for the (following) long message, but these two have two very different uses in my previous work place. I'm not maintaining there anymore, but I still want them to have these changes to continue my work and will tell them how it can be used.
The compile process is quite complex, so I won't fully go over it here, but essentially:

• Legacy technique syntax is parsed and replaced (PSO combinations, entrypoint definitions); coming from a DX9 codebase to DXR/DX12.
• Shader parsing & preprocessing is ran to acquire register bindings & even optimized out registers (and to ensure consistent bindings manually by inserting register annotations)
• Compilation & linking (shader variants with IDxcLinker)
• Shader reflection is used on the final binary to find non optimized out resources (for transitions and knowing if certain debug commands are used)

The pipeline going forward could be:

• Legacy technique syntax the same way, or a different system managing this in C++
• Compile lib with -keep-all-resources to retrieve all resources
• Compile & link each unique define combination with -consistent-bindings (getting rid of optimized out resources, but having consistent bindings)
• Shader reflection (same thing as before) and validate that no new registers are created or have mismatching bindings for safety (though never happens in practice)

-keep-all-resources

is intended to (currently not working as it should in reflection, due to a missing createHandle) allow behavior similar to -Od with spirv; keep all resources even if optimized out. The reason why you don't want this by default is because reflection (presence of a register) may be used to guide for example transitions (or to know which real registers to keep). Why would you even want optimized out registers? Because sometimes you want a consistent interface that knows about the inputs to a shader, even if those shaders don't use the resource. It can still optimize it in the background but the interface will still show this variable as present. An example could be a visual scripting language that relies on this. The one I'm talking about relies on manual shader parsing to find out optimized out registers for the stability, adding this feature would allow to remove the manual parsing eventually. Consistent bindings in comparison, doesn't affect the emitted registers, only the bindings. So -keep-all-resources could be used to discover reflection info first and then a final compile per entrypoint could be used with -consistent-bindings to ensure no registers will be leftover.

-consistent-bindings

is a separate issue in the same system. In modern or legacy pipelines, shaders might have been written without explicit bindings or legacy semantics. These get registers auto assigned. Once again, the same shader parser is used to replace these legacy semantics with register bindings to avoid this issue this PR fixes. A legit use case for this is poor man's bindless, where you have arrays of textures/buffers. You could have 1 central include that defines this layout and puts them at space1 (no explicit bind point). If you change the first array size or add a register to this include it will automatically update. Otherwise you need to adjust all subsequent resource bindings. The system I'm talking about also validates that the final shader reflection matches the expected bindings to avoid accidental mismatches.
With both of those solved, the parser can drop all HLSL preprocessing and rely on C++ annotations for the remainder that can't easily be parsed. I'm not maintaining this system anymore, but in my own framework this would also be useful.

[Nielsbishere]

I see what you mean though, I think this is very possible and I've added behavior to keep-all-resources that would make this possible, but it has a bug I'm trying to figure out now (duplicates functions somehow?).
Basically what it does is it maintains all resources (except $Globals if no members or buffers of 0 size) and has a flag if the register is unused or not. Libraries use something similar but have more to worry about because one function might not reference a register but one of the others might.

The big problem with that PR relative to the consistent bindings (when it works) is that I'm not sure how this would interact with any tools out in the wild. Anything that might assume registers will be stripped if they're unused would get issues if they see it, because they don't utilize this flag to detect if it's really unused or not. So personally I'd see it more as a 'give me more reflection' option rather than something people should ship binaries with until other tools/drivers are aware of this change in behavior.

[hekota]

There is also the option to combine -keep-all-resources with -Qstrip_reflect for shipping binaries to make sure the implicit bindings are consistent and that there is no reflection bloat from it. Would that work in your case?

[Nielsbishere]

@hekota In theory yes, especially with my latest changes made that mark a resource with an unused flag to determine if a resource is used in the final binary. However, with -Qstrip_reflect It ends up only stripping the name for me (context: the simplified consistent bindings test from the other PR):

; Name                                 Type  Format         Dim      ID      HLSL Bind  Count
; ------------------------------ ---------- ------- ----------- ------- -------------- ------
;                                       UAV    byte         r/w      U0             u0     1
;                                       UAV    byte         r/w      U1             u1     1

This one still contains the unused resource. Or do you want me to modify Qstrip_reflect to drop unused resources?