Skip to content

Update DirectX-Headers to latest - #8431

Closed
Antonio Maiorano (amaiorano) wants to merge 3 commits into
microsoft:mainfrom
amaiorano:update-directx-headers
Closed

Antonio Maiorano (amaiorano) wants to merge 3 commits into
microsoft:mainfrom
amaiorano:update-directx-headers

Conversation

@amaiorano

Copy link
Copy Markdown
Collaborator

The version being used was from four years ago. Updating to upstream's main broke a lot of code because DirectX-Headers now has the equivalent of WinAdapter.h under include/wsl for non-Windows machines, and these collided with DXC's own definitions. I fixed this by including the relevant DirectX-Headers in WinAdapter.h, removed duplicate macros, and adapted some of the existing ones.

Fixes #5079

The version being used was from four years ago. Updating to upstream's
main broke a lot of code because DirectX-Headers now has the equivalent
of WinAdapter.h under `include/wsl` for non-Windows machines, and these
collided with DXC's own definitions. I fixed this by including the
relevant DirectX-Headers in WinAdapter.h, removed duplicate macros, and
adapted some of the existing ones.

Fixes microsoft#5079
@llvm-beanz

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@amaiorano

Antonio Maiorano (amaiorano) commented May 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

All tests pass, but I will share some questions/concerns:

The types that I removed from WinAdapter.h are because they are already defined in external/DirectX-Headers/include/wsl/stubs/basetsd.h. Most of them are effectively the same, but there are some differences:

image
  • The BOOL type going from bool to uint32_t might be a concern, especially if used in the public facing API, but I'm not sure.
  • BOOLEAN was char and is now BYTE which is unsigned char
  • The rest went from non-sized types like int, long, char, to sized types like int32_t, int64_t, int8_t, which should be mostly okay on current 32 and 64 bit platforms, except maybe long, which went from long to int32_t. I know that long is usually 32-bits on Windows, and 64-bits on Linux (and Mac?).

The next concern to bring up: I don't know if this was the right move, but I modified the root CMakeLists.txt to move the include_directories of DirectX-Headers paths higher up so that it would be visible to all targets. This allowed me to include one DirectX-Headers file, wsl/winadapter.h, which ultimately includes other files like wsl/stubs/basetsd.h, directly at the top of WinAdapter.h. The concern here is that I noticed that WinAdapter.h is one of the install files in CMake (see DXC_PLATFORM_PUBLIC_HEADERS in include/dxc/CMakeLists.txt). So this file was meant to be a standalone includable file, so I'm not sure if this will work for users of an installed DXC (I don't think there are any tests for this scenario). Users of DXC would also need DirectX-Includes and add DirectX-Headers/include to their include directories. Is this okay? If not, what should be done?

@amaiorano

Copy link
Copy Markdown
Collaborator Author

I did a bit more digging to understand the root of what changed in DirectX-Headers. Basically:

In old DirectX-Headers:

wsl/stubs really contains only stubs. For example, external/DirectX-Headers/include/wsl/stubs/rpcndr.h was basically empty.

external/DirectX-Headers/include/wsl/winadapter.h did contain Windows types for non-Windows, but this file was not included in DXC.

In latest DirectX-Headers:

wsl/stubs are no longer really stubs. Now external/DirectX-Headers/include/wsl/stubs/rpcndr.h defines __CRT_UUID_DECL and includes external/DirectX-Headers/include/wsl/stubs/basetsd.h (also in stubs) which is where most of what was in external/DirectX-Headers/include/wsl/winadapter.h has moved to.

A concrete example of a build error we get when updating DirectX-Headers:

Building lib/DxilContainer/CMakeFiles/LLVMDxilContainer.dir/D3DReflectionStrings.cpp.o fails because of the inclusion chain:

lib/DxilContainer/D3DReflectionStrings.cpp
  include/dxc/Test/D3DReflectionStrings.h
    include/dxc/Support/D3DReflection.h
      external/DirectX-Headers/include/directx/d3d12shader.h
        external/DirectX-Headers/include/directx/d3dcommon.h
          external/DirectX-Headers/include/wsl/stubs/rpcndr.h
            external/DirectX-Headers/include/wsl/stubs/basetsd.h

If we comment out the include of d3d12shader.h in D3DReflection.h, then we get compile errors because of unknown enums:

/home/amaiorano/src/external/DirectXShaderCompiler/include/dxc/Test/D3DReflectionStrings.h:24:17: error: unknown type name 'D3D_CBUFFER_TYPE'
   24 | LPCSTR ToString(D3D_CBUFFER_TYPE CBType);
      |                 ^
/home/amaiorano/src/external/DirectXShaderCompiler/include/dxc/Test/D3DReflectionStrings.h:25:17: error: unknown type name 'D3D_SHADER_INPUT_TYPE'
   25 | LPCSTR ToString(D3D_SHADER_INPUT_TYPE Type);
      |                 ^

I thought of maybe "disabling" the ultimate inclusion of "basetsd.h" by defining the header guard, but unfortunately this file doesn't use a classic header guard, but uses #pragma once. When I delete the contents of the file locally just to test, I am able to compile D3DReflectionStrings.cpp.

In fact, if I delete the contents of "basetsd.h", and "rpcndr.h", but make sure to #define __RPCNDR_H_VERSION__ before including d3d headers, all of dxc builds.

So I'm not really seeing a way out of this that's much different from the change I've put up here. As I wrote in my previous comment, users of DXC on Linux/Mac would need DirectX-Includes and add DirectX-Headers/include/wsl/stubs to their include directories.

Add root of DirectX-Headers/include to CMake include_directories, and
only include 'wsl/winadapter.h' at the top of DXC's WinAdapter.h. This
should make it easier for users of an installed DXC to add
DirectX-Headers as a dep.
@damyanp

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@damyanp Damyan Pepper (damyanp) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left a few questions. I think that the main thing we need to figure out is how do we convince ourselves that this change doesn't break anything? Is it enough that the build pipelines pass? Do we need to do some manual live testing on non-Windows systems?

Tex Riddell (@tex3d), Jesse Natalie (@jenatali), do you have any thoughts here?

Comment thread include/dxc/WinAdapter.h

#define SUCCEEDED(hr) (((HRESULT)(hr)) >= 0)
#define FAILED(hr) (((HRESULT)(hr)) < 0)
#define DXC_FAILED(hr) (((HRESULT)(hr)) < 0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does DXC_FAILED still need to exist?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in a few places. It could be removed and replaced with FAILED, but that should be done in a follow-up change, I think.

Comment thread include/dxc/WinAdapter.h

#define E_ABORT (HRESULT)0x80004004
#define E_ACCESSDENIED (HRESULT)0x80070005
#define E_BOUNDS (HRESULT)0x8000000B

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably the defines removed here conflicted with ones in DirectX headers? Are the remaining ones still required?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and yes. Both E_BOUNDS and E_NOT_VALID_STATE are used in the DXC codebase, but are not defined in wsl/stubs/basetsd.h (DirectX-Includes).

Comment thread include/dxc/WinAdapter.h Resolved
@jenatali

Copy link
Copy Markdown
Member

This looks right to me. The DirectX-Headers definitions mean that datatypes and behaviors will be the same across Windows and Linux so I think standardizing on that is the right call even if it's different than how they were defined before. I'm not sure how thorough the tests are for the Linux builds, but as long as we can compile some basic shaders I suspect that this change would be fine.

So I'm not really seeing a way out of this that's much different from the change I've put up here. As I wrote in my previous comment, users of DXC on Linux/Mac would need DirectX-Includes and add DirectX-Headers/include/wsl/stubs to their include directories.

I don't have a great answer here but at least for now this seems okay?

@amaiorano

Antonio Maiorano (amaiorano) commented May 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

I've left a few questions. I think that the main thing we need to figure out is how do we convince ourselves that this change doesn't break anything? Is it enough that the build pipelines pass? Do we need to do some manual live testing on non-Windows systems?

I suspect it may break projects that depend on an installed DXC on non-Windows as they would now need to also depend on DirectX-Headers (and add it to their include path). I think projects that use CMake add_subdirectory should work out of the box. It would be nice to test these cases, I'm just not sure where to look.

I can at least make sure this new dep on DirectX-Headers is clearly stated in the commit comment, which may help downstream users of DXC.

@damyanp

Copy link
Copy Markdown
Member

So by "use dxc" here, I think we mean people who are compiling and linking against the DXC libraries? This doesn't impact anyone who just installs a binary release of dxc, right?

I suspect it may break projects that depend on an installed DXC on non-Windows as they would now need to also depend on DirectX-Headers (and add it to their include path).

Would also break people including it as a submodule?

I think projects that use CMake add_subdirectory should work out of the box. It would be nice to test these cases, I'm just not sure where to look.

Will this mean that the vcpkg maintainers will need to react to this change as well?

@damyanp

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@amaiorano

Copy link
Copy Markdown
Collaborator Author

So by "use dxc" here, I think we mean people who are compiling and linking against the DXC libraries? This doesn't impact anyone who just installs a binary release of dxc, right?

Yes.

Would also break people including it as a submodule?

I think by this you mean using add_subdirectory in CMake, which I think will work.

Will this mean that the vcpkg maintainers will need to react to this change as well?

Good question.

What I'm going to do is test different scenarios on Linux:

  1. Simple CMake project that uses an "install" of DXC (adds dxc/include to path).
  2. CMake project that depends on DXC using add_subdirectory (we actually do this in Dawn, but it's not easy.
  3. Project that consumes DXC via vcpkg.

As a side note, I tried building with gcc and got a bunch of warnings similar to those I got with Clang, so I'll add some pragmas to ignore them like I did for Clang.

@tex3d

Copy link
Copy Markdown
Contributor

In the past, we have intended to include all the (non-std) headers required to use the API in our GitHub release package. With this change, as mentioned, you must separately get and include at least basestd.h from DirectX-Headers.

Dependencies on DirectX-Headers were originally meant only for the DXC execution tests. DirectX headers weren't meant to be required by users of the DXC API, and developers who acquire the API through the GitHub release package might not be happy with a new project dependency just for a new 12k header that supplies a few definitions that used to be independently declared.

Maybe we could add this header to our GitHub release package to avoid any additional project dependencies when using the DXC API from that package. We could put it under another folder matching DirectX-Headers sub-path structure so it can easily be used as an include path instead of the full DirectX_headers, when minimal dependencies are desired. Otherwise, it seems excessive to add the whole DirectX-Headers project dependency just to include a 12k header.

Alternatively, we could move all our DXC API headers to the DirectX-Headers project, and no longer package them in the GitHub release packages. Then force everyone to get all the DXC API headers form DirectX-Headers for future releases.

@amaiorano

Copy link
Copy Markdown
Collaborator Author

Dependencies on DirectX-Headers were originally meant only for the DXC execution tests. DirectX headers weren't meant to be required by users of the DXC API, and developers who acquire the API through the GitHub release package might not be happy with a new project dependency just for a new 12k header that supplies a few definitions that used to be independently declared.

Yes, I agree, and it's why I raised this concern above.

Maybe we could add this header to our GitHub release package to avoid any additional project dependencies when using the DXC API from that package. We could put it under another folder matching DirectX-Headers sub-path structure so it can easily be used as an include path instead of the full DirectX_headers, when minimal dependencies are desired. Otherwise, it seems excessive to add the whole DirectX-Headers project dependency just to include a 12k header.

I think this is a good idea, and would cause the least friction - at least in the short term. If I understand correctly, you're saying that users of the DXC API would have to update their include path no matter what: either to

  1. Include the new sub-path structure now available in the installed DXC directory;
  2. Or include the one from DirectX-Headers.

This would be a breaking change, no matter what, for all users of the DXC API; however, we provide an easy way out for non-DirectX-Headers users: add an extra include path.

If this is what you mean, and others agree with this, I could modify my PR to do this (add the copying of files needed to the CMake install target, add a comment at the top of WinAdapter.h explaining this, and updating the commit message as well).

@amaiorano

Copy link
Copy Markdown
Collaborator Author

I was discussing this issue with Rafael, and it occured to me that another potential solution here would be to copy the DirectX-Headers includes directly into the DXC repo, and drop the submodule dependency altogether.

Given that DXC has been pinned to is ~4.5 years old, and difficult to update (see above), I think copying the required files directly into the DXC repo is the simplest solution. This allows projects to both build DXC and depend on a more recent DirectX-Headers, as described in the original issue. This is also something we would like for Dawn/Chromium.

What do you think?

@damyanp

Copy link
Copy Markdown
Member

I'm not keen on checking in copies of files - I think we have some checked in already, and we should be working to reduce the number rather than increase them.

We're also prioritizing #8994 which should ensure we keep submodules like DirectX-Headers up to date going forward.

@mkomaiha

Copy link
Copy Markdown

I maintain portacom, a Python COM binding that loads libdxcompiler.so through FFI. I support this change, and I want to flag an issue I haven't seen raised in this thread. On Linux and macOS it is also a binary ABI change for anyone loading prebuilt libraries. BOOL goes from 1 byte to 4: GetEncoding(BOOL*) and HasOutput() cross that boundary. LONG/ULONG go from 64 to 32 bits, which affects AddRef/Release. Mixing old headers with a new .so (or the reverse) will misbehave silently. Could the release notes call this out as a non-Windows ABI break, so bindings like portacom and hassle-rs can switch at a known version? I'm happy to review, test against the CI artifacts, or contribute a small test that checks sizeof(BOOL)/sizeof(ULONG).

@RafaelCintron

Rafael Cintron (RafaelCintron) commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Damyan Pepper (@damyanp) and Chris B (@llvm-beanz), to support Dawn partners, Git dependencies are preferred over Nuget ones.

Antonio's PR has DXC changes needed before we can do autorolls. Can we please merge it while we wait for #8994 to land?

@damyan

This comment was marked as off-topic.

Comment thread include/dxc/WinAdapter.h
Comment on lines +415 to +417
static_cast<uint32_t>(byte_from_hexstr(&(str)[2])) << 16 | \
static_cast<uint32_t>(byte_from_hexstr(&(str)[4])) << 8 | \
byte_from_hexstr(&(str)[6]), \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to skip the first character of the string. Compare to the original version that does str, str+2, str+4 and str+6 - this version only does str.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good catch. Turns out that CI didn't catch this because dxc and tests use the same macro, so the IIDs still match each other.

@damyanp

Copy link
Copy Markdown
Member

I started digging into this all a bit more and ended up with another attempt for it - #9149. I'll close this PR in preference to that one and we can see if we can use that to get a version ready that works for everyone.

@amaiorano

Antonio Maiorano (amaiorano) commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

I started digging into this all a bit more and ended up with another attempt for it - #9149. I'll close this PR in preference to that one and we can see if we can use that to get a version ready that works for everyone.

Hmm, I would have been happy to work with you on making this PR land, especially as I spent quite some time on it (before AI), and because the new PR looks quite similar to mine, including making it a breaking change for non-Windows (DXC's public headers now depend on DirectX-Headers), and doing what Tex suggested and I agreed to in shipping the DirectX-Headers files for DXC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Conflict with DirectX-Headers

8 participants