Closed
Description
Recently tools builds on MSVC and GNU Windows platforms began to fail with errors like:
duplicate artfacts found when compiling a tool, this typically means that something was recompiled because a transitive dependency has different features activated than in a previous build:
winapi 0.3.4 (registry+https://github.com/rust-lang/crates.io-index)
`clippy-driver` enabled features ["consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "minwindef", "namedpipeapi", "ntdef", "processenv", "processthreadsapi", "std", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libwinapi-7cb6d087db077e11.rlib"
`cargo` enabled features ["consoleapi", "dbghelp", "errhandlingapi", "fileapi", "handleapi", "ioapiset", "jobapi", "jobapi2", "lmcons", "minschannel", "minwinbase", "minwindef", "namedpipeapi", "ntdef", "ntsecapi", "ntstatus", "processenv", "processthreadsapi", "profileapi", "psapi", "schannel", "securitybaseapi", "std", "synchapi", "sysinfoapi", "timezoneapi", "winbase", "wincon", "wincrypt", "winerror", "winnt", "winsock2", "ws2def", "ws2ipdef", "ws2tcpip"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libwinapi-d6c198bedd24dbbb.rlib"
backtrace 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)
`clippy-driver` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libbacktrace-4d22e1f3c416866d.rlib"
`cargo` enabled features ["backtrace-sys", "coresymbolication", "dbghelp", "default", "dladdr", "libbacktrace", "libunwind", "winapi"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libbacktrace-1b3034da488e3e14.rlib"
error-chain 0.11.0 (registry+https://github.com/rust-lang/crates.io-index)
`clippy-driver` enabled features ["backtrace", "default", "example_generated"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\liberror_chain-6878f7237b9efb05.rlib"
`rls` enabled features ["backtrace", "default", "example_generated"] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\liberror_chain-1e94c8f434bd4c43.rlib"
cargo_metadata 0.5.4 (registry+https://github.com/rust-lang/crates.io-index)
`clippy-driver` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libcargo_metadata-03e90dedac875643.rlib"
`rls` enabled features [] at "C:\\projects\\rust\\build\\x86_64-pc-windows-gnu\\stage2-tools\\x86_64-pc-windows-gnu\\release\\deps\\libcargo_metadata-39ee2e9489759a28.rlib"
Affected PRs
- rustbuild: Fail the build if we build Cargo twice #49053 (comment) First PR that experienced this,
possibly this is where the bug was introduced.I think this started with Cargo update, cannot find relevant comment right now. - Did you mean to block nightlies on clippy? #51122 CI log Succeeded after adding dependencies hack: rust-lang/rust-clippy@41972f8
- Update RLS and Rustfmt #51677 CI log
- Update the clippy submodule #52012 CI log Succeed after updating the hack
cc @alexcrichton (creator of the first PR with this issue)
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
[-]Tracking issue for tools build error on Windows `duplicate artfacts found when compiling a tool`[/-][+]Tracking issue for tools build error on Windows: `duplicate artfacts found when compiling a tool`[/+]oli-obk commentedon Jul 5, 2018
fixed in #52012 (clippy PR: rust-lang/rust-clippy#2889)
We needed to manually ensure that clippy's and cargo's depedencies all have the same features enabled
alexcrichton commentedon Jul 5, 2018
Is this a flaky error happening on CI? Or is this preventing PRs from landing?
If it's the latter that's how this is supposed to be wokring?
oli-obk commentedon Jul 5, 2018
No, the issue is that it's not quite trivial to sync the features that are active for dependencies of
clippy-driver
,rls
andcargo
for
rls
andclippy-driver
it should become easy becauserls
soon depends onclippy
, butclippy-driver
andcargo
might still decide to diverge when dependencies change their features. We'll see that on CI though and "just" need to change the list of features used inCargo.toml
, even if the features are unused by that cratealexcrichton commentedon Jul 5, 2018
Ah ok, thanks for clarifying! This definitely is a bummer of an issue but we don't have the CI budget to build these tools multiple times on all CI runs. The biggest "offender" so far has been winapi and we could tweak these crates to just enable all features of winapi or some larger subset to prevent this from coming up too much.
mati865 commentedon Jul 5, 2018
@alexcrichton it actually prevents tool related PRs from landing unless they add hack to make them use the same features as cargo which @oli-obk mentioned earlier.
The hack looks like that: https://github.com/rust-lang-nursery/rust-clippy/blob/28daee4c919dd88772847ec5240eb850bb0dcbf3/Cargo.toml#L48-L84
Similar "thing" has to be applied to rls so it can build (and unblock nightlies).
To solve it without hacks build system would have to collect features
cargo
,clippy-driver
,rls
and possiblymiri
are using for each dependency and merge them in one lib. One lib to rule them all and contain every required feature.Actually this issue might be not related to Windows.
All failed jobs come form AppVeyor and I connected it to Windows toolchains but it seems AppVeyor is just faster than Travis. So in fact Travis had no opportunity to fail just yet.
[-]Tracking issue for tools build error on Windows: `duplicate artfacts found when compiling a tool`[/-][+]Tracking issue for tools build error: `duplicate artfacts found when compiling a tool`[/+]alexcrichton commentedon Jul 5, 2018
Indeed it is a hack! And indeed it is a nasty and pretty bad hack! Unfortunately though we don't have many options, we can't rebuild cargo twice on CI, we don't have budget for that. If there's other ways to solve this though that'd be great!
oli-obk commentedon Jul 5, 2018
I'm all for deduplicating as much stuff as possible. I just thought that workspaces already did that and that the entire rustc repo is one big workspace. At least we removed the clippy workspace to be able to integrate into the rustc workspace.
kennytm commentedon Jul 5, 2018
If we just whitelist
winapi
from the duplication check would it slow down the CI too much?mati865 commentedon Jul 5, 2018
Not sure if I'm getting it right but wouldn't whitelisting
winapi
mean enabling other duplicates as well?See how all libs mentioned here have different hashes:
Click there to show
oli-obk commentedon Jul 5, 2018
I'd say we keep what we have and fix the issues as they come. I'll try to cook up better diagnostics and investigate why workspaces don't already fix all this for us.
alexcrichton commentedon Jul 5, 2018
@oli-obk workspaces deduplicate versions of dependencies but doesn't currently deduplicate and/or unify the activated features for dependencies (this is arguably a misfeature of Cargo).
@kennytm unfortunately
winapi
is the source of the problems here and we can't cover over it. Ifwinapi
is recompiled then everything that depends onwinapi
is also recompiled, which typically includes most of the crate graph.Right now the best solution is a pretty bad solution (unfortunately) which is to just manually make sure that all our relevant projects depend on all the same features. That's what I've done in Cargo historically
9 remaining items