The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2977419	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1421610	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#54313
1424097	dotnet/aspnetcore	Run Quarantined Tests	Log
2977224	dotnet-runtime	Run Astred GrcParquet	Log
2977237	dotnet-efcore	Run Cosmos tests	Log
2977169	dotnet-efcore	Run Cosmos tests	Log
2977146	dotnet-runtime	Run Astred Analysis and Copy Results	Log
1423856	dotnet/aspnetcore	Run Quarantined Tests	Log
2977127	dotnet-runtime	Send job to Helix (Windows)	Log
1423847	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126542
1423657	dotnet/maui	Controls WebView	Log
1423672	dotnet/runtime	AOT compile CoreCLR tests	Log
2976890	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
2976863	dotnet-roslyn	Test	Log
1423593	dotnet/runtime	Send tests to Helix (Unix)	Log
1423548	dotnet/maui	Controls WebView	Log
2976778	dotnet-aspnetcore	Run build.cmd helix target	Log
2976761	dotnet-aspnetcore	Run build.cmd helix target	Log
2976753	dotnet-runtime	Run Astred Analysis and Copy Results	Log
2976704	dotnet-aspnetcore	Run build.cmd helix target	Log	#60795
2976697	dotnet-aspnetcore	Run build.cmd helix target	Log	#60694
2976690	dotnet-runtime	Run Astred Analysis and Copy Results	Log
2976640	dotnet-roslyn	Test	Log
2976609	dotnet-roslyn	Test	Log
2976591	dotnet-roslyn	Test	Log
1423397	dotnet/runtime	Build product	Log	dotnet/runtime#128242
1423362	dotnet/runtime	Build product	Log	dotnet/runtime#127780
2976528	dotnet-aspnetcore	Run build.cmd helix target	Log
2976524	dotnet-aspnetcore	Run build.cmd helix target	Log
2976512	dotnet-roslyn	Test	Log
1423146	dotnet/maui	Controls WebView	Log
2976484	dotnet-runtime	Send to Helix	Log
2976466	dotnet-efcore	Send tests to helix	Log
2976426	dotnet-roslyn	Test	Log
2976386	dotnet-roslyn	Test	Log
1422938	dotnet/maui	Controls CollectionView	Log
1422960	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#128237
2976373	dotnet-roslyn	Test	Log
1423053	dotnet/runtime	Send tests to Helix (Unix)	Log
2976362	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
2976348	dotnet-roslyn	Test	Log
1423087	dotnet/aspnetcore	Run Quarantined Tests	Log
1423017	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#128269
2976320	dotnet-roslyn	Test	Log
1422947	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128275
1422963	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128222
2976317	dotnet-roslyn	Test	Log
1423011	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#127989
1422941	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128250
1423000	dotnet/maui	Controls WebView	Log
1422978	dotnet/runtime	Send to Helix	Log	dotnet/runtime#125596
1422954	dotnet/runtime	Build product	Log	dotnet/runtime#128263
1422918	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128274
1422913	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128273
1422876	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128054
1422854	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128271
1422805	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128243
1422817	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128270
1422810	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128230
1422706	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128264
1422539	dotnet/runtime	Publish	Log	dotnet/runtime#128170
1422324	dotnet/maui	Controls Layout	Log
2976269	dotnet-roslyn	Test	Log
2976267	dotnet-roslyn	Test	Log
1422940	dotnet/roslyn	Build	Log	dotnet/roslyn#83725
1422611	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128001
1422787	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#128268
1422401	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#127520
1422587	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127770
1422744	dotnet/aspnetcore	Run Quarantined Tests	Log
1422568	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128262
1422746	dotnet/runtime	Send to Helix	Log
1422528	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126947
1422557	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128261
1422521	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127690
1422511	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127814
1422469	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127970
1422470	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128238
1422456	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128232
1422445	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128116
1421619	dotnet/maui	Controls CollectionView	Log	dotnet/maui#35447
1422066	dotnet/maui	Controls Shell	Log
2976217	dotnet-aspnetcore	Run build.cmd helix target	Log
2976211	dotnet-aspnetcore	Run build.cmd helix target	Log
2976085	dotnet-maui	Controls CollectionView	Log
2976197	dotnet-aspnetcore	Run build.cmd helix target	Log	#61130
2976122	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61144
1422628	dotnet/aspnetcore	Run Quarantined Tests	Log	dotnet/aspnetcore#66528
2976198	dotnet-sdk	Run AoT Tests in Helix	Log	#60891
1422222	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#125436
1422348	dotnet/runtime	Build product	Log	dotnet/runtime#127578
1422343	dotnet/runtime	Build product	Log	dotnet/runtime#128257
2976126	dotnet-runtime	Send tests to Helix (Windows)	Log	#61148
1422163	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#128152
2976152	dotnet-efcore	Send tests to helix	Log
1422449	dotnet/runtime	AOT compile CoreCLR tests	Log	dotnet/runtime#128116
1422167	dotnet/runtime	Send to Helix	Log	dotnet/runtime#128162
2976124	dotnet-aspnetcore	Run build.cmd helix target	Log	#61147
1422346	dotnet/maui	Controls CollectionView	Log	dotnet/maui#35460
2976121	dotnet-efcore	CmdLine@2	Log	#61146
Displaying 100 of 3116 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
12	844	3116

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez