Commit 41013c1
[ci-failure-scan] Tighten KBE filing rules and PR-search coverage (#127961)
## Description
Refines the `ci-failure-scan` agentic-workflow prompt to address failure
modes seen in past runs:
- Generic `ErrorMessage` signatures (bare test names, exception types,
or truncated prefixes) that match `[PASS]` / `[SKIP]` lines for the same
test and turn the KBE into a false-positive matcher against passing
builds.
- Malformed JSON fences (4-backtick opens, mismatched fence lengths,
multiple skeletons in one body) that cause Build Analysis to silently
skip the issue.
- Issues filed under the `Known Build Error` label with no JSON block at
all, so nothing matches future failures.
- Muting PRs that link to a non-existent issue number or duplicate an
in-flight fix authored by a maintainer (because the search was scoped
too narrowly to `[ci-scan]` PRs).
- Muting PRs opened against issues whose area owners had already
provided a PR
- Wrong-KBE links where the candidate KBE matched only on test name but
was filed against a different architecture / failure signature.
## Changes
The body of `.github/workflows/ci-failure-scan.md` is reorganized into a
clear walk-through:
1. **Two-pass KBE → PR flow** is now a numbered six-step pre-flight walk
(existing KBE / area tracker / muting PR / in-flight fix PR / issue
resolves / mute is welcome) followed by an explicit action selection
(file KBE, open muting PR, optionally open fix PR).
2. **KBE-match verification** — four questions (test, signature, OS,
architecture) the agent must answer before linking an existing KBE.
Wrong answers mean filing a fresh KBE rather than reusing the wrong one.
3. **Body checks** — eight explicit checks on the issue body covering
the JSON fence, exact ```json opening, single-line/no-escapes signature,
and a negative-match test against `[PASS]` / `[SKIP]` and build-time
output.
4. **Bad → Good** examples for both signature shape (bare test name,
truncated prefix, bare exception type) and platform/csproj scope
(`linux-arm`-only, single-arch NativeAOT, single stress mode).
5. Coverage-discipline section trimmed to its unique contribution
(pipeline ordering, per-pipeline tally, run summary). Redundant `Submit`
section removed — its content is now covered by the numbered walk.
## Test run results
Workflow run
[25570821336](https://github.com/dotnet/runtime/actions/runs/25570821336)
was dispatched against this branch (commit `3cd6399dd70`,
pre-Copilot-fixup) and completed successfully (~28 min). Outputs:
| # | Type | Title | Linked tracker |
|---|---|---|---|
| [#127963](#127963) | PR (draft)
| `[ci-scan] Skip AsyncProfilerTests on Android and tvOS` | #127951 |
| [#127964](#127964) | PR (draft)
| `[ci-scan] Skip System.Net.Sockets IPv6 tests on Android` | #127565 |
| [#127965](#127965) | Issue
(KBE) | `[ci-scan] Known Build Error: System.Net.NameResolution
DnsGetHostAddresses_LocalhostSubdomainWithTrailingDot fails on Android`
| new |
| [#127966](#127966) | Issue
(regression) | `[ci-scan] Test failure: XslCompiledTransformApiTests (82
tests) on all NativeAOT legs — PlatformNotSupportedException
(Reflection.Emit)` | new |
| [#127967](#127967) | PR (draft)
| `[ci-scan] Exclude Vector3Interop from GC stress` | #127827 |
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 0160e21 commit 41013c1
1 file changed
Lines changed: 157 additions & 35 deletions
0 commit comments