From a0ccbde77b81a4335ff0d648072df5bbbe48bce5 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Thu, 4 Jun 2026 14:11:41 +0800 Subject: [PATCH 01/82] Add upstream-sync agent skill + bootstrap state Cherry-picks new commits from microsoft/terminal into this fork one at a time, preserving per-commit author/committer/dates (verified by smoke test: 5 picks from upstream/main~5..upstream/main matched character-for- character including timezone offsets). Pipeline (scheduler-friendly, resumable): fetch upstream -> compute pending (drop revert pairs + empties) -> branch upstream-sync/YYYY-MM-DD -> cherry-pick loop -> always write report -> open PR (or push direct to main) Safety: - state.json checkpoint (last_synced_upstream_sha + stuck-lock) - Tier-0 known-take-upstream auto-resolve (seed: spelling2.yml) - Stuck -> abort, open GitHub issue, set lock, exit 10 (not an error) - 'DO NOT squash' banner on PR body; -AutoMergeStrategy rebase arms gh auto-merge with the correct strategy Bootstrap: baseline = a325a2fa5 ('Fix settings error text legibility in High Contrast mode' #20098), the merge-base between fork main and upstream/main on 2026-06-04. 16 upstream commits currently pending; the first real sync run will be a follow-up PR after this one merges. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 149 ++++++++++++++ .../upstream-sync/references/bootstrap.md | 89 ++++++++ .../references/conflict-triage.md | 133 ++++++++++++ .../references/known-conflicts.md | 53 +++++ .../upstream-sync/references/reporting.md | 122 +++++++++++ .../upstream-sync/references/state-schema.md | 70 +++++++ .../upstream-sync/references/workflow.md | 181 +++++++++++++++++ .../upstream-sync/scripts/00-bootstrap.ps1 | 70 +++++++ .../scripts/01-fetch-upstream.ps1 | 18 ++ .../scripts/02-compute-pending.ps1 | 102 ++++++++++ .../scripts/03-cherry-pick-one.ps1 | 158 +++++++++++++++ .../upstream-sync/scripts/04-run-batch.ps1 | 190 ++++++++++++++++++ .../upstream-sync/scripts/05-write-report.ps1 | 155 ++++++++++++++ .../upstream-sync/scripts/06-finalize-pr.ps1 | 125 ++++++++++++ .../scripts/06b-finalize-direct.ps1 | 108 ++++++++++ .../scripts/07-open-stuck-issue.ps1 | 97 +++++++++ .../skills/upstream-sync/scripts/Common.ps1 | 100 +++++++++ .../upstream-sync/scripts/clear-stuck.ps1 | 61 ++++++ .github/upstream-sync/reports/.gitkeep | 0 .github/upstream-sync/state.json | 12 ++ 20 files changed, 1993 insertions(+) create mode 100644 .github/skills/upstream-sync/SKILL.md create mode 100644 .github/skills/upstream-sync/references/bootstrap.md create mode 100644 .github/skills/upstream-sync/references/conflict-triage.md create mode 100644 .github/skills/upstream-sync/references/known-conflicts.md create mode 100644 .github/skills/upstream-sync/references/reporting.md create mode 100644 .github/skills/upstream-sync/references/state-schema.md create mode 100644 .github/skills/upstream-sync/references/workflow.md create mode 100644 .github/skills/upstream-sync/scripts/00-bootstrap.ps1 create mode 100644 .github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 create mode 100644 .github/skills/upstream-sync/scripts/02-compute-pending.ps1 create mode 100644 .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 create mode 100644 .github/skills/upstream-sync/scripts/04-run-batch.ps1 create mode 100644 .github/skills/upstream-sync/scripts/05-write-report.ps1 create mode 100644 .github/skills/upstream-sync/scripts/06-finalize-pr.ps1 create mode 100644 .github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 create mode 100644 .github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 create mode 100644 .github/skills/upstream-sync/scripts/Common.ps1 create mode 100644 .github/skills/upstream-sync/scripts/clear-stuck.ps1 create mode 100644 .github/upstream-sync/reports/.gitkeep create mode 100644 .github/upstream-sync/state.json diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md new file mode 100644 index 000000000..c1991eeb9 --- /dev/null +++ b/.github/skills/upstream-sync/SKILL.md @@ -0,0 +1,149 @@ +--- +name: upstream-sync +description: 'Periodically sync new commits from microsoft/terminal into this manually-forked intelligent-terminal repo by cherry-picking commit-by-commit onto a dated sync branch, auto-skipping revert pairs and empty commits, auto-resolving known take-upstream files, and stopping cleanly on genuine conflicts with a written report and a GitHub issue. Use when the user asks to "sync upstream", "pull from microsoft/terminal", "run upstream sync", "catch up to upstream", or wires this into a scheduler (weekly/daily). Designed to be safe under repeated unattended runs.' +license: Complete terms in LICENSE.txt +--- + +# Upstream Sync (microsoft/terminal → intelligent-terminal) + +Cherry-pick commit-by-commit from `https://github.com/microsoft/terminal` +into this fork, preserving per-commit attribution, skipping commits that +cancel each other out, and stopping cleanly the moment a human-judgement +conflict appears. + +## When to Use This Skill + +- User asks to "sync upstream", "pull from microsoft/terminal", "catch up to upstream", or "run upstream sync". +- A scheduler (Task Scheduler, cron, GitHub Actions) invokes + [`scripts/04-run-batch.ps1`](./scripts/04-run-batch.ps1) on a weekly/daily cadence. +- The previous run left a stuck-lock and the human has finished resolving + the conflict — use [`scripts/clear-stuck.ps1`](./scripts/clear-stuck.ps1) and re-run. + +## When NOT to Use This Skill + +- The user wants a **one-shot rebase** of a single feature branch onto upstream — that's a normal `git rebase`, not this skill. +- The fork has never been initialized (`state.json` missing) — first do the one-time bootstrap from [references/bootstrap.md](./references/bootstrap.md). +- A stuck-lock is set — do not re-run; resolve the conflict on the stuck branch first. + +## Prerequisites + +- `git` 2.30+ and `gh` CLI authenticated against `microsoft/intelligent-terminal`. +- PowerShell 7+ (`pwsh`) on PATH. +- Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` + (the scripts create it if missing). +- `state.json` initialized once (see [references/bootstrap.md](./references/bootstrap.md)). + +## Why Cherry-Pick (Not Rebase, Not Merge) + +| Approach | Why rejected / chosen | +|---|---| +| **Rebase** `upstream/main` | ❌ Fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts. Verified failure on sister repo `agentic-terminal`. | +| **Merge** `upstream/main` | ⚠️ Works, but collapses the whole sync into one blob commit — kills per-commit review, kills `git bisect`. | +| **Cherry-pick commit-by-commit** | ✅ Preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR with N small commits. | + +## Step-by-Step Workflow + +The scheduler entrypoint is a single PowerShell script. Full procedure +with commands, exit codes, and the per-step delegation map lives in +[references/workflow.md](./references/workflow.md). + +``` +fetch upstream → compute pending → drop revert pairs → drop empties + → create sync branch → cherry-pick loop (auto-resolve T0/T1, abort on T3) + → write report (always) → push + open PR OR open stuck issue + lock +``` + +**Run it:** + +```pwsh +# Default: open a PR, let a human pick the merge strategy (must be rebase or merge — NOT squash) +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 + +# Open a PR AND arm GitHub auto-merge with rebase strategy (hands-off once CI/approvals pass) +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -AutoMergeStrategy rebase + +# Skip the PR entirely — fast-forward main to the sync tip. Requires admin/bypass on main. +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -PushDirectToMain + +# Compute & report without picking +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -DryRun +``` + +### Finalize modes — what each preserves + +| Mode | Per-commit content | Order on main | Original author date | Reviewer checkpoint | Requires admin? | +|---|---|---|---|---|---| +| PR + **rebase-merge** | ✅ | ✅ | ✅ | ✅ | No | +| PR + **merge commit** | ✅ | ✅ | ✅ | ✅ | No | +| PR + **squash** | ❌ collapsed | ❌ | ⚠️ folded | ✅ | No | +| **`-PushDirectToMain`** | ✅ | ✅ | ✅ | ❌ | Yes (push to main) | + +Committer date is "now" in every mode (git default for cherry-pick) — +that's the semantically correct "when this fork landed it" timestamp. + +Resumability is built into the state file — re-running after a successful +run is a fast no-op (nothing pending), and re-running while the stuck-lock +is set exits early without touching the branch. + +## Gotchas + +- **Never squash-merge the sync PR.** Squash collapses every cherry-picked + upstream commit into one, destroying per-commit attribution, original + author dates, and `git bisect` resolution. Use **"Rebase and merge"** + (preferred) or **"Create a merge commit"**. The PR body opens with a + banner reminding the reviewer; `-AutoMergeStrategy rebase` arms GitHub + auto-merge with the right strategy so a tired reviewer can't get it + wrong. +- **Never rebase `upstream/main` onto this fork.** Old "Merge upstream" + commits in the fork history replay and cascade conflicts. Use cherry-pick. + Verified failure mode on the sister repo `agentic-terminal`. +- **`.github/workflows/spelling2.yml` always conflicts** and the correct + resolution is always "take upstream wholesale". The Tier-0 list in + [references/known-conflicts.md](./references/known-conflicts.md) handles + this automatically — extend the list when you discover the next file + with the same pattern. +- **`gh pr create` on Windows fails with "Head sha can't be blank"** if the + branch is freshly pushed and not yet visible. The finalize script uses + `--head :` and a 5s retry to work around this — do not + "fix" it by removing the retry. +- **Do not run the scheduler twice while stuck.** The lock in + `state.json` makes the second run a no-op, but a human running the + script manually with `-Force` will overwrite the stuck branch and lose + their in-progress resolution. The `-Force` flag is documented but + intentionally not the default. +- **Cherry-pick over `git revert` style commits is intentional, not skipped.** + We only skip revert-pairs where **both** sides are inside the pending + range. A revert of a commit we already merged last week must land as a + normal pick — otherwise the fork diverges silently. +- **Always commit `state.json` and `reports/`** so the next scheduler + invocation (possibly on a different machine) starts from the right + checkpoint. The finalize PR includes the state update. +- **Never push directly to `main`.** The skill always opens a PR. Direct + push bypasses CI and human review of the upstream batch. +- **CRLF/LF on manifest files.** Cherry-picks normally preserve upstream + line endings, but any in-flight resolution touched by an LLM may downgrade + to LF. If a Tier-2 resolution touches a `.yml`/`.xml`/`.csproj`/winget + manifest, re-normalize before staging — see + [references/conflict-triage.md](./references/conflict-triage.md#line-endings). + +## Troubleshooting + +| Issue | Solution | +|---|---| +| `state.json` is missing | Run the one-time bootstrap — see [references/bootstrap.md](./references/bootstrap.md). Do not guess the baseline SHA. | +| Stuck-lock prevents new run | Resolve the conflict on the stuck branch, open a PR, merge, then run [`scripts/clear-stuck.ps1`](./scripts/clear-stuck.ps1) and re-run the batch. | +| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; the loop auto-resets and marks them skipped. No action needed. | +| Same file conflicts every run | Add it to the Tier-0 list in [references/known-conflicts.md](./references/known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | +| `gh pr create` returns "Head sha can't be blank" | Retry — the finalize script already does, but on slow networks may need a manual second run. | +| Report says "no-op" but I expected commits | Run `git fetch upstream main` manually and recompute — the scheduler may have run between upstream pushes. | + +## References + +- [references/workflow.md](./references/workflow.md) — full per-step procedure with exit codes and delegation map. +- [references/state-schema.md](./references/state-schema.md) — `state.json` shape and field semantics. +- [references/bootstrap.md](./references/bootstrap.md) — one-time baseline-SHA discovery and initialization. +- [references/conflict-triage.md](./references/conflict-triage.md) — Tier 0/1/2/3 resolution rubric with examples. +- [references/known-conflicts.md](./references/known-conflicts.md) — files that always need a fixed resolution. +- [references/reporting.md](./references/reporting.md) — report template and stuck-issue template. +- [scripts/04-run-batch.ps1](./scripts/04-run-batch.ps1) — the scheduler entrypoint. +- [scripts/clear-stuck.ps1](./scripts/clear-stuck.ps1) — clear the stuck-lock after human resolution. diff --git a/.github/skills/upstream-sync/references/bootstrap.md b/.github/skills/upstream-sync/references/bootstrap.md new file mode 100644 index 000000000..22ac1cf83 --- /dev/null +++ b/.github/skills/upstream-sync/references/bootstrap.md @@ -0,0 +1,89 @@ +# One-Time Bootstrap + +The skill is incremental — it needs to know which upstream commit the +fork is "caught up to" before it can compute a pending range. This page +covers establishing that baseline exactly once. + +## When to run + +- `state.json` does not exist yet. +- `state.json` exists but `last_synced_upstream_sha` is missing/`null`. +- You manually merged some upstream commits outside this skill and want + to fast-forward the baseline so the next sync doesn't re-pick them. + +**Do NOT** re-run bootstrap on a working skill. It overwrites the +baseline and can cause the next sync to either re-pick already-synced +commits (creating empties — harmless but noisy) or to skip pending +commits (silently dropping upstream changes — bad). + +## How to find the baseline SHA + +The "baseline" is the most recent upstream commit whose tree is +**fully contained** in the fork's history. Pick one of: + +### Method A — known last manual sync (preferred) + +If you remember the last upstream sync (PR or branch), grab the upstream +SHA mentioned in that PR description / commit message: + +```pwsh +git log --all --grep="upstream" --grep="microsoft/terminal" -i --oneline | head -20 +``` + +Look for messages like `Merge upstream main @ ` or +`Sync upstream up to `. That `` is your baseline. + +### Method B — patch-id scan + +For each recent fork commit (last ~200), get its patch-id and search +upstream for a matching patch-id: + +```pwsh +git fetch upstream main +git log --format='%H' -200 | ForEach-Object { + $pid = git show $_ | git patch-id --stable | ForEach-Object { ($_ -split ' ')[0] } + $match = git log upstream/main --format='%H %s' | ForEach-Object { + $usha = ($_ -split ' ',2)[0] + $upid = git show $usha | git patch-id --stable | ForEach-Object { ($_ -split ' ')[0] } + if ($upid -eq $pid) { $_ } + } | Select-Object -First 1 + if ($match) { "$_ matches $match"; break } +} +``` + +Slow but reliable. The first match (newest fork commit with an upstream +twin) gives you the baseline. + +### Method C — ask the human + +If both above fail, ask the user for the baseline SHA. Do **not** guess. +A wrong baseline silently drops upstream commits. + +## Initialize `state.json` + +Once you have ``: + +```pwsh +pwsh .github/skills/upstream-sync/scripts/00-bootstrap.ps1 -BaselineSha +``` + +This script: + +1. Verifies `` exists on `upstream/main`. +2. Writes a fresh `state.json` with the baseline + empty history. +3. Stages and commits `.github/upstream-sync/state.json` on a branch + `chore/upstream-sync-bootstrap`. +4. Tells you to open a PR — do not push state changes straight to main. + +## Verify + +After the bootstrap PR merges, a dry run should report a non-empty +pending list (the commits upstream has made since baseline) without +actually picking anything: + +```pwsh +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -DryRun +``` + +Inspect the latest `reports/*.md` — it should look sane. **Then** enable +the scheduler. diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/conflict-triage.md new file mode 100644 index 000000000..5568823e8 --- /dev/null +++ b/.github/skills/upstream-sync/references/conflict-triage.md @@ -0,0 +1,133 @@ +# Conflict Triage — Resolution Tiers + +When a cherry-pick conflicts, apply tiers **in order**. Stop at the first +tier that fully resolves the conflict. + +## Tier 0 — Known take-{upstream,ours} files + +Some files have a fixed correct resolution that never changes. Examples: + +- `.github/workflows/spelling2.yml` — always take upstream (verified on sister repo `agentic-terminal`). + +The list of these paths lives in [`known-conflicts.md`](./known-conflicts.md). + +**Algorithm:** + +```pwsh +$conflictingPaths = git diff --name-only --diff-filter=U +$tier0List = Get-KnownConflicts # parses known-conflicts.md +foreach ($p in $conflictingPaths) { + $entry = $tier0List | Where-Object { $_.Path -eq $p } + if (-not $entry) { return $false } # Tier 0 doesn't cover this commit + switch ($entry.Strategy) { + 'take-upstream' { git checkout --theirs -- $p; git add -- $p } + 'take-ours' { git checkout --ours -- $p; git add -- $p } + 'union' { git merge-file --union ... } + } +} +git cherry-pick --continue --no-edit +``` + +If `git status` is now clean and the cherry-pick continued, **Tier 0 fully resolved** — record the file(s) auto-resolved and move on. + +## Tier 1 — Empty after staging + +After Tier 0 (or with no conflicts to begin with), if the staged diff is +empty, the commit has already been applied to the fork in some prior +form. Skip it without recording a commit: + +```pwsh +if ((git diff --cached --quiet; $LASTEXITCODE) -eq 0) { + git cherry-pick --skip # equivalent to reset + advance + return @{ status = 'skipped-empty' } +} +``` + +## Tier 2 — LLM-assisted trivial textual (opt-in) + +Disabled by default; enable with `04-run-batch.ps1 -TryTier2`. Even when +enabled, this tier only fires when **all** of the following hold: + +- No more than 3 conflicting files. +- Each file has fewer than 5 conflict hunks. +- Each hunk has fewer than 30 lines on either side. +- No conflicting file is in `src/cascadia/TerminalProtocol/`, + `src/cascadia/WindowsTerminal/TerminalProtocolComServer.cpp`, or + `tools/wta/**` (these are fork-only and shouldn't conflict; if they + somehow do, that's a Tier-3 signal). + +**Delegation:** + +Spawn a fresh sub-agent (Memory Assistant rules require fresh — never +self-review). Prompt template: + +> You are resolving a git cherry-pick conflict mechanically. Below are +> the conflict markers in ``. The fork ("ours") adds AI-agent +> integration; upstream ("theirs") is microsoft/terminal. Produce ONLY +> the resolved file content — no commentary, no markers. If you cannot +> resolve with high confidence (≥0.9), respond with the single token +> `LOW_CONFIDENCE` and nothing else. +> +> Confidence rubric: +> - **High**: changes are non-overlapping in intent (e.g., upstream +> added a new function near our edit; merge order is obvious). +> - **Low**: both sides modified the same logic / same lines / same +> public API — semantic decision needed. + +**Acceptance:** If the agent returns `LOW_CONFIDENCE`, escalate to +Tier 3. If it returns content, **verify with a second fresh agent**: + +> Compare the resolved file against the "ours" version and the "theirs" +> version. Does the resolution preserve all behavioral intent from both +> sides? Respond `OK` or `NOT_OK: `. + +Stage only if both agents agree `high`/`OK`. Otherwise → Tier 3. + +## Tier 3 — Stop and escalate + +Anything not resolved by Tier 0–2: + +```pwsh +git cherry-pick --abort +# Set state.stuck_on_sha = , state.stuck_branch = +# Write the report with the conflict diagnostics +# Open the GitHub issue (07-open-stuck-issue.ps1) +# Exit with code 10 +``` + +The report **must** include: + +- The conflicting commit SHA, subject, author, and upstream URL. +- The list of conflicting paths with a one-line classification each + (`semantic-overlap`, `deleted-by-us`, `binary-merge`, etc.). +- The exact local branch name where the human picks up. +- The exact resume command the human runs after they merge their fix: + ``` + pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha + ``` + +## Line endings + +If any Tier-2 resolution touches a file with CRLF line endings (most +`.csproj`, `.xml`, winget manifests, and many `.yml` files on this repo), +re-normalize before staging: + +```pwsh +# Inside Tier-2, after writing the resolved content: +$bytes = [System.IO.File]::ReadAllBytes($p) +$text = [System.Text.Encoding]::UTF8.GetString($bytes) -replace "`r?`n", "`r`n" +[System.IO.File]::WriteAllText($p, $text, (New-Object System.Text.UTF8Encoding($true))) # BOM +``` + +(Skipping this is how the winget-pkgs submission broke last time — +LF mid-file fails CI even though the rest of the file is CRLF.) + +## What is NOT a conflict for our purposes + +- **Upstream renamed a file we never touched** — git follows the rename + automatically. No conflict. +- **Upstream deleted a file we never touched** — git removes it. No conflict. +- **Upstream modified a file in a fork-only directory** (e.g., upstream + somehow touched `tools/wta/`) — impossible by construction since + upstream doesn't know those files exist. If it ever happens, it's a + Tier-3 signal that the fork-only directory is misnamed. diff --git a/.github/skills/upstream-sync/references/known-conflicts.md b/.github/skills/upstream-sync/references/known-conflicts.md new file mode 100644 index 000000000..b4f576e0c --- /dev/null +++ b/.github/skills/upstream-sync/references/known-conflicts.md @@ -0,0 +1,53 @@ +# Known Conflicts (Tier-0 auto-resolution list) + +This file is the **source of truth** for paths that always conflict the +same way and have a known correct resolution. The Tier-0 step in the +cherry-pick loop parses this file and auto-resolves matching paths. + +## Format + +Each entry is an H2 path heading, a one-line `Strategy:` line, and a +"Why" paragraph. The parser reads the heading text as the literal path +and the `Strategy:` token as one of `take-upstream` | `take-ours` | `union`. + +--- + +## `.github/workflows/spelling2.yml` + +**Strategy:** `take-upstream` + +**Why:** Upstream maintains this workflow and updates it frequently with +new excludes/patterns. The fork has never had a reason to diverge from +upstream's version — every prior conflict on this file was resolved by +taking upstream wholesale. Verified on sister repo `agentic-terminal`. + +--- + +## How to add a new entry + +When the cherry-pick loop hits a Tier-3 conflict on a file that, on +inspection, has the same shape every time (e.g., a generated file, a +workflow upstream owns, a config we copy verbatim), add it here **after** +the manual resolution PR merges. Format: + +```markdown +## `` + +**Strategy:** `take-upstream` + +**Why:** +``` + +Then re-run the next sync — the previously-stuck file should now +auto-resolve under Tier 0. + +## What does NOT belong here + +- Files that conflict but where the correct resolution **depends on the + commit** (those need Tier-2 or human judgement, not a fixed rule). +- Fork-only files (`tools/wta/**`, `src/cascadia/TerminalProtocol/**`) + — these shouldn't conflict at all; if they do, the cause is structural + and a Tier-0 entry would mask it. +- Files where "take upstream" would lose a fork-specific feature. + Re-check by diffing fork-`HEAD` vs upstream for that file before adding. diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md new file mode 100644 index 000000000..c64e777d2 --- /dev/null +++ b/.github/skills/upstream-sync/references/reporting.md @@ -0,0 +1,122 @@ +# Reporting + +Every run writes one markdown report to +`.github/upstream-sync/reports/YYYY-MM-DDTHHmm.md`. The report doubles +as the PR description (success path) and the issue body (stuck path). + +## Report template + +```markdown +# Upstream sync — — + +**Status:** +**Host:** +**Duration:** +**Baseline (before run):** `` () — +**Upstream HEAD:** `` () — + +--- + +## Summary + +- Commits picked: **** +- Revert pairs dropped: **

** (= <2P> commits skipped, net zero) +- Upstream-empty commits skipped:**** +- Tier-0 auto-resolutions: **** (across files) +- Tier-2 LLM resolutions: **** (only when -TryTier2) +- Tier-3 escalation (stuck at): + +## Picked commits (oldest → newest) + +| # | SHA | Subject | Files | Author | +|---|---|---|---|---| +| 1 | | | | | +| ... | | | | | + +## Dropped revert pairs + +| Original SHA | Original subject | Revert SHA | Detected via | +|---|---|---|---| +| | | | "Revert" prefix / "This reverts commit" body | + +## Empty / no-op commits skipped + +| SHA | Subject | Reason | +|---|---|---| +| | | upstream empty / already applied | + +## Tier-0 auto-resolutions + +| Commit SHA | File | Strategy | +|---|---|---| +| | `.github/workflows/spelling2.yml` | take-upstream | + +## (Stuck only) Conflict diagnostics + +**Conflicting commit:** [``](https://github.com/microsoft/terminal/commit/) — +**Author:** +**Files in conflict:** + +| Path | Classification | Notes | +|---|---|---| +| `` | semantic-overlap | both sides changed `` | +| `` | deleted-by-us | upstream modifies a file we removed | + +**Attempted resolutions:** +- Tier 0: +- Tier 1: +- Tier 2: > + +**Pickup branch:** `upstream-sync/` (pushed to origin) + +**How to resume:** + +1. `git switch upstream-sync/` +2. Resolve the conflict in ``. Reference: + - Upstream commit: https://github.com/microsoft/terminal/commit/ + - Fork file at HEAD: `git show HEAD:` +3. `git add ` and `git cherry-pick --continue` +4. Push, open a PR titled `chore(upstream-sync): manual resolution for `, merge it. +5. Run: + ``` + pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha + ``` +6. The next scheduled sync resumes from +1. + +--- + +_Generated by `.github/skills/upstream-sync/scripts/05-write-report.ps1`._ +``` + +## Stuck-issue template + +The issue body is the report itself, plus a short header explaining +urgency: + +```markdown +🛑 **Upstream sync stopped at a conflict that needs human judgement.** + +The scheduler will keep skipping its runs until this issue is resolved +and the stuck-lock is cleared. No alarm — the lock is intentional. + +To unblock: +1. Follow "How to resume" in the report below. +2. Close this issue after `clear-stuck.ps1` runs cleanly. + + +``` + +Label: `upstream-sync-stuck` (apply via `gh issue create --label`). + +## Why always write a report + +- The "ok" report becomes the PR description automatically — reviewers + see what landed and why anything was skipped. +- The "no-op" report proves the scheduler ran (useful for "did it stop + ticking?" debugging) without polluting issue/PR queues. +- The "stuck" report is the issue body — humans don't need to re-derive + what was attempted. +- The "skipped-locked" report shows the lock did its job (no duplicate + destruction). + +Retention: keep all reports indefinitely. They're small markdown. diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md new file mode 100644 index 000000000..18a98baed --- /dev/null +++ b/.github/skills/upstream-sync/references/state-schema.md @@ -0,0 +1,70 @@ +# `state.json` Schema + +Path: `.github/upstream-sync/state.json` (committed on `main`). + +```jsonc +{ + "version": 1, + "upstream_remote_url": "https://github.com/microsoft/terminal.git", + "upstream_branch": "main", + + // The most recent upstream commit that has landed in this fork's main. + // Updated only when a sync PR merges (the PR includes the state update). + "last_synced_upstream_sha": "93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f", + + // Stuck-lock. When non-null, the scheduler exits early without touching + // any branch. Cleared by scripts/clear-stuck.ps1 after a human merges + // the resolution PR. + "stuck_on_sha": null, + "stuck_branch": null, + "stuck_at": null, // ISO 8601 timestamp; null when not stuck + "stuck_issue_url": null, // populated by 07-open-stuck-issue.ps1 + + // Last run summary (for fast inspection without grepping reports). + "last_run": { + "at": "2026-06-04T13:41:45+08:00", + "host": "SH-YEELAM-D11S", + "status": "ok", // "ok" | "no-op" | "stuck" | "skipped-locked" + "branch": "upstream-sync/2026-06-04", + "pr_url": "https://github.com/microsoft/intelligent-terminal/pull/999", + "picked_count": 7, + "dropped_pair_count": 1, + "empty_count": 2, + "tier0_resolutions": 1 + }, + + // Rolling history — keep last 20 runs. + "history": [ + { "at": "...", "status": "ok", "picked_count": 7, "pr_url": "..." }, + { "at": "...", "status": "no-op", "picked_count": 0 }, + { "at": "...", "status": "stuck", "stuck_on_sha": "abc...", "issue_url": "..." } + ] +} +``` + +## Field rules + +- **`last_synced_upstream_sha`** advances **only** when a sync PR is merged. + The orchestrator updates this in the PR commit itself, so it lands + atomically with the picks. Never edit by hand except via + `clear-stuck.ps1`. +- **`stuck_on_sha`** is the gate. When set, `04-run-batch.ps1` exits 0 + without doing anything. This is intentional — the scheduler will keep + ticking but will not clobber the stuck branch. +- **`stuck_branch`** must still exist on `origin` until the human merges + it; `clear-stuck.ps1` does not delete it (the PR merge does). +- **`history`** is for the human reading state.json directly. The reports + in `reports/` are the source of truth. + +## Concurrency + +The scheduler should run on a single host. If multiple hosts run +concurrently, the second one's `git push -u origin upstream-sync/` +will collide on the same-day branch name — `git push` will reject with +non-fast-forward and `04-run-batch.ps1` will exit 20 (hard failure). +This is acceptable: the loser's report is still written locally for +inspection, and no state on `main` has been updated. + +If you genuinely need multi-host scheduling, add a per-host suffix to +the branch name and a state-file mutex via `gh api repos/.../contents/...` +GraphQL check-and-set — out of scope for v1. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md new file mode 100644 index 000000000..7b394e994 --- /dev/null +++ b/.github/skills/upstream-sync/references/workflow.md @@ -0,0 +1,181 @@ +# Upstream Sync — Full Workflow + +This is the authoritative per-step procedure. The orchestrator is +[`scripts/04-run-batch.ps1`](../scripts/04-run-batch.ps1); each step below +maps to a script or an in-orchestrator function. + +## Entry Conditions + +- `state.json` exists (bootstrap done — see [bootstrap.md](./bootstrap.md)). +- Working tree is clean (`git status --porcelain` empty). +- We are on `main` (or the script will `git switch main`). +- `state.stuck_on_sha` is `null` (otherwise exit early — see "Stuck-lock" below). + +## Steps + +### 1. Fetch upstream + +```pwsh +git remote get-url upstream 2>$null || git remote add upstream https://github.com/microsoft/terminal.git +git fetch upstream main --tags=false +``` + +Script: [`01-fetch-upstream.ps1`](../scripts/01-fetch-upstream.ps1). + +Exits with `state.last_run.status = "no-op"` and writes a "nothing to do" +report if `git rev-parse upstream/main` equals `state.last_synced_upstream_sha`. + +### 2. Compute pending range + +```pwsh +git log --reverse --format='%H' "$last_synced..upstream/main" +``` + +Oldest-first ordering is mandatory. Cherry-picking newest-first inverts +dependencies and creates spurious conflicts. + +Script: [`02-compute-pending.ps1`](../scripts/02-compute-pending.ps1) emits +a JSON array on stdout. + +### 3. Detect & drop revert pairs + +A commit is a revert if its **first line** matches `^Revert "..."$` **or** +its body contains `This reverts commit <40-hex>`. + +- If `<40-hex>` is **inside** the pending range AND has not been picked + yet → drop **both** the original and the revert; record the pair. +- If `<40-hex>` is **outside** the pending range (already synced earlier) + → keep the revert; it must land as a normal pick. + +Script: same `02-compute-pending.ps1` (returns `{ pending: [...], dropped_pairs: [[A,B],...] }`). + +### 4. Drop upstream-empty commits + +Before picking, check `git diff-tree --no-commit-id --name-only -r `. +If empty, mark skipped and record. (Cheaper to detect upfront than to +pick and reset.) + +### 5. Create the sync branch + +```pwsh +$branch = "upstream-sync/$(Get-Date -Format 'yyyy-MM-dd')" +git switch -c $branch # or "git switch $branch" if resuming +``` + +If the branch already exists (resume from same-day run), reuse it. + +### 6. Cherry-pick loop + +For each commit in the (now-filtered) pending list: + +```pwsh +git cherry-pick --keep-redundant-commits -x +``` + +- `-x` adds `(cherry picked from commit )` to the message — critical + for audit trail and for the next-run revert-pair detector. +- `--keep-redundant-commits` lets us preserve no-op picks for traceability + (we then `git reset --hard HEAD~1` if Tier-1 fires). + +**On conflict, apply resolution tiers in order:** + +1. **Tier 0 — known take-upstream / take-ours files.** Read + [`known-conflicts.md`](./known-conflicts.md). For every conflicting + path in the Tier-0 list, run `git checkout --theirs ` (or + `--ours`), then `git add `. If **all** conflicting paths are + resolved, run `git cherry-pick --continue` and move on. +2. **Tier 1 — empty after pick.** If `git diff --cached --quiet` returns + zero exit code (no staged changes), the commit was already applied or + is a no-op against fork: `git cherry-pick --skip` and record. +3. **Tier 2 — trivial textual (opt-in via `-TryTier2`).** Delegate to a + fresh sub-agent with the conflict text. Accept only `high` confidence. + See [conflict-triage.md](./conflict-triage.md#tier-2-llm-assisted). +4. **Tier 3 — semantic conflict.** Run `git cherry-pick --abort`. Set + the stuck-lock, write report, exit non-zero. The script that calls + us will then open the stuck issue. + +Script: [`03-cherry-pick-one.ps1`](../scripts/03-cherry-pick-one.ps1) +handles one commit, returns a JSON status object. The orchestrator loops. + +### 7. Write report (always) + +Regardless of outcome (ok / no-op / stuck), write +`.github/upstream-sync/reports/YYYY-MM-DDTHHmm.md` with: + +- Run metadata (start, end, duration, host, status) +- Counts: picked / dropped-pair / empty / known-conflict-resolved / stuck-at +- For each picked commit: SHA, subject, author, files-touched count +- For dropped pairs: the two SHAs and their subjects +- If stuck: the conflicting commit, the conflicting paths, what was attempted, the exact resume command + +Template: [`reporting.md`](./reporting.md). + +Script: [`05-write-report.ps1`](../scripts/05-write-report.ps1). + +### 8a. Success path — push + open PR + +```pwsh +git push -u origin $branch +gh pr create --base main --head "$($me):$branch" --title "chore(upstream): sync up to $shortSha" --body-file $reportPath +``` + +Update `state.last_synced_upstream_sha = upstream/main` and commit +`state.json` + the report into the sync branch (amend the last pick or +add a trailing commit titled `chore(upstream-sync): update state`). + +Script: [`06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1). + +### 8b. Stuck path — open issue + set lock + +```pwsh +gh issue create --label upstream-sync-stuck ` + --title "Upstream sync stuck at : " ` + --body-file $reportPath +``` + +Set `state.stuck_on_sha = ` and `state.stuck_branch = $branch`. +Commit `state.json` and the report on `main` (yes, directly — this is the +lock, and the PR path is blocked). The next scheduled run sees the lock +and exits. + +Script: [`07-open-stuck-issue.ps1`](../scripts/07-open-stuck-issue.ps1). + +## Stuck-Lock + +When `state.stuck_on_sha` is non-null, the orchestrator: + +1. Logs `"stuck-lock set at ; skipping run"`. +2. Writes a `reports/YYYY-MM-DDTHHmm-skipped.md` noting the skip. +3. Exits 0 (the scheduler should not retry on the same lock). + +To clear the lock after the human has merged a PR resolving the stuck +commit: + +```pwsh +pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha +``` + +This sets `state.last_synced_upstream_sha = `, clears `stuck_on_sha` +and `stuck_branch`, and commits `state.json` on `main`. + +## Sub-Agent Delegation Map + +| Step | Delegate to fresh sub-agent? | Why | +|---|---|---| +| 1–2 (fetch, compute) | No | Pure git plumbing, deterministic. | +| 3 (revert-pair detection) | No | Mechanical; the script does it. | +| 6 / Tier-2 (LLM-assisted textual resolution) | **Yes — required** | Implementer bias risk; require `high` confidence and a different agent to verify before staging. | +| 7 (write report) | No | Template fill. | +| 8a (PR body polish) | Optional | If picked > 20 commits, a sub-agent can group them by area for the PR body. | +| 8b (issue summary) | **Yes** | A fresh agent writes a clearer "what's hard about this conflict" summary than the loop that aborted. | + +## Exit Codes (from `04-run-batch.ps1`) + +| Code | Meaning | +|---|---| +| 0 | Success (PR opened) **or** no-op **or** skipped because lock is set | +| 10 | Stuck — issue opened, lock set (this is **not** an error; scheduler should not alarm) | +| 20 | Hard failure (git command failed unexpectedly, network down, gh auth missing) — scheduler **should** alarm | + +Wrap the scheduler invocation accordingly: treat 0 and 10 as healthy, +20 as paging-worthy. diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 new file mode 100644 index 000000000..8872c0417 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -0,0 +1,70 @@ +<# +.SYNOPSIS + One-time bootstrap: initialize state.json with the baseline upstream SHA. + +.DESCRIPTION + Use this exactly once when the skill is first installed in the repo. + See references/bootstrap.md for how to discover the right baseline SHA. + +.PARAMETER BaselineSha + The upstream/microsoft/terminal commit SHA the fork is currently + "caught up to". Must be reachable from upstream/main. + +.PARAMETER Force + Overwrite an existing state.json. Refuses by default to prevent + accidentally rewinding the baseline. + +.EXAMPLE + pwsh scripts/00-bootstrap.ps1 -BaselineSha 93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $BaselineSha, + [switch] $Force +) + +. "$PSScriptRoot/Common.ps1" + +Ensure-UpstreamRemote +git fetch upstream main --no-tags | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } + +# Verify the SHA exists on upstream/main. +$null = git merge-base --is-ancestor $BaselineSha upstream/main +if ($LASTEXITCODE -ne 0) { + throw "Baseline SHA $BaselineSha is not an ancestor of upstream/main. Refusing to write state.json." +} + +$statePath = Get-StatePath +if ((Test-Path $statePath) -and -not $Force) { + throw "state.json already exists at $statePath. Pass -Force to overwrite (rewinding the baseline can cause re-picks)." +} + +$state = @{ + version = 1 + upstream_remote_url = 'https://github.com/microsoft/terminal.git' + upstream_branch = 'main' + last_synced_upstream_sha = $BaselineSha + stuck_on_sha = $null + stuck_branch = $null + stuck_at = $null + stuck_issue_url = $null + last_run = $null + history = @() +} +Write-State $state + +# Stage and commit on a dedicated branch so the human can open the PR. +$branch = 'chore/upstream-sync-bootstrap' +git switch -c $branch 2>$null +if ($LASTEXITCODE -ne 0) { git switch $branch | Out-Null } + +git add -- (Get-StatePath) +if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed." } + +git commit -m "chore(upstream-sync): bootstrap baseline at $($BaselineSha.Substring(0,9))" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit failed; bootstrap aborted." } + +Write-Host "" +Write-Host "Bootstrap committed on branch '$branch'." -ForegroundColor Green +Write-Host "Next: git push -u origin $branch && gh pr create" diff --git a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 new file mode 100644 index 000000000..48e185650 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 @@ -0,0 +1,18 @@ +<# +.SYNOPSIS + Ensure upstream remote exists and fetch upstream/main. +.OUTPUTS + Writes the current upstream/main SHA to stdout. +#> +[CmdletBinding()] +param() + +. "$PSScriptRoot/Common.ps1" + +Ensure-UpstreamRemote +git fetch upstream main --no-tags 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } + +$sha = git rev-parse upstream/main +if ($LASTEXITCODE -ne 0) { throw "git rev-parse upstream/main failed." } +return $sha.Trim() diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 new file mode 100644 index 000000000..2d853f25c --- /dev/null +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -0,0 +1,102 @@ +<# +.SYNOPSIS + Compute the pending cherry-pick list with revert-pair detection. + +.DESCRIPTION + Reads state.last_synced_upstream_sha, lists commits in + state.last_synced..upstream/main (oldest first), detects revert pairs + within the range and drops them, detects upstream-empty commits and + drops them, and emits the final pending list as JSON. + +.OUTPUTS + JSON object on stdout: + { + "from": "", + "to": "", + "pending": [ "", ... ], # in pick order + "dropped_pairs": [ ["", ""], ... ], + "skipped_empty": [ "", ... ] + } +#> +[CmdletBinding()] +param() + +. "$PSScriptRoot/Common.ps1" + +$state = Read-State +$from = [string]$state.last_synced_upstream_sha +if (-not $from) { throw "state.last_synced_upstream_sha is empty. Run bootstrap." } + +$to = (git rev-parse upstream/main).Trim() +if ($LASTEXITCODE -ne 0) { throw "git rev-parse upstream/main failed." } + +if ($from -eq $to) { + @{ from = $from; to = $to; pending = @(); dropped_pairs = @(); skipped_empty = @() } | ConvertTo-Json -Depth 5 + return +} + +# Oldest-first list of full SHAs. +$all = git log --reverse --format='%H' "$from..$to" +if ($LASTEXITCODE -ne 0) { throw "git log failed." } +$all = @($all | Where-Object { $_ }) + +# Build sha -> first line and body map (single git invocation per commit is fine for typical batch sizes). +$info = @{} +foreach ($sha in $all) { + $subj = git log -1 --format='%s' $sha + $body = git log -1 --format='%B' $sha + $info[$sha] = @{ subject = $subj; body = $body } +} + +# Detect revert pairs. Note: when a revert body lists multiple SHAs +# (e.g. "This reverts commit A. This also undoes parts of B"), the first +# match wins — that is, the SHA following the canonical "This reverts +# commit " line introduced by `git revert`. Bodies that list a +# secondary SHA outside the canonical form are ignored on purpose. +$dropped = New-Object 'System.Collections.Generic.HashSet[string]' +$pairs = @() +foreach ($sha in $all) { + if ($dropped.Contains($sha)) { continue } + $body = $info[$sha].body + $subj = $info[$sha].subject + + $targetSha = $null + if ($body -match 'This reverts commit ([0-9a-f]{40})\b') { + $targetSha = $Matches[1] + } elseif ($subj -match '^Revert "') { + # Best-effort: try to find the original by matching the quoted subject. + $origSubject = $subj -replace '^Revert "', '' -replace '"\s*$', '' -replace '"\.?\s*$','' + $candidate = $all | Where-Object { + $info[$_].subject -eq $origSubject -and -not $dropped.Contains($_) + } | Select-Object -First 1 + if ($candidate) { $targetSha = $candidate } + } + + if ($targetSha -and $info.ContainsKey($targetSha) -and -not $dropped.Contains($targetSha)) { + [void] $dropped.Add($sha) + [void] $dropped.Add($targetSha) + $pairs += ,@($targetSha, $sha) + } +} + +# Detect upstream-empty (no files touched). +$empty = @() +foreach ($sha in $all) { + if ($dropped.Contains($sha)) { continue } + $files = git diff-tree --no-commit-id --name-only -r $sha + if (-not $files) { + $empty += $sha + [void] $dropped.Add($sha) + } +} + +$pending = $all | Where-Object { -not $dropped.Contains($_) } + +$result = [ordered] @{ + from = $from + to = $to + pending = @($pending) + dropped_pairs = @($pairs) + skipped_empty = @($empty) +} +$result | ConvertTo-Json -Depth 5 diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 new file mode 100644 index 000000000..40fe65f18 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -0,0 +1,158 @@ +<# +.SYNOPSIS + Cherry-pick one upstream commit with Tier-0/Tier-1 auto-resolution. + +.DESCRIPTION + Runs `git cherry-pick -x `. On conflict, attempts Tier-0 (known + take-{upstream,ours} files from known-conflicts.md), then Tier-1 (empty + after staging → skip). Anything else returns 'stuck' and leaves the + cherry-pick aborted. + +.PARAMETER Sha + The upstream commit to pick. + +.OUTPUTS + JSON status object on stdout: + { "sha": "...", "status": "picked|skipped-empty|stuck", + "tier0_paths": ["..."], "conflict_paths": ["..."] } +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $Sha +) + +. "$PSScriptRoot/Common.ps1" + +function Get-KnownConflicts { + $md = Join-Path (Split-Path $PSScriptRoot -Parent) 'references/known-conflicts.md' + if (-not (Test-Path $md)) { return @() } + $lines = Get-Content -LiteralPath $md + $entries = @() + $current = $null + foreach ($l in $lines) { + if ($l -match '^##\s+`([^`]+)`\s*$') { + if ($current) { $entries += $current } + $current = @{ Path = $Matches[1]; Strategy = $null } + } elseif ($current -and $l -match '^\*\*Strategy:\*\*\s+`(take-upstream|take-ours|union)`') { + $current.Strategy = $Matches[1] + } + } + if ($current) { $entries += $current } + return $entries | Where-Object { $_.Strategy } +} + +function Get-ConflictPaths { + # core.quotepath=off keeps non-ASCII paths in raw UTF-8 so Tier-0 + # path matching against known-conflicts.md works without C-quoting. + $u = git -c core.quotepath=off diff --name-only --diff-filter=U + if (-not $u) { return @() } + return @($u -split "`n" | Where-Object { $_ }) +} + +$result = [ordered] @{ + sha = $Sha + status = 'unknown' + tier0_paths = @() + conflict_paths = @() +} + +# Capture upstream's author AND committer identity + dates so the +# resulting commit is per-commit identical (modulo SHA + GPG signature) +# to the upstream original. cherry-pick preserves author by default; +# we pin both sides via env for symmetry and clarity. Each iteration of +# the pick loop runs this lookup against the specific upstream SHA, so +# every commit gets its own original dates — never a single "run time" +# fixed timestamp. +# +# Note: this does NOT reproduce GPG signatures (we don't hold upstream's +# keys). The fork doesn't enforce signed commits, so "committed by X but +# unsigned" is acceptable. +$info = (git log -1 --format='%an%x09%ae%x09%aI%x09%cn%x09%ce%x09%cI' $Sha) -split "`t" +$env:GIT_AUTHOR_NAME = $info[0] +$env:GIT_AUTHOR_EMAIL = $info[1] +$env:GIT_AUTHOR_DATE = $info[2] +$env:GIT_COMMITTER_NAME = $info[3] +$env:GIT_COMMITTER_EMAIL = $info[4] +$env:GIT_COMMITTER_DATE = $info[5] + +try { + +# Attempt the pick. +git cherry-pick --keep-redundant-commits -x $Sha 2>&1 | Out-Host +$pickCode = $LASTEXITCODE + +if ($pickCode -eq 0) { + # Defensive: ensure HEAD is the commit we just picked before any reset. + # If a future hook ever inserts work between pick and check, HEAD~1 + # would silently discard arbitrary commits. + $headTree = (git log -1 --format='%T' HEAD).Trim() + $pickedTree = (git log -1 --format='%T' $Sha).Trim() + # Tier-1 check: did we just create an empty commit (allowed by --keep-redundant-commits)? + $changed = git diff-tree --no-commit-id --name-only -r HEAD + if (-not $changed) { + if ($headTree -ne $pickedTree) { + throw "Refusing to reset --hard HEAD~1: HEAD tree ($headTree) does not match picked commit's tree ($pickedTree). Investigate before retrying." + } + git reset --hard HEAD~1 | Out-Null + $result.status = 'skipped-empty' + } else { + $result.status = 'picked' + } + $result | ConvertTo-Json -Compress + return +} + +# Conflict. Try Tier-0. +$conflicts = Get-ConflictPaths +$result.conflict_paths = $conflicts +$known = Get-KnownConflicts +$unhandled = @() +foreach ($p in $conflicts) { + $e = $known | Where-Object { $_.Path -eq $p } | Select-Object -First 1 + if (-not $e) { $unhandled += $p; continue } + switch ($e.Strategy) { + 'take-upstream' { git checkout --theirs -- $p | Out-Null } + 'take-ours' { git checkout --ours -- $p | Out-Null } + 'union' { Write-Warning "union strategy not implemented yet for $p"; $unhandled += $p; continue } + } + git add -- $p | Out-Null + $result.tier0_paths += $p +} + +if ($unhandled.Count -gt 0) { + git cherry-pick --abort | Out-Null + $result.status = 'stuck' + $result.conflict_paths = $unhandled + $result | ConvertTo-Json -Compress + return +} + +# All conflicts handled by Tier-0; continue the pick (preserve original message). +$env:GIT_EDITOR = 'true' +git cherry-pick --continue 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { + # Could still be empty after Tier-0. + $staged = git diff --cached --name-only + if (-not $staged) { + git cherry-pick --skip | Out-Null + $result.status = 'skipped-empty' + $result | ConvertTo-Json -Compress + return + } + git cherry-pick --abort | Out-Null + $result.status = 'stuck' + $result | ConvertTo-Json -Compress + return +} + +$result.status = 'picked' +$result | ConvertTo-Json -Compress + +} finally { + Remove-Item Env:GIT_AUTHOR_NAME -ErrorAction SilentlyContinue + Remove-Item Env:GIT_AUTHOR_EMAIL -ErrorAction SilentlyContinue + Remove-Item Env:GIT_AUTHOR_DATE -ErrorAction SilentlyContinue + Remove-Item Env:GIT_COMMITTER_NAME -ErrorAction SilentlyContinue + Remove-Item Env:GIT_COMMITTER_EMAIL -ErrorAction SilentlyContinue + Remove-Item Env:GIT_COMMITTER_DATE -ErrorAction SilentlyContinue +} diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 new file mode 100644 index 000000000..fc30339ed --- /dev/null +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -0,0 +1,190 @@ +<# +.SYNOPSIS + Orchestrator: run one upstream-sync pass. Safe to invoke from a + scheduler on a weekly/daily cadence. + +.DESCRIPTION + Reads state.json. If the stuck-lock is set, writes a skipped-locked + report and exits 0. Otherwise: + 1. Fetches upstream/main. + 2. Computes pending commits, dropping revert pairs and empties. + 3. Creates branch upstream-sync/YYYY-MM-DD. + 4. Cherry-picks one-by-one with Tier-0/Tier-1 auto-resolution. + 5. Writes a report. + 6. On success → pushes branch, opens PR (exit 0). + On stuck → pushes branch, opens issue, sets lock (exit 10). + On no-op → exits 0 with a "no-op" report. + +.PARAMETER DryRun + Compute & report only; do not create the branch or pick anything. + +.PARAMETER TryTier2 + Reserved: enable LLM-assisted Tier-2 conflict resolution (NOT YET IMPLEMENTED). + +.PARAMETER Force + Override the stuck-lock. DANGEROUS — clobbers the in-progress branch. + Use only when you know the lock is stale. + +.PARAMETER MaxPicks + Cap the number of cherry-picks per run (default: unlimited). Useful for + smoke-testing the scheduler with a few commits at a time. + +.PARAMETER PushDirectToMain + Skip the PR and fast-forward main directly to the sync branch tip. + Requires push permission on main (admin / branch-protection bypass). + Preserves per-commit content, order, and original author dates — strictly + better than a squash-merged PR. Use when there's no need for a human + review checkpoint per sync. + +.PARAMETER AutoMergeStrategy + PR mode only. After opening the PR, run `gh pr merge -- --auto` + so when CI/approvals pass, GitHub auto-merges with the right strategy. + Allowed: 'rebase' (preserves per-commit, recommended), 'merge' (adds a + merge commit; per-commit also preserved), or 'none' (default; human + picks the strategy manually — but they must NOT pick squash). + +.OUTPUTS + Writes status to stdout. Exit codes: + 0 = success (PR opened) OR no-op OR skipped-locked + 10 = stuck (issue opened, lock set) — NOT an error + 20 = hard failure (git/gh broken) — alarm-worthy +#> +[CmdletBinding()] +param( + [switch] $DryRun, + [switch] $TryTier2, + [switch] $Force, + [int] $MaxPicks = 0, + [switch] $PushDirectToMain, + [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' +) + +. "$PSScriptRoot/Common.ps1" + +function Exit-Hard([string] $msg) { + Write-Error $msg + exit 20 +} + +try { + $state = Read-State + $ctx = New-RunContext + + # --- Stuck-lock gate --- + if ($state.stuck_on_sha -and -not $Force) { + Write-Host "Stuck-lock set at $($state.stuck_on_sha) (issue: $($state.stuck_issue_url)). Skipping." -ForegroundColor Yellow + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $state.last_synced_upstream_sha -To $state.last_synced_upstream_sha -Status 'skipped-locked' + Write-Host "Skip report: $reportPath" + exit 0 + } + + Assert-CleanWorktree + git switch main 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } + git pull --ff-only 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only main failed." } + + # --- 1. Fetch upstream --- + $toSha = (& "$PSScriptRoot/01-fetch-upstream.ps1").Trim() + $fromSha = $state.last_synced_upstream_sha + + if ($toSha -eq $fromSha) { + Write-Host "Already at upstream HEAD ($toSha). No-op." -ForegroundColor Green + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' + Write-Host "No-op report: $reportPath" + exit 0 + } + + # --- 2. Compute pending --- + $pendingJson = & "$PSScriptRoot/02-compute-pending.ps1" + $pending = $pendingJson | ConvertFrom-Json + Write-Host ("Pending: {0} commits, {1} revert pairs dropped, {2} empties dropped." -f $pending.pending.Count, $pending.dropped_pairs.Count, $pending.skipped_empty.Count) + + $ctx.DroppedPairs = @($pending.dropped_pairs) + $ctx.SkippedEmpty = @($pending.skipped_empty) + + if ($pending.pending.Count -eq 0) { + Write-Host "Nothing to pick after filtering. Effective no-op." -ForegroundColor Green + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' + Write-Host "Report: $reportPath" + exit 0 + } + + if ($DryRun) { + Write-Host "DryRun: skipping branch creation and cherry-picks." -ForegroundColor Cyan + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' + Write-Host "DryRun report: $reportPath" + exit 0 + } + + # --- 3. Create / switch to sync branch --- + $branch = $ctx.Branch + git switch -c $branch 2>$null + if ($LASTEXITCODE -ne 0) { + git switch $branch 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not create or switch to $branch." } + } + + # --- 4. Cherry-pick loop --- + $picks = $pending.pending + if ($MaxPicks -gt 0 -and $picks.Count -gt $MaxPicks) { $picks = $picks[0..($MaxPicks-1)] } + + foreach ($sha in $picks) { + Write-Host "" + Write-Host "=== Cherry-pick $sha ===" -ForegroundColor Cyan + $resJson = & "$PSScriptRoot/03-cherry-pick-one.ps1" -Sha $sha + $res = $resJson | ConvertFrom-Json + switch ($res.status) { + 'picked' { + $ctx.Picked += $sha + foreach ($p in @($res.tier0_paths)) { + $ctx.Tier0 += [pscustomobject] @{ Sha = $sha; Path = $p } + } + } + 'skipped-empty' { + $ctx.SkippedEmpty += $sha + } + 'stuck' { + $ctx.StuckSha = $sha + $ctx.StuckPaths = @($res.conflict_paths) + $ctx.Status = 'stuck' + Write-Warning "Stuck at $sha on paths: $($res.conflict_paths -join ', ')" + break + } + default { Exit-Hard "Unknown cherry-pick-one status: $($res.status)" } + } + if ($ctx.Status -eq 'stuck') { break } + } + + # --- 5. Report + finalize --- + if ($ctx.Status -eq 'stuck') { + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'stuck' + $ctx.ReportPath = $reportPath + Write-Host "Stuck report: $reportPath" + $issueUrl = & "$PSScriptRoot/07-open-stuck-issue.ps1" -Ctx $ctx -ReportPath $reportPath + Write-Host "Stuck issue: $issueUrl" -ForegroundColor Yellow + exit 10 + } + + $ctx.Status = 'ok' + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'ok' + $ctx.ReportPath = $reportPath + Write-Host "Report: $reportPath" + + if ($PushDirectToMain) { + $mainHead = & "$PSScriptRoot/06b-finalize-direct.ps1" -Ctx $ctx -To $toSha -ReportPath $reportPath + Write-Host "" + Write-Host "✅ Sync fast-forwarded onto main at $($mainHead.Substring(0,9))" -ForegroundColor Green + exit 0 + } + + $prUrl = & "$PSScriptRoot/06-finalize-pr.ps1" -Ctx $ctx -To $toSha -ReportPath $reportPath -AutoMergeStrategy $AutoMergeStrategy + Write-Host "" + Write-Host "✅ Sync PR opened: $prUrl" -ForegroundColor Green + exit 0 +} +catch { + Write-Error $_.Exception.Message + Write-Error $_.ScriptStackTrace + exit 20 +} diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 new file mode 100644 index 000000000..4f75ab65e --- /dev/null +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -0,0 +1,155 @@ +<# +.SYNOPSIS + Generate a sync run report markdown file. + +.PARAMETER Ctx + The run-context hashtable built by 04-run-batch.ps1. + +.PARAMETER From + Baseline upstream SHA before the run. + +.PARAMETER To + Upstream HEAD SHA at fetch time. + +.PARAMETER Status + ok | no-op | stuck | skipped-locked + +.OUTPUTS + Absolute path to the written report file. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] $Ctx, + [Parameter(Mandatory)] [string] $From, + [Parameter(Mandatory)] [string] $To, + [Parameter(Mandatory)] [ValidateSet('ok','no-op','stuck','skipped-locked')] [string] $Status +) + +. "$PSScriptRoot/Common.ps1" + +$started = $Ctx.StartedAt +$ended = Get-Date +$dur = $ended - $started +$durStr = "{0}m {1}s" -f [int]$dur.TotalMinutes, ($dur.Seconds) + +function Get-Subj([string] $sha) { + if (-not $sha) { return '' } + try { return (git log -1 --format='%s' $sha 2>$null) } catch { return '' } +} + +$fromSubj = Get-Subj $From +$toSubj = Get-Subj $To + +$lines = New-Object System.Collections.Generic.List[string] +$lines.Add("# Upstream sync — $Status — $(Format-Iso8601 $started)") +$lines.Add("") +$lines.Add("**Status:** $Status ") +$lines.Add("**Host:** $($Ctx.Host) ") +$lines.Add("**Duration:** $durStr ") +$lines.Add("**Baseline (before run):** ``$From`` — $fromSubj ") +$lines.Add("**Upstream HEAD:** ``$To`` — $toSubj ") +$lines.Add("**Branch:** ``$($Ctx.Branch)`` ") +$lines.Add("") +$lines.Add("## Summary") +$lines.Add("") +$lines.Add("- Commits picked: **$($Ctx.Picked.Count)**") +$lines.Add("- Revert pairs dropped: **$($Ctx.DroppedPairs.Count)** (= $($Ctx.DroppedPairs.Count * 2) commits skipped, net zero)") +$lines.Add("- Upstream-empty commits skipped: **$($Ctx.SkippedEmpty.Count)**") +$lines.Add("- Tier-0 auto-resolutions: **$($Ctx.Tier0.Count)**") +$lines.Add("- Tier-2 LLM resolutions: **$($Ctx.Tier2.Count)**") +if ($Ctx.StuckSha) { + $lines.Add("- Tier-3 stuck at: ``$($Ctx.StuckSha)``") +} +$lines.Add("") + +if ($Ctx.Picked.Count -gt 0) { + $lines.Add("## Picked commits (oldest → newest)") + $lines.Add("") + $lines.Add("| # | SHA | Subject | Author |") + $lines.Add("|---|---|---|---|") + $i = 0 + foreach ($sha in $Ctx.Picked) { + $i++ + $s = (git log -1 --format='%s' $sha) -replace '\|','\|' + $a = git log -1 --format='%an' $sha + $lines.Add("| $i | ``$($sha.Substring(0,9))`` | $s | $a |") + } + $lines.Add("") +} + +if ($Ctx.DroppedPairs.Count -gt 0) { + $lines.Add("## Dropped revert pairs") + $lines.Add("") + $lines.Add("| Original SHA | Original subject | Revert SHA |") + $lines.Add("|---|---|---|") + foreach ($pair in $Ctx.DroppedPairs) { + $os = (git log -1 --format='%s' $pair[0]) -replace '\|','\|' + $lines.Add("| ``$($pair[0].Substring(0,9))`` | $os | ``$($pair[1].Substring(0,9))`` |") + } + $lines.Add("") +} + +if ($Ctx.SkippedEmpty.Count -gt 0) { + $lines.Add("## Empty / no-op commits skipped") + $lines.Add("") + $lines.Add("| SHA | Subject |") + $lines.Add("|---|---|") + foreach ($sha in $Ctx.SkippedEmpty) { + $s = (git log -1 --format='%s' $sha) -replace '\|','\|' + $lines.Add("| ``$($sha.Substring(0,9))`` | $s |") + } + $lines.Add("") +} + +if ($Ctx.Tier0.Count -gt 0) { + $lines.Add("## Tier-0 auto-resolutions") + $lines.Add("") + $lines.Add("| Commit SHA | File |") + $lines.Add("|---|---|") + foreach ($r in $Ctx.Tier0) { + $lines.Add("| ``$($r.Sha.Substring(0,9))`` | ``$($r.Path)`` |") + } + $lines.Add("") +} + +if ($Status -eq 'stuck' -and $Ctx.StuckSha) { + $stuckSubj = Get-Subj $Ctx.StuckSha + $stuckAuthor = git log -1 --format='%an <%ae>' $Ctx.StuckSha + $lines.Add("## Conflict diagnostics") + $lines.Add("") + $lines.Add("**Conflicting commit:** [`$($Ctx.StuckSha)`](https://github.com/microsoft/terminal/commit/$($Ctx.StuckSha)) — $stuckSubj ") + $lines.Add("**Author:** $stuckAuthor") + $lines.Add("") + $lines.Add("**Files in conflict:**") + $lines.Add("") + foreach ($p in $Ctx.StuckPaths) { $lines.Add("- ``$p``") } + $lines.Add("") + $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") + $lines.Add("") + $lines.Add("**How to resume:**") + $lines.Add("") + $lines.Add("1. ``git switch $($Ctx.Branch)``") + $lines.Add("2. Manually cherry-pick the stuck commit and resolve:") + $lines.Add(" ``````") + $lines.Add(" git cherry-pick -x $($Ctx.StuckSha)") + $lines.Add(" # resolve conflicts, then:") + $lines.Add(" git add -A && git cherry-pick --continue") + $lines.Add(" ``````") + $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual resolution for $($Ctx.StuckSha.Substring(0,9))``, merge it.") + $lines.Add("4. Clear the lock:") + $lines.Add(" ``````") + $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha $($Ctx.StuckSha)") + $lines.Add(" ``````") + $lines.Add("5. The next scheduled sync resumes from the commit after this one.") + $lines.Add("") +} + +$lines.Add("---") +$lines.Add("") +$lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``._") + +$suffix = if ($Status -eq 'skipped-locked') { 'skipped' } elseif ($Status -eq 'stuck') { 'stuck' } elseif ($Status -eq 'no-op') { 'noop' } else { '' } +$name = Format-ReportFilename -When $started -Suffix $suffix +$path = Join-Path (Get-ReportsDir) $name +[System.IO.File]::WriteAllText($path, ($lines -join "`n"), (New-Object System.Text.UTF8Encoding($false))) +return $path diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 new file mode 100644 index 000000000..8167a0b03 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -0,0 +1,125 @@ +<# +.SYNOPSIS + Push the sync branch and open a PR. Commits state.json + report onto + the branch first so the merge atomically advances last_synced. + +.PARAMETER Ctx + Run context from 04-run-batch.ps1. + +.PARAMETER To + Upstream HEAD SHA at fetch time (becomes new last_synced_upstream_sha). + +.PARAMETER ReportPath + Absolute path to the report markdown to use as the PR body. + +.OUTPUTS + PR URL on stdout (and writes Ctx.PrUrl). +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] $Ctx, + [Parameter(Mandatory)] [string] $To, + [Parameter(Mandatory)] [string] $ReportPath, + [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' +) + +. "$PSScriptRoot/Common.ps1" + +# Prepend the squash-warning banner to the report so it lands as the +# first thing reviewers see in the PR body. The same banner is also +# rendered into the report file in scripts/05-write-report.ps1 — keep +# both consistent. +$banner = @" +> ⚠️ **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked +> upstream commit into one, destroying per-commit attribution, original +> author dates, and ``git bisect`` resolution. Merge with **"Rebase and +> merge"** (preferred — flat history, all $($Ctx.Picked.Count) commits land +> individually) or **"Create a merge commit"** (also preserves per-commit +> content). + +--- + +"@ +$bodyPath = New-TemporaryFile +$bodyContent = $banner + (Get-Content -Raw -LiteralPath $ReportPath) +[System.IO.File]::WriteAllText($bodyPath, $bodyContent, (New-Object System.Text.UTF8Encoding($false))) + +$branch = $Ctx.Branch +$shortTo = $To.Substring(0,9) + +# Update state.json with new baseline and run summary, plus commit the report. +$state = Read-State +$state.last_synced_upstream_sha = $To +$runSummary = [ordered] @{ + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host + status = 'ok' + branch = $branch + pr_url = $null # filled in after PR creation + picked_count = $Ctx.Picked.Count + dropped_pair_count = $Ctx.DroppedPairs.Count + empty_count = $Ctx.SkippedEmpty.Count + tier0_resolutions = $Ctx.Tier0.Count +} +$state.last_run = $runSummary +$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 +Write-State $state + +git add -- (Get-StatePath) $ReportPath +if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } +git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting without push so baseline is not lost." } + +git push -u origin $branch | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git push failed for $branch." } + +$title = "chore(upstream): sync microsoft/terminal up to $shortTo" + +# Same-repo PR: branch was pushed to origin (= microsoft/intelligent-terminal), +# so --head takes the bare branch name. `--head OWNER:BRANCH` would tell gh to +# look on a fork owned by OWNER, which is wrong for this scheduler. +# +# Retry once after a short delay: `gh pr create` on Windows occasionally fails +# with "Head sha can't be blank" right after a push (see SKILL.md gotcha). +$prUrl = $null +for ($attempt = 1; $attempt -le 3; $attempt++) { + $prUrl = gh pr create --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 + if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } + Write-Warning "gh pr create attempt $attempt failed: $prUrl" + Start-Sleep -Seconds 5 +} +if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { + throw "gh pr create did not return a PR URL after 3 attempts. Last output: $prUrl" +} + +$Ctx.PrUrl = $prUrl.Trim() + +Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue + +# Optional: arm GitHub auto-merge with the strategy that preserves per-commit +# history. 'rebase' is the recommended default when auto-merge is enabled — +# it lands all N commits flatly on main once CI + approvals pass. +if ($AutoMergeStrategy -ne 'none') { + $strategyFlag = "--$AutoMergeStrategy" + gh pr merge $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host + if ($LASTEXITCODE -ne 0) { + Write-Warning "gh pr merge --auto failed. PR is open at $($Ctx.PrUrl); merge manually with '$AutoMergeStrategy' strategy (NOT squash)." + } else { + Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green + } +} + +# Backfill PR URL into state.last_run on the branch (best-effort follow-up +# commit). If this push fails the PR is still open and the baseline is still +# advanced on the branch — the only loss is the pr_url field in state.history, +# which is recoverable from the PR itself. +$state.last_run.pr_url = $Ctx.PrUrl +Write-State $state +git add -- (Get-StatePath) | Out-Null +git commit -m "chore(upstream-sync): record PR url" | Out-Host +if ($LASTEXITCODE -eq 0) { + git push origin $branch | Out-Host + if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push pr_url backfill; PR is still open at $($Ctx.PrUrl)." } +} + +return $Ctx.PrUrl diff --git a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 new file mode 100644 index 000000000..73a48eabc --- /dev/null +++ b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 @@ -0,0 +1,108 @@ +<# +.SYNOPSIS + Finalize the sync by fast-forwarding main directly to the sync branch. + Use when you have admin/bypass perms on main and want zero PR latency. + +.DESCRIPTION + Preserves per-commit content, order, and original author dates. + Committer date becomes "now" (git default for cherry-pick) — that's + the semantically correct "when this fork landed it" timestamp. + + Assumes the caller is currently on the sync branch with all picks + applied. Performs: + 1. Add state.json + report, commit on the sync branch. + 2. Switch to main, pull --ff-only. + 3. Fast-forward main to the sync branch tip (refuses if main diverged). + 4. Push main. + 5. Delete the local sync branch (remote was never pushed). + +.PARAMETER Ctx + Run context. + +.PARAMETER To + Upstream HEAD SHA at fetch time (becomes new last_synced_upstream_sha). + +.PARAMETER ReportPath + Path to the markdown report (will be committed onto the sync branch). + +.OUTPUTS + Writes the new main HEAD SHA to stdout and to Ctx.MainHeadSha. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] $Ctx, + [Parameter(Mandatory)] [string] $To, + [Parameter(Mandatory)] [string] $ReportPath +) + +. "$PSScriptRoot/Common.ps1" + +$branch = $Ctx.Branch +$shortTo = $To.Substring(0,9) + +# 1. Update state.json + commit (still on the sync branch). +$state = Read-State +$state.last_synced_upstream_sha = $To +$runSummary = [ordered] @{ + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host + status = 'ok' + branch = $branch + merge_mode = 'direct-push' + pr_url = $null + main_head_sha = $null # filled in after push + picked_count = $Ctx.Picked.Count + dropped_pair_count = $Ctx.DroppedPairs.Count + empty_count = $Ctx.SkippedEmpty.Count + tier0_resolutions = $Ctx.Tier0.Count +} +$state.last_run = $runSummary +$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 +Write-State $state + +git add -- (Get-StatePath) $ReportPath +if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } +git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting before touching main." } + +$syncTip = (git rev-parse HEAD).Trim() + +# 2. Switch to main and ensure we're current. +git switch main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git switch main failed." } +git pull --ff-only origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "main is not fast-forwardable from origin. Resolve manually." } + +# 3. Fast-forward main to the sync tip. If main has moved (someone landed +# something concurrently), this fails — and that's the correct behaviour: +# direct-push assumes you own the merge moment. Re-run the sync to rebase. +git merge --ff-only $syncTip | Out-Host +if ($LASTEXITCODE -ne 0) { + throw "Cannot fast-forward main onto $syncTip — main has commits the sync branch does not. Re-run the sync to start from the new main, or finalize via the PR path." +} + +# 4. Push main. +git push origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git push origin main failed. The picks are local-only — push manually before the next scheduler tick or it will re-pick everything." } + +# 5. Local cleanup. Remote branch was never pushed in direct-push mode. +git branch -D $branch | Out-Host + +$mainHead = (git rev-parse HEAD).Trim() +$Ctx.MainHeadSha = $mainHead + +# Backfill main_head_sha into state.last_run (best-effort). +$state = Read-State +if ($state.last_run -and $state.history -and $state.history.Count -gt 0) { + $state.last_run.main_head_sha = $mainHead + $state.history[0].main_head_sha = $mainHead + Write-State $state + git add -- (Get-StatePath) | Out-Null + git commit -m "chore(upstream-sync): record main head $($mainHead.Substring(0,9))" | Out-Host + if ($LASTEXITCODE -eq 0) { + git push origin main | Out-Host + if ($LASTEXITCODE -ne 0) { Write-Warning "Backfill push failed (cosmetic only; sync content already on main)." } + } +} + +return $mainHead diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 new file mode 100644 index 000000000..5518db754 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -0,0 +1,97 @@ +<# +.SYNOPSIS + Set the stuck-lock and open a GitHub issue. Pushes the stuck branch + to origin so the human can pick it up. + +.PARAMETER Ctx + Run context (must have StuckSha, StuckPaths, Branch set). + +.PARAMETER ReportPath + Absolute path to the stuck report markdown. + +.OUTPUTS + Issue URL on stdout (and writes Ctx.IssueUrl). +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] $Ctx, + [Parameter(Mandatory)] [string] $ReportPath +) + +. "$PSScriptRoot/Common.ps1" + +if (-not $Ctx.StuckSha) { throw "Ctx.StuckSha is empty — nothing to escalate." } + +# Push the stuck branch so the human can resume on it. +git push -u origin $Ctx.Branch 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } + +$shortSha = $Ctx.StuckSha.Substring(0,9) +$subj = git log -1 --format='%s' $Ctx.StuckSha +$title = "Upstream sync stuck at ${shortSha}: $subj" + +# Header prepended to report content. +$header = @" +🛑 **Upstream sync stopped at a conflict that needs human judgement.** + +The scheduler will keep skipping its runs until this issue is resolved +and the stuck-lock is cleared. No alarm — the lock is intentional. + +To unblock, follow "How to resume" in the report below, then close this +issue after ``clear-stuck.ps1`` runs cleanly. + +--- + +"@ +$body = $header + (Get-Content -Raw -LiteralPath $ReportPath) +$tmp = New-TemporaryFile +[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) + +# Ensure label exists (best-effort; ignore if already present). +gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' 2>$null | Out-Null + +$issueUrl = gh issue create --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 +Remove-Item $tmp -Force +if ($LASTEXITCODE -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed: $issueUrl" +} +$Ctx.IssueUrl = $issueUrl.Trim() + +# Set the stuck-lock on main (direct push — the lock is the gate; PR path is blocked). +git switch main | Out-Null +git pull --ff-only origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run; the stuck branch + issue are already in place." } + +$state = Read-State +$state.stuck_on_sha = $Ctx.StuckSha +$state.stuck_branch = $Ctx.Branch +$state.stuck_at = Format-Iso8601 $Ctx.StartedAt +$state.stuck_issue_url = $Ctx.IssueUrl +$runSummary = [ordered] @{ + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host + status = 'stuck' + branch = $Ctx.Branch + issue_url = $Ctx.IssueUrl + stuck_on_sha = $Ctx.StuckSha + picked_count = $Ctx.Picked.Count +} +$state.last_run = $runSummary +$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 +Write-State $state + +# Copy the report into main too (so the report is visible without checking out the stuck branch). +$reportName = Split-Path -Leaf $ReportPath +$reportOnMain = Join-Path (Get-ReportsDir) $reportName +if ($ReportPath -ne $reportOnMain) { + Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force +} + +git add -- (Get-StatePath) $reportOnMain +if ($LASTEXITCODE -ne 0) { throw "git add of stuck state failed." } +git commit -m "chore(upstream-sync): stuck at $shortSha (#$($Ctx.IssueUrl -replace '.*/',''))" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit of stuck-lock failed — lock NOT set on origin/main. The next scheduled run will not see the lock; resolve manually." } +git push origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — stuck-lock is local only. Push manually before the next scheduler tick or it will re-run over the conflict." } + +return $Ctx.IssueUrl diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 new file mode 100644 index 000000000..72acb76ee --- /dev/null +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -0,0 +1,100 @@ +# Common.ps1 — shared helpers for upstream-sync scripts. +# Dot-source from each script: . "$PSScriptRoot/Common.ps1" + +$ErrorActionPreference = 'Stop' +Set-StrictMode -Version Latest + +function Get-RepoRoot { + $r = git rev-parse --show-toplevel 2>$null + if ($LASTEXITCODE -ne 0) { throw "Not inside a git repo." } + return $r.Trim() +} + +function Get-StateDir { + Join-Path (Get-RepoRoot) '.github/upstream-sync' +} + +function Get-StatePath { + Join-Path (Get-StateDir) 'state.json' +} + +function Get-ReportsDir { + $d = Join-Path (Get-StateDir) 'reports' + if (-not (Test-Path $d)) { New-Item -ItemType Directory -Path $d | Out-Null } + return $d +} + +function Read-State { + $p = Get-StatePath + if (-not (Test-Path $p)) { + throw "state.json not found at $p. Run scripts/00-bootstrap.ps1 first — see references/bootstrap.md." + } + return Get-Content -Raw -LiteralPath $p | ConvertFrom-Json -AsHashtable +} + +function Write-State { + param([Parameter(Mandatory)] $State) + $p = Get-StatePath + $json = $State | ConvertTo-Json -Depth 12 + # Use UTF-8 *without* BOM to match git's default text handling on this repo. + [System.IO.File]::WriteAllText($p, $json, (New-Object System.Text.UTF8Encoding($false))) +} + +function Ensure-UpstreamRemote { + param( + [string] $Name = 'upstream', + [string] $Url = 'https://github.com/microsoft/terminal.git' + ) + $existing = git remote get-url $Name 2>$null + if ($LASTEXITCODE -ne 0) { + git remote add $Name $Url | Out-Null + if ($LASTEXITCODE -ne 0) { throw "Failed to add remote $Name." } + } elseif ($existing.Trim() -ne $Url) { + Write-Warning "Remote '$Name' exists but points at '$existing' (expected '$Url'). Leaving as-is." + } +} + +function Assert-CleanWorktree { + $dirty = git status --porcelain + if ($LASTEXITCODE -ne 0) { throw "git status failed." } + if ($dirty) { + throw "Working tree is not clean:`n$dirty`nCommit or stash first." + } +} + +function Get-GhUserLogin { + $login = gh api user --jq '.login' 2>$null + if ($LASTEXITCODE -ne 0 -or -not $login) { throw "gh CLI is not authenticated. Run 'gh auth login'." } + return $login.Trim() +} + +function Format-Iso8601 { + param([DateTime] $When = (Get-Date)) + return $When.ToString('yyyy-MM-ddTHH:mm:sszzz') +} + +function Format-ReportFilename { + param([DateTime] $When = (Get-Date), [string] $Suffix = '') + $stamp = $When.ToString('yyyy-MM-ddTHHmm') + if ($Suffix) { return "$stamp-$Suffix.md" } + return "$stamp.md" +} + +function New-RunContext { + [pscustomobject] @{ + StartedAt = Get-Date + Host = $env:COMPUTERNAME + Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))" + Picked = @() + DroppedPairs= @() + SkippedEmpty= @() + Tier0 = @() + Tier2 = @() + StuckSha = $null + StuckPaths = @() + Status = 'unknown' + ReportPath = $null + PrUrl = $null + IssueUrl = $null + } +} diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 new file mode 100644 index 000000000..4f83e6c89 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -0,0 +1,61 @@ +<# +.SYNOPSIS + Clear the stuck-lock after a human has merged the manual-resolution PR. + +.DESCRIPTION + Validates that -ResolvedThroughSha is an ancestor of upstream/main and + is at least as new as the stuck SHA, then advances last_synced to it + and clears stuck_on_sha / stuck_branch / stuck_at / stuck_issue_url. + Commits state.json on main. + +.PARAMETER ResolvedThroughSha + The upstream SHA the manual-resolution PR brought the fork up to. + This becomes the new last_synced_upstream_sha. Typically this is the + same SHA that was stuck — the next scheduled run picks up from + ResolvedThroughSha + 1. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $ResolvedThroughSha +) + +. "$PSScriptRoot/Common.ps1" + +$state = Read-State +if (-not $state.stuck_on_sha) { + Write-Warning "No stuck-lock is set. Nothing to clear." + return +} + +Ensure-UpstreamRemote +git fetch upstream main --no-tags | Out-Null + +# Validate the new SHA is on upstream/main. +$null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main +if ($LASTEXITCODE -ne 0) { + throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." +} + +# Validate it's >= the stuck SHA (i.e., stuck is ancestor of resolved). +$null = git merge-base --is-ancestor $state.stuck_on_sha $ResolvedThroughSha +if ($LASTEXITCODE -ne 0) { + throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $ResolvedThroughSha. Refusing — pass the same SHA or a later one." +} + +git switch main | Out-Null +git pull --ff-only | Out-Null + +$state.last_synced_upstream_sha = $ResolvedThroughSha +$state.stuck_on_sha = $null +$state.stuck_branch = $null +$state.stuck_at = $null +$state.stuck_issue_url = $null +Write-State $state + +git add -- (Get-StatePath) | Out-Null +git commit -m "chore(upstream-sync): clear stuck-lock at $($ResolvedThroughSha.Substring(0,9))" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } + +git push origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — lock cleared locally only. Push manually before the next scheduler tick." } +Write-Host "Stuck-lock cleared. Next scheduled run will resume from $($ResolvedThroughSha.Substring(0,9))+1." -ForegroundColor Green diff --git a/.github/upstream-sync/reports/.gitkeep b/.github/upstream-sync/reports/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/.github/upstream-sync/state.json b/.github/upstream-sync/state.json new file mode 100644 index 000000000..cd76cba7c --- /dev/null +++ b/.github/upstream-sync/state.json @@ -0,0 +1,12 @@ +{ + "version": 1, + "upstream_remote_url": "https://github.com/microsoft/terminal.git", + "upstream_branch": "main", + "last_synced_upstream_sha": "a325a2fa5a1cee7e46c23934fa6f82bc1922af3c", + "stuck_on_sha": null, + "stuck_branch": null, + "stuck_at": null, + "stuck_issue_url": null, + "last_run": null, + "history": [] +} \ No newline at end of file From 91f2fff377be398fe88860b0b13728a5e83c2a17 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Thu, 4 Jun 2026 15:25:02 +0800 Subject: [PATCH 02/82] Add post-pick validation (static scan + try-build) to upstream-sync skill Closes the PR #220 audit gap: clean cherry-picks were producing broken PRs because git-level conflict detection missed content-level issues (duplicate .resw keys, dropped fork-specific warning suppressions). New gates run AFTER the cherry-pick loop, BEFORE push/PR: 1. Toolchain preflight -> detect missing PlatformToolset (infra) 2. Static breakage scan -> resw dup baseline-diff + fork invariants 3. Try-build -> razzle + bz no_clean (45min timeout) Failures become Tier-4 stuck (post-pick validation), distinct from Tier-3 cherry-pick conflicts. State schema extended with a new stuck_validation field alongside the existing stuck_on_sha; either being set causes the scheduler to skip. Smoke-tested 08-static-scan against the known-bad PR #220 worktree: returns 26 critical resw-duplicate findings + 1 high C4459 invariant finding, blocking=true (matches the PR #220 audit exactly). Toolchain preflight verified to detect missing v143 on a v145-only host. 10-try-build is wired but not end-to-end tested (would take 30+ min). Also fixed: - gh now uses -R microsoft/intelligent-terminal explicitly on issue/pr calls (the upstream remote was making gh default to microsoft/terminal). - workflow.md: --tags=false -> --no-tags (invalid flag). v1 scope: resw duplicate keys (baseline-aware) + fork invariants from references/fork-invariants.json (seed: C4459 suppression). v2 deferred: missing-include / vcxproj drift (try-build catches these with zero false positives, so not worth a separate static check). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 32 +++- .../references/build-verification.md | 101 +++++++++++ .../references/conflict-triage.md | 39 +++- .../references/fork-invariants.json | 15 ++ .../upstream-sync/references/state-schema.md | 36 +++- .../upstream-sync/references/static-scan.md | 113 ++++++++++++ .../upstream-sync/references/workflow.md | 101 +++++++++-- .../upstream-sync/scripts/04-run-batch.ps1 | 148 +++++++++++++--- .../upstream-sync/scripts/05-write-report.ps1 | 76 +++++++- .../upstream-sync/scripts/06-finalize-pr.ps1 | 4 +- .../scripts/07-open-stuck-issue.ps1 | 3 +- .../07b-open-validation-stuck-issue.ps1 | 147 +++++++++++++++ .../upstream-sync/scripts/08-static-scan.ps1 | 167 ++++++++++++++++++ .../scripts/09-toolchain-preflight.ps1 | 106 +++++++++++ .../upstream-sync/scripts/10-try-build.ps1 | 113 ++++++++++++ .../skills/upstream-sync/scripts/Common.ps1 | 44 +++-- .../upstream-sync/scripts/clear-stuck.ps1 | 100 +++++++---- .github/upstream-sync/.gitignore | 1 + 18 files changed, 1248 insertions(+), 98 deletions(-) create mode 100644 .github/skills/upstream-sync/references/build-verification.md create mode 100644 .github/skills/upstream-sync/references/fork-invariants.json create mode 100644 .github/skills/upstream-sync/references/static-scan.md create mode 100644 .github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 create mode 100644 .github/skills/upstream-sync/scripts/08-static-scan.ps1 create mode 100644 .github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 create mode 100644 .github/skills/upstream-sync/scripts/10-try-build.ps1 create mode 100644 .github/upstream-sync/.gitignore diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index c1991eeb9..9c14c6042 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -50,9 +50,23 @@ with commands, exit codes, and the per-step delegation map lives in ``` fetch upstream → compute pending → drop revert pairs → drop empties → create sync branch → cherry-pick loop (auto-resolve T0/T1, abort on T3) + → toolchain preflight → static breakage scan → try-build (Tier-4 gates) → write report (always) → push + open PR OR open stuck issue + lock ``` +**Safety guarantees (post-pick validation pipeline).** Even when every +cherry-pick applies cleanly, content-level breakage can slip in (duplicate +`.resw` keys from a fork-local commit + an upstream rename touching the +same names; a take-upstream resolution silently dropping a fork-specific +warning suppression; etc. — see PR #220 audit). Before any push or PR, +the orchestrator now runs three hard gates: + +1. **Toolchain preflight** ([`scripts/09-toolchain-preflight.ps1`](./scripts/09-toolchain-preflight.ps1)) — verifies the host has the `PlatformToolset` versions the repo requires. Missing → **Tier-4d infra-stuck** (lock set, NO GitHub issue — PR review can't fix host provisioning). +2. **Static breakage scan** ([`scripts/08-static-scan.ps1`](./scripts/08-static-scan.ps1)) — baseline-diffs `.resw` files for newly-duplicated `` keys, and regex-checks fork invariants from [`references/fork-invariants.json`](./references/fork-invariants.json). Blocking → **Tier-4a stuck**. +3. **Try-build** ([`scripts/10-try-build.ps1`](./scripts/10-try-build.ps1)) — runs `tools\razzle.cmd && bz no_clean` with a 45-minute wall-clock cap. Build failed → **Tier-4b stuck**; timeout → **Tier-4c stuck** (unless `-AllowInconclusiveBuild`). + +See [references/static-scan.md](./references/static-scan.md), [references/build-verification.md](./references/build-verification.md), and [references/conflict-triage.md](./references/conflict-triage.md) Tier-4 for details. + **Run it:** ```pwsh @@ -67,6 +81,19 @@ pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -PushDirectToMain # Compute & report without picking pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -DryRun + +# Skip the static scan (debugging only — schedulers must run it) +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -SkipStaticScan + +# Skip the try-build (debugging only — schedulers must run it). +# Also skips toolchain preflight since they share the same infra prerequisite. +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -SkipBuild + +# Don't treat a build timeout as Tier-4 stuck (dev opt-in; never in a scheduler) +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -AllowInconclusiveBuild + +# Override build timeout (default 45 minutes) +pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -BuildTimeoutMinutes 60 ``` ### Finalize modes — what each preserves @@ -142,8 +169,11 @@ is set exits early without touching the branch. - [references/workflow.md](./references/workflow.md) — full per-step procedure with exit codes and delegation map. - [references/state-schema.md](./references/state-schema.md) — `state.json` shape and field semantics. - [references/bootstrap.md](./references/bootstrap.md) — one-time baseline-SHA discovery and initialization. -- [references/conflict-triage.md](./references/conflict-triage.md) — Tier 0/1/2/3 resolution rubric with examples. +- [references/conflict-triage.md](./references/conflict-triage.md) — Tier 0/1/2/3/4 resolution rubric with examples. - [references/known-conflicts.md](./references/known-conflicts.md) — files that always need a fixed resolution. +- [references/static-scan.md](./references/static-scan.md) — post-pick static breakage scan rules. +- [references/fork-invariants.json](./references/fork-invariants.json) — fork-specific patterns that must survive any upstream pick. +- [references/build-verification.md](./references/build-verification.md) — try-build pipeline + toolchain preflight policy. - [references/reporting.md](./references/reporting.md) — report template and stuck-issue template. - [scripts/04-run-batch.ps1](./scripts/04-run-batch.ps1) — the scheduler entrypoint. - [scripts/clear-stuck.ps1](./scripts/clear-stuck.ps1) — clear the stuck-lock after human resolution. diff --git a/.github/skills/upstream-sync/references/build-verification.md b/.github/skills/upstream-sync/references/build-verification.md new file mode 100644 index 000000000..ed9e23555 --- /dev/null +++ b/.github/skills/upstream-sync/references/build-verification.md @@ -0,0 +1,101 @@ +# Build verification + +Post-batch hard gate. Runs after the static scan passes and **before** +push / PR creation. If the build fails, the run is marked Tier-4 stuck. + +## Why this exists + +Static scan catches a specific set of content drifts. The compiler +catches everything else (missing includes, type mismatches, +vcxproj drift, MIDL/winrt projection errors, ...) with zero false +positives. A scheduler that opens PRs without proof the codebase still +builds is opening broken PRs — exactly what PR #220 risked. + +## Pipeline + +``` +toolchain preflight ─→ static scan ─→ try-build ─→ push / PR + │ │ │ + (infra-stuck) (Tier-4) (Tier-4) +``` + +## Toolchain preflight (`scripts/09-toolchain-preflight.ps1`) + +Runs first. Discovers the required `PlatformToolset` from +`src/common.build.pre.props` (and other props files) and verifies it +is installed under any Visual Studio install on the host. + +Outcomes: + +| Outcome | Behavior | +|---------------------|--------------------------------------------------------| +| All toolsets found | Continue to static scan + build. | +| Required missing | Tier-4 **infra-stuck** — separate kind from code-stuck. Does NOT open a stuck issue (it's a host problem, not a code problem). | +| Skipped (`-SkipBuild`) | Preflight not run. Caller accepts risk. | + +**The preflight does NOT auto-bump v143→v145.** That recipe is +intentionally kept as a *local-only* developer workaround (see the +`v143-v145` memory notes). Auto-bump risks silently changing the +toolset for everyone, which would break the rest of the team. +Schedulers should be provisioned with the correct VS install instead. + +## Try-build (`scripts/10-try-build.ps1`) + +Default invocation: + +```cmd +cmd.exe /c "tools\razzle.cmd && bz no_clean" +``` + +(`bz no_clean` = incremental Debug build of the full solution.) + +Configurable via the orchestrator's `-BuildCommand` parameter. The +default is verified on the maintainer host and documented in the +state.json `last_run.build_command` field for traceability. + +Output: + +- Full build log → `.github/upstream-sync/build-logs/.log` + (gitignored — these are big and noisy). +- Last ~200 lines → captured into the run report and any Tier-4 stuck + issue. +- Exit code + duration → state.json. + +Timeout: + +- Default 45 minutes (cold builds on a new sync branch with many + picks can hit ~30 min; 45 gives headroom). +- Configurable via `-BuildTimeoutMinutes N`. +- On timeout the build is killed and classified as + **build-inconclusive**. + +| Outcome | Default scheduler behavior | Dev opt-out | +|--------------------|--------------------------------|--------------------------------------| +| Build succeeded | Continue to push / PR. | n/a | +| Build failed | Tier-4 stuck, open issue. | n/a | +| Build inconclusive | Tier-4 stuck (be safe). | `-AllowInconclusiveBuild` → proceed with warning in report. | +| Skipped | Reject in scheduler context. | `-SkipBuild` for fast dev iteration. | + +The "be safe" default for inconclusive is deliberate. A hung build is +indistinguishable from a real failure in scheduler mode; opening a PR +on an unproven sync defeats the whole point of this gate. + +## When the build fails for fork-unrelated reasons + +If a flaky build (unrelated env issue, transient toolchain glitch) +trips the gate: + +1. The stuck issue gives a clear log tail. +2. A human can `clear-stuck.ps1 -ResolvedThroughSha ` after + re-running the build locally to confirm it's a transient. +3. The next scheduler tick will re-attempt the same pick range. + +Distinguishing transient-build from real-pick-broke-build is left to +the human reviewing the issue — too noisy to automate, and the cost +of a manual cross-check is small (~once per N runs). + +## Build artifacts + +`.github/upstream-sync/build-logs/` is **not** committed (added to +`.gitignore` by the skill PR). Build artifacts under `bin/`, `obj/`, +etc. follow the repo's existing `.gitignore`. diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/conflict-triage.md index 5568823e8..1af95b99c 100644 --- a/.github/skills/upstream-sync/references/conflict-triage.md +++ b/.github/skills/upstream-sync/references/conflict-triage.md @@ -83,7 +83,7 @@ Tier 3. If it returns content, **verify with a second fresh agent**: Stage only if both agents agree `high`/`OK`. Otherwise → Tier 3. -## Tier 3 — Stop and escalate +## Tier 3 — Stop and escalate (cherry-pick conflict) Anything not resolved by Tier 0–2: @@ -106,6 +106,43 @@ The report **must** include: pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha ``` +## Tier 4 — Post-pick validation failed + +The cherry-picks all applied cleanly, but a hard gate after the loop +said NO before any push or PR. This catches the class of bug missed by +git-level conflict detection: clean-merge-but-broken-content (PR #220 +audit found duplicate `.resw` keys + a dropped fork-specific `C4459` +suppression — both committed without git ever printing a conflict). + +The orchestrator runs three gates after the cherry-pick loop, in order: + +| Sub-tier | Gate | Symptom | Action | +|---|---|---|---| +| **4a** | Static breakage scan ([`scripts/08-static-scan.ps1`](../scripts/08-static-scan.ps1)) | New duplicate `.resw` keys vs base, or a missing fork invariant from [`fork-invariants.json`](./fork-invariants.json) | Lock + issue + exit 10 | +| **4b** | Try-build ([`scripts/10-try-build.ps1`](../scripts/10-try-build.ps1)) | Build exited non-zero within timeout | Lock + issue + exit 10 | +| **4c** | Try-build timeout | Wall-clock cap (default 45m) hit | Lock + issue + exit 10 — unless `-AllowInconclusiveBuild` (dev opt-in) | +| **4d** | Toolchain preflight ([`scripts/09-toolchain-preflight.ps1`](../scripts/09-toolchain-preflight.ps1)) | Required `PlatformToolset` (e.g. v143) not present on host | Lock + **NO issue** — provisioning problem, not code | + +**Why these three gates and not more.** They were sized to catch the +historical real-world failures with zero false positives: +- 4a covers content-level pattern breakage where git is happy but the + resulting file violates a fork-specific invariant. +- 4b is the broadest possible "did this even compile" check. +- 4c distinguishes "build hung — needs investigation" from "build + failed for a discoverable reason". +- 4d distinguishes "this code is broken" from "this host can't even + try to build it" — the latter must never open a GitHub issue. + +Tier-4 state lives in `state.stuck_validation` (separate from +Tier-3's `state.stuck_on_sha`); either being set causes the +scheduler to skip. Clear with [`clear-stuck.ps1`](../scripts/clear-stuck.ps1) +(omit `-ResolvedThroughSha` to keep the watermark and re-attempt the +same range; pass it to advance past the broken upstream batch). + +The Tier-4 report includes a `findings_hash` (16-hex prefix). Re-runs +that produce the same hash mean the underlying defect is unchanged; +a changed hash means validation has moved to a new failure mode. + ## Line endings If any Tier-2 resolution touches a file with CRLF line endings (most diff --git a/.github/skills/upstream-sync/references/fork-invariants.json b/.github/skills/upstream-sync/references/fork-invariants.json new file mode 100644 index 000000000..a78ac35c5 --- /dev/null +++ b/.github/skills/upstream-sync/references/fork-invariants.json @@ -0,0 +1,15 @@ +{ + "$schema_comment": "Invariants the fork must preserve after every upstream pick. The static-scan step ('scripts/08-static-scan.ps1') fails the run if any of these patterns is missing from the post-pick worktree. Use this for fork-specific content that an upstream cherry-pick can silently strip (e.g. take-upstream conflict resolution removing a fork-added warning suppression).", + "version": 1, + "invariants": [ + { + "id": "common-build-c4459-suppression", + "path": "src/common.build.pre.props", + "must_contain_regex": "\\b4459\\b", + "severity": "high", + "reason": "Fork added C4459 (declaration of 'X' hides class member) suppression because TreatWarningAsError=true and fork-specific code triggers it. Upstream removed C4459 in the ATL/MFC cleanup; a take-upstream resolution drops it and the next build fails. See PR #220 audit (2026-06-04).", + "introduced_in_fork_sha": "4bf2b6a45", + "added_after_pr_220_audit": true + } + ] +} diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md index 18a98baed..272fa8a3d 100644 --- a/.github/skills/upstream-sync/references/state-schema.md +++ b/.github/skills/upstream-sync/references/state-schema.md @@ -12,19 +12,36 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). // Updated only when a sync PR merges (the PR includes the state update). "last_synced_upstream_sha": "93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f", - // Stuck-lock. When non-null, the scheduler exits early without touching - // any branch. Cleared by scripts/clear-stuck.ps1 after a human merges - // the resolution PR. + // Stuck-lock (Tier-3 — cherry-pick conflict). When non-null, the + // scheduler exits early without touching any branch. Cleared by + // scripts/clear-stuck.ps1 after a human merges the resolution PR. "stuck_on_sha": null, "stuck_branch": null, "stuck_at": null, // ISO 8601 timestamp; null when not stuck "stuck_issue_url": null, // populated by 07-open-stuck-issue.ps1 + // Stuck-lock (Tier-4 — post-pick validation failed). Set when picks + // applied cleanly but static-scan / try-build / toolchain-preflight + // blocked the push. Independent of stuck_on_sha but treated the same + // way by the scheduler gate: either lock present → skip. + // Shape (when non-null): + // { + // "kind": "static-scan" | "build-failed" | "build-inconclusive" | "toolchain-missing", + // "base": "", + // "head": "", + // "branch": "upstream-sync/YYYY-MM-DD", + // "range": ["", "", ...], // picks that landed before validation said no + // "findings_hash": "<16-hex sha256 prefix>", // dedup signal across re-runs + // "at": "ISO 8601", + // "issue_url": "https://..." | null // null for toolchain-missing (infra) + // } + "stuck_validation": null, + // Last run summary (for fast inspection without grepping reports). "last_run": { "at": "2026-06-04T13:41:45+08:00", "host": "SH-YEELAM-D11S", - "status": "ok", // "ok" | "no-op" | "stuck" | "skipped-locked" + "status": "ok", // "ok" | "no-op" | "stuck" | "stuck-static-scan" | "stuck-build-failed" | "stuck-build-inconclusive" | "stuck-toolchain-missing" | "skipped-locked" "branch": "upstream-sync/2026-06-04", "pr_url": "https://github.com/microsoft/intelligent-terminal/pull/999", "picked_count": 7, @@ -48,9 +65,18 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). The orchestrator updates this in the PR commit itself, so it lands atomically with the picks. Never edit by hand except via `clear-stuck.ps1`. -- **`stuck_on_sha`** is the gate. When set, `04-run-batch.ps1` exits 0 +- **`stuck_on_sha`** is the Tier-3 gate. When set, `04-run-batch.ps1` exits 0 without doing anything. This is intentional — the scheduler will keep ticking but will not clobber the stuck branch. +- **`stuck_validation`** is the Tier-4 gate, independent of `stuck_on_sha`. + Either one being non-null causes the scheduler to skip. Cleared by + `clear-stuck.ps1` — `-ResolvedThroughSha` is optional for Tier-4 (omit + to keep the watermark and have the next run re-attempt the same range + after the human fixed whatever validation caught). +- **`findings_hash`** in `stuck_validation` is a stable 16-hex prefix of + the SHA-256 of the normalized findings list. If a re-run produces the + same hash, the human knows the underlying fault is unchanged; if it + changes, validation has moved to a new failure mode. - **`stuck_branch`** must still exist on `origin` until the human merges it; `clear-stuck.ps1` does not delete it (the PR merge does). - **`history`** is for the human reading state.json directly. The reports diff --git a/.github/skills/upstream-sync/references/static-scan.md b/.github/skills/upstream-sync/references/static-scan.md new file mode 100644 index 000000000..4a6298f1e --- /dev/null +++ b/.github/skills/upstream-sync/references/static-scan.md @@ -0,0 +1,113 @@ +# Static breakage scan + +Post-batch hard gate. Runs after all cherry-picks succeed and **before** +push / PR creation. If any finding is `critical` or `high`, the run is +marked Tier-4 stuck (see [conflict-triage.md](./conflict-triage.md)). + +## Why this exists + +A clean cherry-pick is not a working cherry-pick. PR #220 demonstrated +two failure modes that git-level conflict detection misses: + +1. **Duplicate `.resw` keys** — fork had appended translations for new + keys, upstream loc-bot pick renamed old keys to the same names → + duplicates in 26 files / 216 keys. +2. **Dropped fork-specific build suppressions** — `take-upstream` + resolution of `src/common.build.pre.props` removed the fork-added + `C4459` warning suppression. + +Both were caught post-hoc by audit. This scan catches them pre-PR. + +## What v1 checks + +### 1. Duplicate `.resw` keys (baseline-diff) + +For every `*.resw` file modified anywhere in the pick range, count +duplicate `` entries in the **pre-pick** state vs the +**post-pick** worktree. Gate on **newly-introduced** duplicates only — +pre-existing duplicates are reported as `info`, not blocking. + +Pre-pick state = the file content at `origin/main` (the orchestrator's +base before the picks). Post-pick = the worktree on the sync branch. + +Severity: +- `critical` — any new duplicate key introduced by the pick range. +- `info` — pre-existing duplicates carried forward. + +### 2. Fork invariants (regex must-match) + +Reads `references/fork-invariants.json`. For each entry: +- Load the file at `path` from the post-pick worktree. +- Test `must_contain_regex` (case-sensitive, single-line). +- If it doesn't match → finding at the configured `severity`. + +Seed: `C4459` suppression in `src/common.build.pre.props`. + +Add new invariants whenever an audit finds another fork-specific item +that an upstream pick could silently strip. Keep the regex narrow. + +## What v1 does NOT check (deferred to v2) + +Documented here so contributors don't re-invent them and so v2 has a +clear scope: + +- **Missing `#include`s** — narrow scope (newly-added quote includes + in modified .h/.cpp/.idl, skipping `<...>`, `*.g.h`, `*.g.cpp`, + `precomp.h`, `winrt/.*`) is technically straightforward but each + exclusion list is a tail of false-positives. The try-build step (see + [build-verification.md](./build-verification.md)) catches missing + includes as compile errors with zero false positives, so v1 relies + on the build step for this class. +- **vcxproj / vcxproj.filters drift** — same reasoning: a missing + `ClCompile Include=` reference fails the build with a clear error. +- **MTSMSettings / GlobalAppSettings drift** — relies on the build + step for now; a v2 addition could be a focused cross-check if a real + miss slips past the build. + +## Output + +The scan emits a JSON document on stdout: + +```json +{ + "findings": [ + { + "check": "resw-duplicate-keys", + "severity": "critical", + "path": "src/cascadia/TerminalApp/Resources/zh-CN/Resources.resw", + "detail": "15 newly-duplicated entries", + "examples": ["ConfirmCloseDialog_Cancel", "..."] + }, + { + "check": "fork-invariant", + "severity": "high", + "id": "common-build-c4459-suppression", + "path": "src/common.build.pre.props", + "detail": "regex '\\b4459\\b' did not match" + } + ], + "summary": { + "critical": 13, + "high": 1, + "medium": 0, + "low": 0, + "info": 0 + }, + "blocking": true +} +``` + +`blocking` = true ⇔ any `critical` or `high` finding present. + +## Extending the scan + +To add a new check: + +1. Add a function to `scripts/08-static-scan.ps1` that returns a list + of finding hashtables (same shape as above). +2. Wire it into the main loop in `08-static-scan.ps1`. +3. Document the check here. +4. Add a baseline-aware test (compare pre-pick vs post-pick) **only + when** the check would otherwise gate on pre-existing issues. +5. Wire its severities into the `blocking` calculation if it should + block the run. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index 7b394e994..b46e20a85 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -9,7 +9,7 @@ maps to a script or an in-orchestrator function. - `state.json` exists (bootstrap done — see [bootstrap.md](./bootstrap.md)). - Working tree is clean (`git status --porcelain` empty). - We are on `main` (or the script will `git switch main`). -- `state.stuck_on_sha` is `null` (otherwise exit early — see "Stuck-lock" below). +- `state.stuck_on_sha` is `null` AND `state.stuck_validation` is `null` (otherwise exit early — see "Stuck-lock" below). ## Steps @@ -17,7 +17,7 @@ maps to a script or an in-orchestrator function. ```pwsh git remote get-url upstream 2>$null || git remote add upstream https://github.com/microsoft/terminal.git -git fetch upstream main --tags=false +git fetch upstream main --no-tags ``` Script: [`01-fetch-upstream.ps1`](../scripts/01-fetch-upstream.ps1). @@ -97,26 +97,79 @@ git cherry-pick --keep-redundant-commits -x Script: [`03-cherry-pick-one.ps1`](../scripts/03-cherry-pick-one.ps1) handles one commit, returns a JSON status object. The orchestrator loops. -### 7. Write report (always) +### 7. Post-pick validation gates (Tier-4) -Regardless of outcome (ok / no-op / stuck), write -`.github/upstream-sync/reports/YYYY-MM-DDTHHmm.md` with: +After all cherry-picks complete cleanly, the orchestrator runs three +hard gates **before** writing the report or pushing anything. The order +matters: cheapest infra check first, then content, then full build. + +#### 7a. Toolchain preflight + +```pwsh +pwsh .github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +# Emits JSON { required_toolsets, available_toolsets, missing, vs_installs, ok } +``` + +Detects required `` values from `src/common.build.*.props` +and checks they exist under `\MSBuild\Microsoft\VC\\Platforms\x64\PlatformToolsets\`. +If `ok=false`, this is **Tier-4d infra-stuck**: lock + NO GitHub issue +(PR review cannot fix host provisioning). Skipped when `-SkipBuild` is set. + +#### 7b. Static breakage scan + +```pwsh +pwsh .github/skills/upstream-sync/scripts/08-static-scan.ps1 -BaseSha $preBase +# Emits JSON { base, head, findings: [...], summary: { critical, high, ... }, blocking } +``` + +`$preBase` is `git rev-parse origin/main` captured BEFORE the cherry-pick +loop. The scan: + +- Baseline-diffs every changed `.resw` file for NEW duplicate `` + keys (pre-existing dups are reported as `info`, not blocking). +- Runs regex assertions from [`fork-invariants.json`](./fork-invariants.json) + against the post-pick worktree. + +If `blocking=true` (any `critical` or `high` finding), this is **Tier-4a +stuck**: lock + GitHub issue + exit 10. Skipped when `-SkipStaticScan`. + +#### 7c. Try-build + +```pwsh +pwsh .github/skills/upstream-sync/scripts/10-try-build.ps1 -BuildCommand 'tools\razzle.cmd && bz no_clean' -TimeoutMinutes 45 +# Emits JSON { kind, exit_code, duration_ms, log_path, log_tail } +``` + +- `kind = build-ok` → continue to step 8. +- `kind = build-failed` → **Tier-4b stuck**. +- `kind = build-inconclusive` (timeout) → **Tier-4c stuck**, unless + `-AllowInconclusiveBuild` (dev opt-in; never in a scheduler). + +Skipped when `-SkipBuild`. Logs land in `.github/upstream-sync/build-logs/` +(git-ignored). + +### 8. Write report (always) + +Regardless of outcome (ok / no-op / stuck / stuck-static-scan / +stuck-build-failed / stuck-build-inconclusive / stuck-toolchain-missing), +write `.github/upstream-sync/reports/YYYY-MM-DDTHHmm[-suffix].md` with: - Run metadata (start, end, duration, host, status) - Counts: picked / dropped-pair / empty / known-conflict-resolved / stuck-at - For each picked commit: SHA, subject, author, files-touched count - For dropped pairs: the two SHAs and their subjects -- If stuck: the conflicting commit, the conflicting paths, what was attempted, the exact resume command +- If stuck (Tier-3): the conflicting commit, the conflicting paths, what was attempted, the exact resume command +- If stuck (Tier-4): the validation findings, the build log tail, the exact resume command Template: [`reporting.md`](./reporting.md). Script: [`05-write-report.ps1`](../scripts/05-write-report.ps1). -### 8a. Success path — push + open PR +### 9a. Success path — push + open PR ```pwsh git push -u origin $branch -gh pr create --base main --head "$($me):$branch" --title "chore(upstream): sync up to $shortSha" --body-file $reportPath +gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title "chore(upstream): sync up to $shortSha" --body-file $reportPath ``` Update `state.last_synced_upstream_sha = upstream/main` and commit @@ -125,10 +178,10 @@ add a trailing commit titled `chore(upstream-sync): update state`). Script: [`06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1). -### 8b. Stuck path — open issue + set lock +### 9b. Stuck path (Tier-3) — open issue + set lock ```pwsh -gh issue create --label upstream-sync-stuck ` +gh issue create -R microsoft/intelligent-terminal --label upstream-sync-stuck ` --title "Upstream sync stuck at : " ` --body-file $reportPath ``` @@ -140,23 +193,39 @@ and exits. Script: [`07-open-stuck-issue.ps1`](../scripts/07-open-stuck-issue.ps1). +### 9c. Stuck path (Tier-4) — open issue + set lock + +For Tier-4a/b/c, the same flow as 9b but the issue title carries the +validation kind and findings hash; `state.stuck_validation` is set +instead of `state.stuck_on_sha`. For Tier-4d (toolchain-missing), only +the lock is set — NO issue is opened. + +Script: [`07b-open-validation-stuck-issue.ps1`](../scripts/07b-open-validation-stuck-issue.ps1). + ## Stuck-Lock -When `state.stuck_on_sha` is non-null, the orchestrator: +When **either** `state.stuck_on_sha` (Tier-3) **or** `state.stuck_validation` +(Tier-4) is non-null, the orchestrator: -1. Logs `"stuck-lock set at ; skipping run"`. +1. Logs `"stuck-lock set: ; skipping run"`. 2. Writes a `reports/YYYY-MM-DDTHHmm-skipped.md` noting the skip. 3. Exits 0 (the scheduler should not retry on the same lock). -To clear the lock after the human has merged a PR resolving the stuck -commit: +To clear the lock after the human has resolved the underlying issue: ```pwsh +# Tier-3: -ResolvedThroughSha is REQUIRED and advances the watermark. pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha + +# Tier-4: -ResolvedThroughSha is OPTIONAL. Omit it to keep the watermark +# and have the next run re-attempt the same range (recommended when the +# fix lands as a separate PR on main — the next sync will pick up the +# upstream batch atop the now-fixed main and re-validate). +pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 ``` -This sets `state.last_synced_upstream_sha = `, clears `stuck_on_sha` -and `stuck_branch`, and commits `state.json` on `main`. +This sets `state.last_synced_upstream_sha` (when advanced), clears the +appropriate lock fields, and commits `state.json` on `main`. ## Sub-Agent Delegation Map diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index fc30339ed..555cb42a5 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -4,16 +4,24 @@ scheduler on a weekly/daily cadence. .DESCRIPTION - Reads state.json. If the stuck-lock is set, writes a skipped-locked - report and exits 0. Otherwise: + Reads state.json. If the stuck-lock (Tier-3 stuck_on_sha OR Tier-4 + stuck_validation) is set, writes a skipped-locked report and exits 0. + Otherwise: 1. Fetches upstream/main. 2. Computes pending commits, dropping revert pairs and empties. 3. Creates branch upstream-sync/YYYY-MM-DD. 4. Cherry-picks one-by-one with Tier-0/Tier-1 auto-resolution. - 5. Writes a report. - 6. On success → pushes branch, opens PR (exit 0). - On stuck → pushes branch, opens issue, sets lock (exit 10). - On no-op → exits 0 with a "no-op" report. + On cherry-pick conflict → Tier-3 stuck path (07). + 5. Post-batch HARD GATES (in order, before any push/PR): + a. Toolchain preflight (09) — missing toolset = infra stuck. + b. Static breakage scan (08) — duplicate resw / fork invariants. + c. Try-build (10) — razzle + bz no_clean. + Any failure → Tier-4 stuck path (07b). + 6. Writes a report. + 7. On success → pushes branch, opens PR (exit 0). + On Tier-3 → pushes branch, opens issue, sets lock (exit 10). + On Tier-4 → pushes branch, opens issue (except infra), sets lock (exit 10). + On no-op → exits 0 with a "no-op" report. .PARAMETER DryRun Compute & report only; do not create the branch or pick anything. @@ -22,31 +30,42 @@ Reserved: enable LLM-assisted Tier-2 conflict resolution (NOT YET IMPLEMENTED). .PARAMETER Force - Override the stuck-lock. DANGEROUS — clobbers the in-progress branch. - Use only when you know the lock is stale. + Override the stuck-lock (Tier-3 OR Tier-4). DANGEROUS — clobbers the + in-progress branch. Use only when you know the lock is stale. .PARAMETER MaxPicks - Cap the number of cherry-picks per run (default: unlimited). Useful for - smoke-testing the scheduler with a few commits at a time. + Cap the number of cherry-picks per run (default: unlimited). .PARAMETER PushDirectToMain Skip the PR and fast-forward main directly to the sync branch tip. - Requires push permission on main (admin / branch-protection bypass). - Preserves per-commit content, order, and original author dates — strictly - better than a squash-merged PR. Use when there's no need for a human - review checkpoint per sync. + Requires push permission on main. .PARAMETER AutoMergeStrategy - PR mode only. After opening the PR, run `gh pr merge -- --auto` - so when CI/approvals pass, GitHub auto-merges with the right strategy. - Allowed: 'rebase' (preserves per-commit, recommended), 'merge' (adds a - merge commit; per-commit also preserved), or 'none' (default; human - picks the strategy manually — but they must NOT pick squash). + PR mode only. After opening the PR, run `gh pr merge -- --auto`. + Allowed: 'rebase' (recommended), 'merge', or 'none' (default). + +.PARAMETER SkipStaticScan + Skip step 5b. Default: scan. Schedulers MUST run the scan. + +.PARAMETER SkipBuild + Skip steps 5a + 5c. Default: build. Schedulers MUST build. + +.PARAMETER AllowInconclusiveBuild + Don't treat a build timeout as Tier-4 stuck — proceed with a warning + in the report. Dev opt-in only; schedulers should leave it off so + hung builds don't escape into unproven PRs. + +.PARAMETER BuildTimeoutMinutes + Wall-clock cap for try-build. Default 45. + +.PARAMETER BuildCommand + Override the default build command (passed to cmd.exe). Default: + 'tools\razzle.cmd && bz no_clean'. .OUTPUTS Writes status to stdout. Exit codes: 0 = success (PR opened) OR no-op OR skipped-locked - 10 = stuck (issue opened, lock set) — NOT an error + 10 = stuck (Tier-3 or Tier-4) — NOT an error 20 = hard failure (git/gh broken) — alarm-worthy #> [CmdletBinding()] @@ -56,7 +75,12 @@ param( [switch] $Force, [int] $MaxPicks = 0, [switch] $PushDirectToMain, - [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' + [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none', + [switch] $SkipStaticScan, + [switch] $SkipBuild, + [switch] $AllowInconclusiveBuild, + [int] $BuildTimeoutMinutes = 45, + [string] $BuildCommand = 'tools\razzle.cmd && bz no_clean' ) . "$PSScriptRoot/Common.ps1" @@ -66,13 +90,36 @@ function Exit-Hard([string] $msg) { exit 20 } +function Invoke-Tier4Stuck { + param( + $Ctx, + [string] $Kind, + [string] $FromSha, + [string] $ToSha + ) + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $Ctx -From $FromSha -To $ToSha -Status "stuck-$Kind" + $Ctx.ReportPath = $reportPath + Write-Host "Tier-4 stuck report: $reportPath" + $issueUrl = & "$PSScriptRoot/07b-open-validation-stuck-issue.ps1" -Ctx $Ctx -ReportPath $reportPath -Kind $Kind + if ($issueUrl) { Write-Host "Stuck issue: $issueUrl" -ForegroundColor Yellow } + exit 10 +} + try { $state = Read-State $ctx = New-RunContext - # --- Stuck-lock gate --- - if ($state.stuck_on_sha -and -not $Force) { - Write-Host "Stuck-lock set at $($state.stuck_on_sha) (issue: $($state.stuck_issue_url)). Skipping." -ForegroundColor Yellow + # --- Stuck-lock gate (Tier-3 OR Tier-4) --- + $stuckTier3 = [bool] $state.stuck_on_sha + $stuckTier4 = [bool] $state.stuck_validation + if (($stuckTier3 -or $stuckTier4) -and -not $Force) { + $lockDesc = if ($stuckTier3) { + "Tier-3 at $($state.stuck_on_sha) (issue: $($state.stuck_issue_url))" + } else { + $v = $state.stuck_validation + "Tier-4 $($v.kind) [hash $($v.findings_hash)] (issue: $($v.issue_url))" + } + Write-Host "Stuck-lock set: $lockDesc. Skipping." -ForegroundColor Yellow $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $state.last_synced_upstream_sha -To $state.last_synced_upstream_sha -Status 'skipped-locked' Write-Host "Skip report: $reportPath" exit 0 @@ -117,6 +164,10 @@ try { exit 0 } + # Capture pre-pick base SHA (origin/main) — used as static-scan baseline. + $preBase = git rev-parse origin/main + if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not resolve origin/main for scan baseline." } + # --- 3. Create / switch to sync branch --- $branch = $ctx.Branch git switch -c $branch 2>$null @@ -156,7 +207,7 @@ try { if ($ctx.Status -eq 'stuck') { break } } - # --- 5. Report + finalize --- + # --- 5. Tier-3 short-circuit --- if ($ctx.Status -eq 'stuck') { $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'stuck' $ctx.ReportPath = $reportPath @@ -166,6 +217,53 @@ try { exit 10 } + # --- 5a. Toolchain preflight (Tier-4 gate: infra-missing) --- + if (-not $SkipBuild) { + Write-Host "" + Write-Host "=== Toolchain preflight ===" -ForegroundColor Cyan + $preflightJson = & "$PSScriptRoot/09-toolchain-preflight.ps1" + $ctx.Preflight = $preflightJson | ConvertFrom-Json + Write-Host "Required: $($ctx.Preflight.required_toolsets -join ', '); available: $($ctx.Preflight.available_toolsets -join ', ')" + if (-not $ctx.Preflight.ok) { + Write-Warning "Toolchain preflight FAILED — missing: $($ctx.Preflight.missing -join ', ')" + Invoke-Tier4Stuck -Ctx $ctx -Kind 'toolchain-missing' -FromSha $fromSha -ToSha $toSha + } + } + + # --- 5b. Static breakage scan (Tier-4 gate: scan-blocking) --- + if (-not $SkipStaticScan) { + Write-Host "" + Write-Host "=== Static breakage scan ===" -ForegroundColor Cyan + $scanJson = & "$PSScriptRoot/08-static-scan.ps1" -BaseSha $preBase -HeadRef 'HEAD' + $ctx.Scan = $scanJson | ConvertFrom-Json + $sm = $ctx.Scan.summary + Write-Host "Findings: critical=$($sm.critical), high=$($sm.high), medium=$($sm.medium), low=$($sm.low), info=$($sm.info); blocking=$($ctx.Scan.blocking)" + if ($ctx.Scan.blocking) { + Invoke-Tier4Stuck -Ctx $ctx -Kind 'static-scan' -FromSha $fromSha -ToSha $toSha + } + } + + # --- 5c. Try-build (Tier-4 gate: build-failed / build-inconclusive) --- + if (-not $SkipBuild) { + Write-Host "" + Write-Host "=== Try-build (timeout ${BuildTimeoutMinutes}m) ===" -ForegroundColor Cyan + $buildJson = & "$PSScriptRoot/10-try-build.ps1" -BuildCommand $BuildCommand -TimeoutMinutes $BuildTimeoutMinutes + $ctx.Build = $buildJson | ConvertFrom-Json + Write-Host "Build: $($ctx.Build.kind) (exit=$($ctx.Build.exit_code), duration=$([int]($ctx.Build.duration_ms / 1000))s)" + switch ($ctx.Build.kind) { + 'build-failed' { Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-failed' -FromSha $fromSha -ToSha $toSha } + 'build-inconclusive' { + if ($AllowInconclusiveBuild) { + Write-Warning "Build inconclusive — proceeding (--AllowInconclusiveBuild)." + } else { + Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-inconclusive' -FromSha $fromSha -ToSha $toSha + } + } + 'build-ok' { Write-Host "Build OK." -ForegroundColor Green } + } + } + + # --- 6. Report + finalize --- $ctx.Status = 'ok' $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'ok' $ctx.ReportPath = $reportPath diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 4f75ab65e..3f058b306 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -22,7 +22,7 @@ param( [Parameter(Mandatory)] $Ctx, [Parameter(Mandatory)] [string] $From, [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [ValidateSet('ok','no-op','stuck','skipped-locked')] [string] $Status + [Parameter(Mandatory)] [ValidateSet('ok','no-op','stuck','skipped-locked','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status ) . "$PSScriptRoot/Common.ps1" @@ -144,11 +144,83 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { $lines.Add("") } +if ($Status -like 'stuck-*') { + $kind = $Status -replace '^stuck-','' + $lines.Add("## Validation diagnostics — Tier-4 ($kind)") + $lines.Add("") + $lines.Add("All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch validation step blocked the push.") + $lines.Add("") + switch ($kind) { + 'static-scan' { + $sm = $Ctx.Scan.summary + $lines.Add("**Static scan summary:** critical=$($sm.critical), high=$($sm.high), medium=$($sm.medium), low=$($sm.low), info=$($sm.info)") + $lines.Add("") + $blocking = @($Ctx.Scan.findings | Where-Object { $_.severity -in @('critical','high') }) + if ($blocking.Count -gt 0) { + $lines.Add("**Blocking findings:**") + $lines.Add("") + $lines.Add("| Severity | Kind | Where | Detail |") + $lines.Add("|---|---|---|---|") + foreach ($f in $blocking) { + $where = if ($f.path) { "``$($f.path)``" } else { '—' } + $detail = ($f | ConvertTo-Json -Compress -Depth 4) + $lines.Add("| $($f.severity) | $($f.kind) | $where | ``$detail`` |") + } + $lines.Add("") + } + } + 'build-failed' { + $lines.Add("**Build exit code:** $($Ctx.Build.exit_code) ") + $lines.Add("**Build duration:** $([int]($Ctx.Build.duration_ms / 1000))s ") + $lines.Add("**Build log:** ``$($Ctx.Build.log_path)``") + $lines.Add("") + $lines.Add("**Last lines of build log:**") + $lines.Add("") + $lines.Add('```') + $lines.Add(($Ctx.Build.log_tail -split "`n" | Select-Object -Last 80) -join "`n") + $lines.Add('```') + $lines.Add("") + } + 'build-inconclusive' { + $lines.Add("**Build hit timeout** after $([int]($Ctx.Build.duration_ms / 1000))s.") + $lines.Add("**Build log:** ``$($Ctx.Build.log_path)``") + $lines.Add("") + $lines.Add("If this was a legitimate hang, investigate. If it was a slow build host, re-run with ``-BuildTimeoutMinutes `` or ``-AllowInconclusiveBuild``.") + $lines.Add("") + } + 'toolchain-missing' { + $lines.Add("**Required toolsets:** $($Ctx.Preflight.required_toolsets -join ', ')") + $lines.Add("**Available toolsets:** $($Ctx.Preflight.available_toolsets -join ', ')") + $lines.Add("**Missing:** $($Ctx.Preflight.missing -join ', ')") + $lines.Add("") + $lines.Add("This is an **infrastructure** problem — provision the host with the required Visual Studio toolset(s). No GitHub issue was opened because PR review cannot fix it.") + $lines.Add("") + } + } + $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") + $lines.Add("") + $lines.Add("**How to resume:**") + $lines.Add("") + $lines.Add("1. ``git switch $($Ctx.Branch)``") + $lines.Add("2. Fix the issue above (e.g. resw dedup, restored fork invariant, build fix, host provisioning).") + $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it.") + $lines.Add("4. Clear the lock:") + $lines.Add(" ``````") + $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") + $lines.Add(" ``````") + $lines.Add("5. The next scheduled sync runs the same range — validation must pass before any PR is opened.") + $lines.Add("") +} + $lines.Add("---") $lines.Add("") $lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``._") -$suffix = if ($Status -eq 'skipped-locked') { 'skipped' } elseif ($Status -eq 'stuck') { 'stuck' } elseif ($Status -eq 'no-op') { 'noop' } else { '' } +$suffix = if ($Status -eq 'skipped-locked') { 'skipped' } + elseif ($Status -like 'stuck-*') { $Status } + elseif ($Status -eq 'stuck') { 'stuck' } + elseif ($Status -eq 'no-op') { 'noop' } + else { '' } $name = Format-ReportFilename -When $started -Suffix $suffix $path = Join-Path (Get-ReportsDir) $name [System.IO.File]::WriteAllText($path, ($lines -join "`n"), (New-Object System.Text.UTF8Encoding($false))) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 8167a0b03..50be9f7c0 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -83,7 +83,7 @@ $title = "chore(upstream): sync microsoft/terminal up to $shortTo" # with "Head sha can't be blank" right after a push (see SKILL.md gotcha). $prUrl = $null for ($attempt = 1; $attempt -le 3; $attempt++) { - $prUrl = gh pr create --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 + $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } Write-Warning "gh pr create attempt $attempt failed: $prUrl" Start-Sleep -Seconds 5 @@ -101,7 +101,7 @@ Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue # it lands all N commits flatly on main once CI + approvals pass. if ($AutoMergeStrategy -ne 'none') { $strategyFlag = "--$AutoMergeStrategy" - gh pr merge $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host + gh pr merge -R microsoft/intelligent-terminal $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host if ($LASTEXITCODE -ne 0) { Write-Warning "gh pr merge --auto failed. PR is open at $($Ctx.PrUrl); merge manually with '$AutoMergeStrategy' strategy (NOT squash)." } else { diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index 5518db754..a60bdb3b4 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -50,7 +50,8 @@ $tmp = New-TemporaryFile # Ensure label exists (best-effort; ignore if already present). gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' 2>$null | Out-Null -$issueUrl = gh issue create --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 +# -R is explicit because the `upstream` remote can make gh default to microsoft/terminal. +$issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 Remove-Item $tmp -Force if ($LASTEXITCODE -ne 0 -or $issueUrl -notmatch '^https://github.com/') { throw "gh issue create failed: $issueUrl" diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 new file mode 100644 index 000000000..0559a27ac --- /dev/null +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -0,0 +1,147 @@ +<# +.SYNOPSIS + Tier-4 (post-pick validation failure) stuck-issue opener. + + Counterpart to 07-open-stuck-issue.ps1 (which handles Tier-3 = a + cherry-pick stopped mid-pick on a real merge conflict). Tier-4 means + all picks completed cleanly but the static scan, toolchain preflight, + or try-build step said NO. + +.PARAMETER Ctx + Run context. Must have Branch set; uses Picked, Preflight, Scan, Build. + +.PARAMETER ReportPath + Absolute path to the stuck report markdown. + +.PARAMETER Kind + One of: 'static-scan', 'build-failed', 'build-inconclusive', + 'toolchain-missing'. Determines the issue title and report header. + +.OUTPUTS + Issue URL on stdout (and writes Ctx.IssueUrl + Ctx.StuckValidation). + + toolchain-missing is special: it's an INFRA problem (this host lacks + the required VS toolset), not a CODE problem. We do not open a + GitHub issue for it — issues are noise for things humans can't fix + from the PR side. Instead we set the stuck-lock so the scheduler + won't keep retrying, and the host owner notices via the next dev + run / monitoring. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] $Ctx, + [Parameter(Mandatory)] [string] $ReportPath, + [Parameter(Mandatory)] [ValidateSet('static-scan','build-failed','build-inconclusive','toolchain-missing')] [string] $Kind +) + +. "$PSScriptRoot/Common.ps1" + +# Compute a findings hash so re-runs of the same broken batch are detectable. +$findingsForHash = switch ($Kind) { + 'static-scan' { $Ctx.Scan.findings } + 'build-failed' { @(@{ exit_code = $Ctx.Build.exit_code; tail_excerpt = ($Ctx.Build.log_tail -split "`n" | Select-Object -Last 20) -join "`n" }) } + 'build-inconclusive' { @(@{ kind = 'inconclusive'; duration_ms = $Ctx.Build.duration_ms }) } + 'toolchain-missing' { @(@{ missing = $Ctx.Preflight.missing }) } +} +$findingsHash = Get-FindingsHash $findingsForHash + +# Establish base/head for this batch (best-effort; tolerate detached states). +$base = git rev-parse origin/main 2>$null +$head = git rev-parse HEAD 2>$null + +# Push the sync branch so the human can resume on it (even toolchain-missing — +# the picks are still useful artifacts for whoever owns the host). +git push -u origin $Ctx.Branch 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { + Write-Warning "Could not push sync branch — stuck-lock will still be set." +} + +# Build validation payload (stored in state.json regardless of whether an issue is filed). +$validation = [ordered] @{ + kind = $Kind + base = $base + head = $head + branch = $Ctx.Branch + range = @($Ctx.Picked) + findings_hash = $findingsHash + at = Format-Iso8601 $Ctx.StartedAt + issue_url = $null +} + +# For toolchain-missing we do NOT open an issue (infra problem, not code). +if ($Kind -ne 'toolchain-missing') { + $titleKindLabel = switch ($Kind) { + 'static-scan' { 'static scan' } + 'build-failed' { 'build failure' } + 'build-inconclusive' { 'build inconclusive (timeout)' } + } + $title = "Upstream sync stuck after $($Ctx.Picked.Count) clean picks: $titleKindLabel ($findingsHash)" + + $header = @" +🛑 **Upstream sync stopped after validation failed.** + +All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch +validation step said NO before any PR was opened. Stop reason: **$Kind**. + +The scheduler will keep skipping its runs until this is resolved and +``clear-stuck.ps1`` runs cleanly. No alarm — the lock is intentional. + +Sync branch: ``$($Ctx.Branch)`` (pushed to origin). +Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). + +--- + +"@ + $body = $header + (Get-Content -Raw -LiteralPath $ReportPath) + $tmp = New-TemporaryFile + [System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) + + # Ensure label exists (best-effort). + gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' 2>$null | Out-Null + + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 + Remove-Item $tmp -Force + if ($LASTEXITCODE -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed: $issueUrl" + } + $validation.issue_url = $issueUrl.Trim() + $Ctx.IssueUrl = $validation.issue_url +} + +$Ctx.StuckValidation = $validation + +# Write the lock onto origin/main. +git switch main | Out-Null +git pull --ff-only origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run." } + +$state = Read-State +$state.stuck_validation = $validation +$runSummary = [ordered] @{ + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host + status = "stuck-$Kind" + branch = $Ctx.Branch + issue_url = $validation.issue_url + findings_hash = $findingsHash + picked_count = $Ctx.Picked.Count +} +$state.last_run = $runSummary +$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 +Write-State $state + +# Copy report into main reports/ so it's discoverable without checking out the branch. +$reportName = Split-Path -Leaf $ReportPath +$reportOnMain = Join-Path (Get-ReportsDir) $reportName +if ($ReportPath -ne $reportOnMain) { Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force } + +git add -- (Get-StatePath) $reportOnMain | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git add failed for Tier-4 stuck state." } + +$msgIssueRef = if ($validation.issue_url) { " (#$($validation.issue_url -replace '.*/',''))" } else { ' (no issue — infra)' } +git commit -m "chore(upstream-sync): stuck post-pick: $Kind$msgIssueRef" | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git commit of Tier-4 stuck-lock failed — lock NOT set on origin/main." } +git push origin main | Out-Host +if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — Tier-4 stuck-lock local only." } + +return $validation.issue_url diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 new file mode 100644 index 000000000..2e4ea3495 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 @@ -0,0 +1,167 @@ +<# +.SYNOPSIS + Static breakage scan. Runs AFTER all cherry-picks succeed and BEFORE + push / PR creation. Catches "clean cherry-pick but broken content" + failures that git-level conflict detection misses (PR #220 audit). + +.DESCRIPTION + Two v1 checks (see references/static-scan.md for v2 deferred items): + + 1. Duplicate entries in *.resw files (baseline-diff — + only gates on NEW duplicates introduced by the pick range). + + 2. Fork invariants — regex patterns from references/fork-invariants.json + that must still match in the post-pick worktree. + +.PARAMETER BaseSha + Pre-pick base commit (usually origin/main at orchestrator start). Used + to compute baseline-diff for the resw check. Required. + +.PARAMETER HeadRef + Post-pick worktree ref (default: HEAD). + +.OUTPUTS + Emits a single JSON document on stdout. Exit code: + 0 = scan ran cleanly (findings may still be present — inspect JSON) + 20 = scan itself errored out (broken script, missing files, etc.) +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $BaseSha, + [string] $HeadRef = 'HEAD' +) + +. "$PSScriptRoot/Common.ps1" + +function Get-ReswDuplicateNames { + param([string] $Text) + if (-not $Text) { return @() } + $names = [System.Collections.Generic.List[string]]::new() + $re = [regex]'$null + if ($LASTEXITCODE -ne 0) { throw "git diff failed computing changed resw files." } + return @($out | Where-Object { $_ -like '*.resw' }) +} + +function Get-FileTextAtRef { + param([string] $Ref, [string] $Path) + $text = git show "${Ref}:$Path" 2>$null + if ($LASTEXITCODE -ne 0) { return $null } # file didn't exist at that ref + return ($text -join "`n") +} + +function Get-FileTextOnDisk { + param([string] $Path) + if (-not (Test-Path -LiteralPath $Path)) { return $null } + return [System.IO.File]::ReadAllText($Path) +} + +function Scan-ReswDuplicates { + param([string] $Base, [string] $Head) + $findings = @() + foreach ($f in (Get-ChangedReswFiles -Base $Base -Head $Head)) { + $baseText = Get-FileTextAtRef -Ref $Base -Path $f + $headText = if ($Head -eq 'HEAD') { Get-FileTextOnDisk -Path $f } else { Get-FileTextAtRef -Ref $Head -Path $f } + $baseDups = @(Get-ReswDuplicateNames -Text $baseText) + $headDups = @(Get-ReswDuplicateNames -Text $headText) + $newDups = @($headDups | Where-Object { $baseDups -notcontains $_ }) + $oldStill = @($headDups | Where-Object { $baseDups -contains $_ }) + if ($newDups.Count -gt 0) { + $findings += [ordered] @{ + check = 'resw-duplicate-keys' + severity = 'critical' + path = $f + detail = "$($newDups.Count) newly-duplicated entries (was $($baseDups.Count) at base)" + examples = @($newDups | Select-Object -First 5) + } + } + if ($oldStill.Count -gt 0) { + $findings += [ordered] @{ + check = 'resw-duplicate-keys' + severity = 'info' + path = $f + detail = "$($oldStill.Count) duplicate entries also present pre-pick (not blocking)" + examples = @($oldStill | Select-Object -First 5) + } + } + } + return ,$findings +} + +function Scan-ForkInvariants { + $findings = @() + $invPath = Join-Path (Split-Path -Parent $PSScriptRoot) 'references/fork-invariants.json' + if (-not (Test-Path -LiteralPath $invPath)) { + return ,@([ordered] @{ + check = 'fork-invariants' + severity = 'medium' + path = $invPath + detail = 'fork-invariants.json missing — cannot check fork-protected items' + }) + } + $doc = Get-Content -Raw -LiteralPath $invPath | ConvertFrom-Json + foreach ($inv in @($doc.invariants)) { + $absPath = Join-Path (Get-RepoRoot) $inv.path + if (-not (Test-Path -LiteralPath $absPath)) { + $findings += [ordered] @{ + check = 'fork-invariant' + severity = $inv.severity + id = $inv.id + path = $inv.path + detail = "protected file does not exist in worktree" + reason = $inv.reason + } + continue + } + $text = [System.IO.File]::ReadAllText($absPath) + $re = [regex]::new($inv.must_contain_regex) + if (-not $re.IsMatch($text)) { + $findings += [ordered] @{ + check = 'fork-invariant' + severity = $inv.severity + id = $inv.id + path = $inv.path + detail = "regex '$($inv.must_contain_regex)' did not match in post-pick file" + reason = $inv.reason + } + } + } + return ,$findings +} + +try { + $findings = @() + $findings += Scan-ReswDuplicates -Base $BaseSha -Head $HeadRef + $findings += Scan-ForkInvariants + + $summary = [ordered] @{ + critical = @($findings | Where-Object { $_.severity -eq 'critical' }).Count + high = @($findings | Where-Object { $_.severity -eq 'high' }).Count + medium = @($findings | Where-Object { $_.severity -eq 'medium' }).Count + low = @($findings | Where-Object { $_.severity -eq 'low' }).Count + info = @($findings | Where-Object { $_.severity -eq 'info' }).Count + } + $blocking = ($summary.critical + $summary.high) -gt 0 + + $doc = [ordered] @{ + base = $BaseSha + head = $HeadRef + findings = @($findings) + summary = $summary + blocking = $blocking + } + $doc | ConvertTo-Json -Depth 8 + exit 0 +} +catch { + Write-Error $_.Exception.Message + Write-Error $_.ScriptStackTrace + exit 20 +} diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 new file mode 100644 index 000000000..f838faaff --- /dev/null +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -0,0 +1,106 @@ +<# +.SYNOPSIS + Toolchain preflight. Detects the required PlatformToolset from the + repo's build props and verifies it's installed on this host. + +.DESCRIPTION + Returns a JSON document. Used by the orchestrator BEFORE try-build + to distinguish "code broke the build" from "host doesn't have the + toolset". Critical for unattended schedulers — an infra problem + must not be filed as a code-stuck issue. + + Does NOT auto-bump v143→v145 or any other toolset. That recipe is + kept as a local-only developer workaround. + +.OUTPUTS + JSON to stdout: + { + "required_toolsets": ["v143"], + "available_toolsets": ["v143", "v145"], + "missing": [], + "vs_installs": ["C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise"], + "ok": true + } + + Exit codes: + 0 = preflight ran (inspect JSON for ok=true/false) + 20 = preflight itself errored out +#> +[CmdletBinding()] +param() + +. "$PSScriptRoot/Common.ps1" + +function Get-RequiredToolsets { + $root = Get-RepoRoot + $candidates = @( + 'src/common.build.pre.props', + 'src/common.openconsole.props', + 'src/common.build.tests.props' + ) + $found = [System.Collections.Generic.HashSet[string]]::new() + foreach ($rel in $candidates) { + $p = Join-Path $root $rel + if (-not (Test-Path -LiteralPath $p)) { continue } + $text = [System.IO.File]::ReadAllText($p) + foreach ($m in ([regex]']*>([^<]+)').Matches($text)) { + $val = $m.Groups[1].Value.Trim() + # Skip MSBuild property references like $(DefaultPlatformToolset) — those resolve at build time. + if ($val -and $val -notmatch '^\$\(') { [void]$found.Add($val) } + } + } + return ,@($found) +} + +function Get-VsInstalls { + $vswhere = Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio\Installer\vswhere.exe' + if (-not (Test-Path -LiteralPath $vswhere)) { return ,@() } + $out = & $vswhere -all -products * -property installationPath 2>$null + if ($LASTEXITCODE -ne 0) { return ,@() } + return ,@($out | Where-Object { $_ }) +} + +function Get-AvailableToolsets { + param([string[]] $VsInstalls) + $found = [System.Collections.Generic.HashSet[string]]::new() + foreach ($inst in $VsInstalls) { + # The authoritative location for installed platform toolsets is + # MSBuild\Microsoft\VC\\Platforms\\PlatformToolsets\. + # The bare `MSBuild\Microsoft\VC\v???` directories are MSBuild target + # versions, not toolsets — older versions of this script confused the two. + $vcRoot = Join-Path $inst 'MSBuild\Microsoft\VC' + if (-not (Test-Path -LiteralPath $vcRoot)) { continue } + Get-ChildItem -Directory -LiteralPath $vcRoot -ErrorAction SilentlyContinue | ForEach-Object { + $ptRoot = Join-Path $_.FullName 'Platforms\x64\PlatformToolsets' + if (Test-Path -LiteralPath $ptRoot) { + Get-ChildItem -Directory -LiteralPath $ptRoot -ErrorAction SilentlyContinue | ForEach-Object { + if ($_.Name -match '^v\d{3}$') { [void]$found.Add($_.Name) } + } + } + } + } + return ,@($found) +} + +try { + $required = Get-RequiredToolsets + $vsInstalls = Get-VsInstalls + $available = Get-AvailableToolsets -VsInstalls $vsInstalls + $missing = @($required | Where-Object { $available -notcontains $_ }) + $ok = ($missing.Count -eq 0) -and ($required.Count -gt 0 -or $vsInstalls.Count -gt 0) + + $doc = [ordered] @{ + required_toolsets = @($required) + available_toolsets = @($available) + missing = @($missing) + vs_installs = @($vsInstalls) + ok = $ok + } + $doc | ConvertTo-Json -Depth 4 + exit 0 +} +catch { + Write-Error $_.Exception.Message + Write-Error $_.ScriptStackTrace + exit 20 +} diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 new file mode 100644 index 000000000..7a9813886 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -0,0 +1,113 @@ +<# +.SYNOPSIS + Try build. Runs the configured build command in a razzle environment + and captures the result. Default: `cmd /c "tools\razzle.cmd && bz no_clean"`. + +.PARAMETER BuildCommand + Override the default build command. Must be a string the cmd.exe + shell can execute (razzle is cmd-based; PowerShell-keyword chaining + like 'if' won't work — keep it cmd-shell-friendly with &&). + +.PARAMETER TimeoutMinutes + Wall-clock cap. Default 45. On timeout, the build is killed and the + result is classified as 'build-inconclusive'. + +.PARAMETER LogDir + Where to write the full build log. Default: + /.github/upstream-sync/build-logs/. + +.OUTPUTS + JSON on stdout: + { + "kind": "build-ok" | "build-failed" | "build-inconclusive", + "exit_code": , + "duration_ms": , + "command": "", + "log_path": "", + "log_tail": "" + } + + Exit codes: + 0 = wrapper ran (inspect JSON for build status) + 20 = wrapper itself errored out (couldn't start the build at all) +#> +[CmdletBinding()] +param( + [string] $BuildCommand = 'tools\razzle.cmd && bz no_clean', + [int] $TimeoutMinutes = 45, + [string] $LogDir +) + +. "$PSScriptRoot/Common.ps1" + +try { + $root = Get-RepoRoot + if (-not $LogDir) { + $LogDir = Join-Path $root '.github/upstream-sync/build-logs' + } + if (-not (Test-Path -LiteralPath $LogDir)) { New-Item -ItemType Directory -Path $LogDir -Force | Out-Null } + + $stamp = (Get-Date).ToString('yyyy-MM-ddTHHmmss') + $logPath = Join-Path $LogDir "$stamp.log" + + $cmdLine = "/c `"cd /d `"$root`" && $BuildCommand`"" + $started = Get-Date + + # Use Start-Process with redirection so we can both tail and tee. + $psi = New-Object System.Diagnostics.ProcessStartInfo + $psi.FileName = $env:ComSpec + $psi.Arguments = $cmdLine + $psi.WorkingDirectory = $root + $psi.RedirectStandardOutput = $true + $psi.RedirectStandardError = $true + $psi.UseShellExecute = $false + $psi.CreateNoWindow = $true + + $proc = [System.Diagnostics.Process]::Start($psi) + + # Tee stdout/stderr into the log file as the build runs. + $writer = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) + $proc.add_OutputDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) + $proc.add_ErrorDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) + $proc.BeginOutputReadLine() + $proc.BeginErrorReadLine() + + $timeoutMs = $TimeoutMinutes * 60 * 1000 + $exited = $proc.WaitForExit($timeoutMs) + $kind = $null + $exitCode = $null + + if (-not $exited) { + try { $proc.Kill($true) } catch { } + $kind = 'build-inconclusive' + $exitCode = -1 + } else { + $exitCode = $proc.ExitCode + $kind = if ($exitCode -eq 0) { 'build-ok' } else { 'build-failed' } + } + + $writer.Flush(); $writer.Close() + $ended = Get-Date + $durationMs = [int]($ended - $started).TotalMilliseconds + + # Capture the last ~200 lines for the report / stuck issue. + $tailLines = if (Test-Path -LiteralPath $logPath) { + @(Get-Content -LiteralPath $logPath -Tail 200) -join "`n" + } else { '' } + + $doc = [ordered] @{ + kind = $kind + exit_code = $exitCode + duration_ms = $durationMs + command = $BuildCommand + log_path = $logPath + log_tail = $tailLines + } + $doc | ConvertTo-Json -Depth 4 + exit 0 +} +catch { + Write-Error $_.Exception.Message + Write-Error $_.ScriptStackTrace + exit 20 +} diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 72acb76ee..520b9977b 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -82,19 +82,35 @@ function Format-ReportFilename { function New-RunContext { [pscustomobject] @{ - StartedAt = Get-Date - Host = $env:COMPUTERNAME - Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))" - Picked = @() - DroppedPairs= @() - SkippedEmpty= @() - Tier0 = @() - Tier2 = @() - StuckSha = $null - StuckPaths = @() - Status = 'unknown' - ReportPath = $null - PrUrl = $null - IssueUrl = $null + StartedAt = Get-Date + Host = $env:COMPUTERNAME + Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))" + Picked = @() + DroppedPairs = @() + SkippedEmpty = @() + Tier0 = @() + Tier2 = @() + StuckSha = $null + StuckPaths = @() + # Tier-4 (post-pick validation failure): + StuckValidation = $null # hashtable { kind, base, head, range, findings_hash, branch, at, issue_url, ... } + # Validation step results (whether or not they blocked): + Preflight = $null # JSON from 09-toolchain-preflight.ps1 + Scan = $null # JSON from 08-static-scan.ps1 + Build = $null # JSON from 10-try-build.ps1 + Status = 'unknown' + ReportPath = $null + PrUrl = $null + IssueUrl = $null } } + +function Get-FindingsHash { + param([Parameter(Mandatory)] $Findings) + # Stable hash of a findings list — used as a stuck_validation.findings_hash + # so repeat-runs of the same broken batch can be detected (and not re-issued). + $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) + $sha = [System.Security.Cryptography.SHA256]::Create() + $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) + return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) +} diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index 4f83e6c89..d1a03b89f 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -1,28 +1,42 @@ <# .SYNOPSIS - Clear the stuck-lock after a human has merged the manual-resolution PR. + Clear the stuck-lock (Tier-3 or Tier-4) after a human has resolved + the underlying issue. .DESCRIPTION - Validates that -ResolvedThroughSha is an ancestor of upstream/main and - is at least as new as the stuck SHA, then advances last_synced to it - and clears stuck_on_sha / stuck_branch / stuck_at / stuck_issue_url. - Commits state.json on main. + Detects which kind of stuck is currently set in state.json and clears it: + + - Tier-3 (stuck_on_sha set): the scheduler stopped mid-cherry-pick on + a real merge conflict. -ResolvedThroughSha is REQUIRED and is + validated to be on upstream/main and >= stuck_on_sha; it becomes + the new last_synced_upstream_sha. + + - Tier-4 (stuck_validation set): the picks were clean but post-pick + validation failed (static scan / build / toolchain). The human + fixed it (e.g. resw dedup, restored fork invariant, fixed build, + or provisioned the missing toolset). -ResolvedThroughSha is + OPTIONAL — if omitted, last_synced_upstream_sha is left as-is so + the next scheduler run re-attempts the same range; if provided, + it advances the watermark just like Tier-3. .PARAMETER ResolvedThroughSha - The upstream SHA the manual-resolution PR brought the fork up to. - This becomes the new last_synced_upstream_sha. Typically this is the - same SHA that was stuck — the next scheduled run picks up from - ResolvedThroughSha + 1. + See above. + +.PARAMETER Reason + Optional human-readable note recorded in the history entry. #> [CmdletBinding()] param( - [Parameter(Mandatory)] [string] $ResolvedThroughSha + [string] $ResolvedThroughSha, + [string] $Reason = '' ) . "$PSScriptRoot/Common.ps1" $state = Read-State -if (-not $state.stuck_on_sha) { +$tier3 = [bool] $state.stuck_on_sha +$tier4 = [bool] $state.stuck_validation +if (-not ($tier3 -or $tier4)) { Write-Warning "No stuck-lock is set. Nothing to clear." return } @@ -30,32 +44,56 @@ if (-not $state.stuck_on_sha) { Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null -# Validate the new SHA is on upstream/main. -$null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main -if ($LASTEXITCODE -ne 0) { - throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." -} +git switch main | Out-Null +git pull --ff-only | Out-Null -# Validate it's >= the stuck SHA (i.e., stuck is ancestor of resolved). -$null = git merge-base --is-ancestor $state.stuck_on_sha $ResolvedThroughSha -if ($LASTEXITCODE -ne 0) { - throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $ResolvedThroughSha. Refusing — pass the same SHA or a later one." +if ($tier3) { + if (-not $ResolvedThroughSha) { + throw "Tier-3 stuck_on_sha is set ($($state.stuck_on_sha)) — -ResolvedThroughSha is required to clear it." + } + $null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main + if ($LASTEXITCODE -ne 0) { + throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." + } + $null = git merge-base --is-ancestor $state.stuck_on_sha $ResolvedThroughSha + if ($LASTEXITCODE -ne 0) { + throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $ResolvedThroughSha. Refusing — pass the same SHA or a later one." + } + $state.last_synced_upstream_sha = $ResolvedThroughSha + $state.stuck_on_sha = $null + $state.stuck_branch = $null + $state.stuck_at = $null + $state.stuck_issue_url = $null } -git switch main | Out-Null -git pull --ff-only | Out-Null +if ($tier4) { + if ($ResolvedThroughSha) { + $null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main + if ($LASTEXITCODE -ne 0) { + throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." + } + $state.last_synced_upstream_sha = $ResolvedThroughSha + } + $state.stuck_validation = $null +} -$state.last_synced_upstream_sha = $ResolvedThroughSha -$state.stuck_on_sha = $null -$state.stuck_branch = $null -$state.stuck_at = $null -$state.stuck_issue_url = $null +# Append a history note so we can see when locks were cleared. +$entry = [ordered] @{ + at = Format-Iso8601 + host = $env:COMPUTERNAME + status = if ($tier3 -and $tier4) { 'cleared-stuck (tier3+tier4)' } + elseif ($tier3) { 'cleared-stuck (tier3)' } + else { 'cleared-stuck (tier4)' } + advanced_to = $ResolvedThroughSha + reason = $Reason +} +$state.history = @($entry) + @($state.history) | Select-Object -First 20 Write-State $state git add -- (Get-StatePath) | Out-Null -git commit -m "chore(upstream-sync): clear stuck-lock at $($ResolvedThroughSha.Substring(0,9))" | Out-Host +$shortLabel = if ($ResolvedThroughSha) { $ResolvedThroughSha.Substring(0,9) } else { 'no-advance' } +git commit -m "chore(upstream-sync): clear stuck-lock ($shortLabel)" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } - git push origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — lock cleared locally only. Push manually before the next scheduler tick." } -Write-Host "Stuck-lock cleared. Next scheduled run will resume from $($ResolvedThroughSha.Substring(0,9))+1." -ForegroundColor Green +if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — lock cleared locally only. Push manually." } +Write-Host "Stuck-lock cleared." -ForegroundColor Green diff --git a/.github/upstream-sync/.gitignore b/.github/upstream-sync/.gitignore new file mode 100644 index 000000000..2d32d9453 --- /dev/null +++ b/.github/upstream-sync/.gitignore @@ -0,0 +1 @@ +build-logs/ From be2542de783153cd8c4c94c774e786f1a8c2488d Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 16:06:20 +0800 Subject: [PATCH 03/82] fix(upstream-sync/scan): use absolute paths in resw dedup scan (PR #220 audit miss) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Get-FileTextOnDisk used [System.IO.File]::ReadAllText with a relative path. .NET resolves relative paths against [System.Environment]::CurrentDirectory, NOT PowerShell's $PWD, so the scan was silently reading from `Q:\official\intelligent-terminal` (the unrelated main worktree) while the orchestrator was cd'd into `Q:\official\it-sync` (the sync worktree with the duplicates). PR #220 symptom: the scan returned locking=false / 0 critical even though the cherry-pick batch had introduced 63 new duplicate `` entries across 6 pseudo-locale files, and the build subsequently failed with PRI175 / PRI277. Fix: - Get-FileTextOnDisk: resolve to absolute via Join-Path (Get-RepoRoot) when the input is relative. - Get-FileTextAtRef: capture `git show` output through a temp file via `cmd /c "git show ref:path > tmp 2>nul"` instead of PowerShell's pipeline, which truncates/mangles high-Unicode pseudo-locale bytes when the subprocess's stdout binds to PSObject string formatting. Verified against PR #220 commits: - HEAD = fd8332bba (the buggy state): 6 critical findings, blocking=true ✅ - HEAD = d6b4dc3ac (the fixed state): 0 critical, blocking=false ✅ Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/references/static-scan.md | 22 +++++++++++++++ .../upstream-sync/scripts/08-static-scan.ps1 | 27 +++++++++++++++---- 2 files changed, 44 insertions(+), 5 deletions(-) diff --git a/.github/skills/upstream-sync/references/static-scan.md b/.github/skills/upstream-sync/references/static-scan.md index 4a6298f1e..6bb01cba4 100644 --- a/.github/skills/upstream-sync/references/static-scan.md +++ b/.github/skills/upstream-sync/references/static-scan.md @@ -34,6 +34,28 @@ Severity: - `critical` — any new duplicate key introduced by the pick range. - `info` — pre-existing duplicates carried forward. +> **Implementation note (PR #220 follow-up).** Both `Get-FileTextAtRef` +> and `Get-FileTextOnDisk` MUST use absolute paths when calling +> `[System.IO.File]::ReadAllText`. .NET resolves relative paths against +> `[System.Environment]::CurrentDirectory`, NOT PowerShell's `$PWD`, so +> a relative path silently reads from a different worktree. PR #220's +> first dedup pass returned `blocking=false` while the build was still +> broken because of this exact bug — the scan was reading the dup-free +> `intelligent-terminal` main worktree instead of the dup-laden sync +> worktree. Same caveat for `git show` — capture via `cmd /c "git show +> ref:path > tmp 2>nul"` instead of PowerShell-pipeline join, otherwise +> high-Unicode (pseudo-locale) bytes get truncated by the PSObject +> formatter. + +> **Operator note: manual dedup regex.** When fixing duplicates by hand +> (the orchestrator does NOT auto-dedup), the matching regex must be +> CRLF-agnostic: `(?s)([ \t]*)]*>.*?(\r?\n)?`. +> A `\r?\n`-anchored regex misses inline-concatenated entries +> (`......`) that the loc-bot +> commits as a single appended line for fork-only keys in +> `qps-ploc/qps-ploca/qps-plocm` — exactly the second failure mode that +> blocked PR #220 build retry #4. + ### 2. Fork invariants (regex must-match) Reads `references/fork-invariants.json`. For each entry: diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 index 2e4ea3495..3e922a28f 100644 --- a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 +++ b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 @@ -52,15 +52,32 @@ function Get-ChangedReswFiles { function Get-FileTextAtRef { param([string] $Ref, [string] $Path) - $text = git show "${Ref}:$Path" 2>$null - if ($LASTEXITCODE -ne 0) { return $null } # file didn't exist at that ref - return ($text -join "`n") + # Capture via a temp file to avoid PowerShell mangling binary-ish output + # (UTF-8 BOM, mixed CRLF/LF, high-Unicode pseudo-locale glyphs) when the + # subprocess's stdout is bound to a PSObject pipeline. + $tmp = [System.IO.Path]::GetTempFileName() + try { + & cmd /c "git show ""${Ref}:$Path"" > ""$tmp"" 2>nul" + if ($LASTEXITCODE -ne 0) { return $null } + return [System.IO.File]::ReadAllText($tmp) + } finally { + Remove-Item -LiteralPath $tmp -ErrorAction SilentlyContinue + } } function Get-FileTextOnDisk { param([string] $Path) - if (-not (Test-Path -LiteralPath $Path)) { return $null } - return [System.IO.File]::ReadAllText($Path) + # IMPORTANT: [System.IO.File]::* APIs resolve relative paths against + # [Environment]::CurrentDirectory, NOT PowerShell's $PWD. Any relative + # path passed in here would silently read from the wrong worktree (the + # PR #220 audit miss). Resolve to absolute against the repo root. + if ([System.IO.Path]::IsPathRooted($Path)) { + $abs = $Path + } else { + $abs = Join-Path (Get-RepoRoot) $Path + } + if (-not (Test-Path -LiteralPath $abs)) { return $null } + return [System.IO.File]::ReadAllText($abs) } function Scan-ReswDuplicates { From 08a6c16487bb4f0eb5ec8cbd4dd3c1e9dbd45fa7 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 17:34:09 +0800 Subject: [PATCH 04/82] fix(upstream-sync): address review feedback Update skill docs to match same-repo PR creation, direct-push mode, and pinned upstream author/committer dates. Harden native git error checks and empty cherry-pick skip validation, keep state writes newline-terminated, seed the Tier-4 state field, and route stable spelling identifiers through patterns. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../actions/spelling/patterns/patterns.txt | 5 ++++ .github/skills/upstream-sync/SKILL.md | 21 ++++++++------- .../references/conflict-triage.md | 2 +- .../upstream-sync/references/static-scan.md | 8 +++--- .../upstream-sync/references/workflow.md | 2 +- .../scripts/03-cherry-pick-one.ps1 | 26 ++++++++++--------- .../scripts/06b-finalize-direct.ps1 | 5 ++-- .../skills/upstream-sync/scripts/Common.ps1 | 2 +- .../upstream-sync/scripts/clear-stuck.ps1 | 3 +++ .github/upstream-sync/state.json | 3 ++- 10 files changed, 44 insertions(+), 33 deletions(-) diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index e16ba9e70..8be4bbaf0 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -320,3 +320,8 @@ env_remove\("[A-Z_]+"\) # GitHub GraphQL node ID prefix for PullRequestReviewThread \bPRRT_[A-Za-z0-9_-]+\b + +# upstream-sync skill: PowerShell variable names, git/config tokens, toolchain labels, and timestamp placeholders +\b(?:ACMR|MFC)\b +\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables|[Tt]oolsets)\b +\b(?:DDTH(?:mmss)?|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 9c14c6042..199150f9f 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -1,7 +1,7 @@ --- name: upstream-sync description: 'Periodically sync new commits from microsoft/terminal into this manually-forked intelligent-terminal repo by cherry-picking commit-by-commit onto a dated sync branch, auto-skipping revert pairs and empty commits, auto-resolving known take-upstream files, and stopping cleanly on genuine conflicts with a written report and a GitHub issue. Use when the user asks to "sync upstream", "pull from microsoft/terminal", "run upstream sync", "catch up to upstream", or wires this into a scheduler (weekly/daily). Designed to be safe under repeated unattended runs.' -license: Complete terms in LICENSE.txt +license: MIT --- # Upstream Sync (microsoft/terminal → intelligent-terminal) @@ -98,15 +98,15 @@ pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -BuildTimeoutMinutes ### Finalize modes — what each preserves -| Mode | Per-commit content | Order on main | Original author date | Reviewer checkpoint | Requires admin? | +| Mode | Per-commit content | Order on main | Original author + committer dates | Reviewer checkpoint | Requires admin? | |---|---|---|---|---|---| | PR + **rebase-merge** | ✅ | ✅ | ✅ | ✅ | No | | PR + **merge commit** | ✅ | ✅ | ✅ | ✅ | No | | PR + **squash** | ❌ collapsed | ❌ | ⚠️ folded | ✅ | No | | **`-PushDirectToMain`** | ✅ | ✅ | ✅ | ❌ | Yes (push to main) | -Committer date is "now" in every mode (git default for cherry-pick) — -that's the semantically correct "when this fork landed it" timestamp. +The cherry-pick loop pins both author and committer identity/dates to the +upstream commit so audit timestamps match the original commit-by-commit history. Resumability is built into the state file — re-running after a successful run is a fast no-op (nothing pending), and re-running while the stuck-lock @@ -129,10 +129,10 @@ is set exits early without touching the branch. [references/known-conflicts.md](./references/known-conflicts.md) handles this automatically — extend the list when you discover the next file with the same pattern. -- **`gh pr create` on Windows fails with "Head sha can't be blank"** if the - branch is freshly pushed and not yet visible. The finalize script uses - `--head :` and a 5s retry to work around this — do not - "fix" it by removing the retry. +- **`gh pr create` on Windows can fail with "Head sha can't be blank"** if the + branch is freshly pushed and not yet visible. The same-repo finalize script + intentionally uses `--head ` plus a 5s retry — do not "fix" it to + `--head :`, which would point `gh` at a fork. - **Do not run the scheduler twice while stuck.** The lock in `state.json` makes the second run a no-op, but a human running the script manually with `-Force` will overwrite the stuck branch and lose @@ -145,8 +145,9 @@ is set exits early without touching the branch. - **Always commit `state.json` and `reports/`** so the next scheduler invocation (possibly on a different machine) starts from the right checkpoint. The finalize PR includes the state update. -- **Never push directly to `main`.** The skill always opens a PR. Direct - push bypasses CI and human review of the upstream batch. +- **Prefer the PR path.** The default workflow always opens a PR so CI and + human review checkpoint the upstream batch. Use `-PushDirectToMain` only + for an explicit admin/bypass run where skipping PR latency is intentional. - **CRLF/LF on manifest files.** Cherry-picks normally preserve upstream line endings, but any in-flight resolution touched by an LLM may downgrade to LF. If a Tier-2 resolution touches a `.yml`/`.xml`/`.csproj`/winget diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/conflict-triage.md index 1af95b99c..8920c78fd 100644 --- a/.github/skills/upstream-sync/references/conflict-triage.md +++ b/.github/skills/upstream-sync/references/conflict-triage.md @@ -81,7 +81,7 @@ Tier 3. If it returns content, **verify with a second fresh agent**: > version. Does the resolution preserve all behavioral intent from both > sides? Respond `OK` or `NOT_OK: `. -Stage only if both agents agree `high`/`OK`. Otherwise → Tier 3. +Stage only if both agents agree `high`/`OK`. Otherwise, route to Tier 3. ## Tier 3 — Stop and escalate (cherry-pick conflict) diff --git a/.github/skills/upstream-sync/references/static-scan.md b/.github/skills/upstream-sync/references/static-scan.md index 6bb01cba4..71046a47b 100644 --- a/.github/skills/upstream-sync/references/static-scan.md +++ b/.github/skills/upstream-sync/references/static-scan.md @@ -25,14 +25,14 @@ Both were caught post-hoc by audit. This scan catches them pre-PR. For every `*.resw` file modified anywhere in the pick range, count duplicate `` entries in the **pre-pick** state vs the **post-pick** worktree. Gate on **newly-introduced** duplicates only — -pre-existing duplicates are reported as `info`, not blocking. +preexisting duplicates are reported as `info`, not blocking. Pre-pick state = the file content at `origin/main` (the orchestrator's base before the picks). Post-pick = the worktree on the sync branch. Severity: - `critical` — any new duplicate key introduced by the pick range. -- `info` — pre-existing duplicates carried forward. +- `info` — preexisting duplicates carried forward. > **Implementation note (PR #220 follow-up).** Both `Get-FileTextAtRef` > and `Get-FileTextOnDisk` MUST use absolute paths when calling @@ -43,7 +43,7 @@ Severity: > broken because of this exact bug — the scan was reading the dup-free > `intelligent-terminal` main worktree instead of the dup-laden sync > worktree. Same caveat for `git show` — capture via `cmd /c "git show -> ref:path > tmp 2>nul"` instead of PowerShell-pipeline join, otherwise +> ref:path > tmp 2>nul"` instead of PowerShell-pipeline join; otherwise > high-Unicode (pseudo-locale) bytes get truncated by the PSObject > formatter. @@ -130,6 +130,6 @@ To add a new check: 2. Wire it into the main loop in `08-static-scan.ps1`. 3. Document the check here. 4. Add a baseline-aware test (compare pre-pick vs post-pick) **only - when** the check would otherwise gate on pre-existing issues. + when** the check would otherwise gate on preexisting issues. 5. Wire its severities into the `blocking` calculation if it should block the run. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index b46e20a85..c0f561998 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -126,7 +126,7 @@ pwsh .github/skills/upstream-sync/scripts/08-static-scan.ps1 -BaseSha $preBase loop. The scan: - Baseline-diffs every changed `.resw` file for NEW duplicate `` - keys (pre-existing dups are reported as `info`, not blocking). + keys (preexisting dups are reported as `info`, not blocking). - Runs regex assertions from [`fork-invariants.json`](./fork-invariants.json) against the post-pick worktree. diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 40fe65f18..8614eeb6f 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -67,7 +67,12 @@ $result = [ordered] @{ # Note: this does NOT reproduce GPG signatures (we don't hold upstream's # keys). The fork doesn't enforce signed commits, so "committed by X but # unsigned" is acceptable. -$info = (git log -1 --format='%an%x09%ae%x09%aI%x09%cn%x09%ce%x09%cI' $Sha) -split "`t" +$fullSha = (git rev-parse $Sha).Trim() +if ($LASTEXITCODE -ne 0) { throw "Could not resolve upstream commit $Sha." } +$prePickHead = (git rev-parse HEAD).Trim() +if ($LASTEXITCODE -ne 0) { throw "Could not record pre-pick HEAD before cherry-picking $Sha." } + +$info = (git log -1 --format='%an%x09%ae%x09%aI%x09%cn%x09%ce%x09%cI' $fullSha) -split "`t" $env:GIT_AUTHOR_NAME = $info[0] $env:GIT_AUTHOR_EMAIL = $info[1] $env:GIT_AUTHOR_DATE = $info[2] @@ -78,22 +83,20 @@ $env:GIT_COMMITTER_DATE = $info[5] try { # Attempt the pick. -git cherry-pick --keep-redundant-commits -x $Sha 2>&1 | Out-Host +git cherry-pick --keep-redundant-commits -x $fullSha 2>&1 | Out-Host $pickCode = $LASTEXITCODE if ($pickCode -eq 0) { - # Defensive: ensure HEAD is the commit we just picked before any reset. - # If a future hook ever inserts work between pick and check, HEAD~1 - # would silently discard arbitrary commits. - $headTree = (git log -1 --format='%T' HEAD).Trim() - $pickedTree = (git log -1 --format='%T' $Sha).Trim() # Tier-1 check: did we just create an empty commit (allowed by --keep-redundant-commits)? $changed = git diff-tree --no-commit-id --name-only -r HEAD if (-not $changed) { - if ($headTree -ne $pickedTree) { - throw "Refusing to reset --hard HEAD~1: HEAD tree ($headTree) does not match picked commit's tree ($pickedTree). Investigate before retrying." + $commitMessage = (git log -1 --format='%B' HEAD) -join "`n" + $expectedFooter = "\(cherry picked from commit $([regex]::Escape($fullSha))\)" + if ($commitMessage -notmatch $expectedFooter) { + throw "Refusing to reset --hard ${prePickHead}: HEAD does not contain the cherry-pick footer for $fullSha. Investigate before retrying." } - git reset --hard HEAD~1 | Out-Null + git reset --hard $prePickHead | Out-Null + if ($LASTEXITCODE -ne 0) { throw "Failed to reset empty cherry-pick back to $prePickHead." } $result.status = 'skipped-empty' } else { $result.status = 'picked' @@ -128,8 +131,7 @@ if ($unhandled.Count -gt 0) { } # All conflicts handled by Tier-0; continue the pick (preserve original message). -$env:GIT_EDITOR = 'true' -git cherry-pick --continue 2>&1 | Out-Host +git cherry-pick --continue --no-edit 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { # Could still be empty after Tier-0. $staged = git diff --cached --name-only diff --git a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 index 73a48eabc..99ef9672e 100644 --- a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 +++ b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 @@ -4,9 +4,8 @@ Use when you have admin/bypass perms on main and want zero PR latency. .DESCRIPTION - Preserves per-commit content, order, and original author dates. - Committer date becomes "now" (git default for cherry-pick) — that's - the semantically correct "when this fork landed it" timestamp. + Preserves per-commit content, order, and original author + committer + dates, because the cherry-pick loop pins both from each upstream commit. Assumes the caller is currently on the sync branch with all picks applied. Performs: diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 520b9977b..111ff05d5 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -35,7 +35,7 @@ function Read-State { function Write-State { param([Parameter(Mandatory)] $State) $p = Get-StatePath - $json = $State | ConvertTo-Json -Depth 12 + $json = ($State | ConvertTo-Json -Depth 12) + [Environment]::NewLine # Use UTF-8 *without* BOM to match git's default text handling on this repo. [System.IO.File]::WriteAllText($p, $json, (New-Object System.Text.UTF8Encoding($false))) } diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index d1a03b89f..6613bd1d4 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -43,9 +43,12 @@ if (-not ($tier3 -or $tier4)) { Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed; refusing to clear stuck-lock against stale refs." } git switch main | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git switch main failed; refusing to clear stuck-lock from the wrong branch." } git pull --ff-only | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only failed; refusing to clear stuck-lock until main is current." } if ($tier3) { if (-not $ResolvedThroughSha) { diff --git a/.github/upstream-sync/state.json b/.github/upstream-sync/state.json index cd76cba7c..49c68d382 100644 --- a/.github/upstream-sync/state.json +++ b/.github/upstream-sync/state.json @@ -7,6 +7,7 @@ "stuck_branch": null, "stuck_at": null, "stuck_issue_url": null, + "stuck_validation": null, "last_run": null, "history": [] -} \ No newline at end of file +} From d6c077ece1435a3b8e1133ab00684704864a6f27 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 17:42:23 +0800 Subject: [PATCH 05/82] fix(upstream-sync): cover toolsets spelling identifier Extend the upstream-sync spelling pattern to cover underscore-prefixed toolsets identifiers emitted by the report script. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/patterns/patterns.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index 8be4bbaf0..688ab76c0 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -323,5 +323,5 @@ env_remove\("[A-Z_]+"\) # upstream-sync skill: PowerShell variable names, git/config tokens, toolchain labels, and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables|[Tt]oolsets)\b +\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables|(?:[a-z]+_)?[Tt]oolsets)\b \b(?:DDTH(?:mmss)?|Hmmss|sszzz)\b From 9837b73cddb73774d4499292b6277d0bd695f7b2 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 17:48:17 +0800 Subject: [PATCH 06/82] fix(upstream-sync): harden stuck-path operations Validate native git branch switches before stuck-lock writes, seed Tier-4 state during bootstrap, and wait for asynchronous build output callbacks before closing the build log writer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/00-bootstrap.ps1 | 6 +++++- .../skills/upstream-sync/scripts/07-open-stuck-issue.ps1 | 1 + .../scripts/07b-open-validation-stuck-issue.ps1 | 1 + .github/skills/upstream-sync/scripts/10-try-build.ps1 | 4 ++++ 4 files changed, 11 insertions(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 index 8872c0417..213003d5f 100644 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -49,6 +49,7 @@ $state = @{ stuck_branch = $null stuck_at = $null stuck_issue_url = $null + stuck_validation = $null last_run = $null history = @() } @@ -57,7 +58,10 @@ Write-State $state # Stage and commit on a dedicated branch so the human can open the PR. $branch = 'chore/upstream-sync-bootstrap' git switch -c $branch 2>$null -if ($LASTEXITCODE -ne 0) { git switch $branch | Out-Null } +if ($LASTEXITCODE -ne 0) { + git switch $branch | Out-Null + if ($LASTEXITCODE -ne 0) { throw "Could not create or switch to bootstrap branch '$branch'. Refusing to commit state.json on the current HEAD." } +} git add -- (Get-StatePath) if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed." } diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index a60bdb3b4..520203d99 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -60,6 +60,7 @@ $Ctx.IssueUrl = $issueUrl.Trim() # Set the stuck-lock on main (direct push — the lock is the gate; PR path is blocked). git switch main | Out-Null +if ($LASTEXITCODE -ne 0) { throw "Could not switch to main before writing stuck-lock. Resolve manually and re-run; the stuck branch + issue are already in place." } git pull --ff-only origin main | Out-Host if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run; the stuck branch + issue are already in place." } diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 0559a27ac..7bd779cc8 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -112,6 +112,7 @@ $Ctx.StuckValidation = $validation # Write the lock onto origin/main. git switch main | Out-Null +if ($LASTEXITCODE -ne 0) { throw "Could not switch to main before writing stuck-lock. Resolve manually and re-run." } git pull --ff-only origin main | Out-Host if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run." } diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index 7a9813886..c8d2baf25 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -79,9 +79,13 @@ try { if (-not $exited) { try { $proc.Kill($true) } catch { } + $proc.WaitForExit() $kind = 'build-inconclusive' $exitCode = -1 } else { + # WaitForExit(timeout) can return before async output callbacks drain. + # The parameterless wait completes only after redirected output events finish. + $proc.WaitForExit() $exitCode = $proc.ExitCode $kind = if ($exitCode -eq 0) { 'build-ok' } else { 'build-failed' } } From a22edd5d256b245e5754775d7e8c17c5d6213401 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 17:55:10 +0800 Subject: [PATCH 07/82] fix(upstream-sync): align reporting docs Use the static-scan finding check field in Tier-4 reports and update reference docs/templates to match the report generator and implemented Tier-0 strategies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../references/build-verification.md | 4 +-- .../references/known-conflicts.md | 7 ++--- .../upstream-sync/references/reporting.md | 26 +++++++++---------- .../upstream-sync/references/workflow.md | 2 +- .../upstream-sync/scripts/05-write-report.ps1 | 3 ++- 5 files changed, 22 insertions(+), 20 deletions(-) diff --git a/.github/skills/upstream-sync/references/build-verification.md b/.github/skills/upstream-sync/references/build-verification.md index ed9e23555..e14baa28b 100644 --- a/.github/skills/upstream-sync/references/build-verification.md +++ b/.github/skills/upstream-sync/references/build-verification.md @@ -50,8 +50,8 @@ cmd.exe /c "tools\razzle.cmd && bz no_clean" (`bz no_clean` = incremental Debug build of the full solution.) Configurable via the orchestrator's `-BuildCommand` parameter. The -default is verified on the maintainer host and documented in the -state.json `last_run.build_command` field for traceability. +default is verified on the maintainer host; if validation blocks the run, +the generated Tier-4 diagnostics include the build log path and tail. Output: diff --git a/.github/skills/upstream-sync/references/known-conflicts.md b/.github/skills/upstream-sync/references/known-conflicts.md index b4f576e0c..f56d645e4 100644 --- a/.github/skills/upstream-sync/references/known-conflicts.md +++ b/.github/skills/upstream-sync/references/known-conflicts.md @@ -7,8 +7,9 @@ cherry-pick loop parses this file and auto-resolves matching paths. ## Format Each entry is an H2 path heading, a one-line `Strategy:` line, and a -"Why" paragraph. The parser reads the heading text as the literal path -and the `Strategy:` token as one of `take-upstream` | `take-ours` | `union`. +"Why" paragraph. The parser reads the heading text as the literal path. +Implemented Tier-0 strategies are `take-upstream` and `take-ours`; +`union` is reserved and currently escalates instead of auto-resolving. --- @@ -33,7 +34,7 @@ the manual resolution PR merges. Format: ```markdown ## `` -**Strategy:** `take-upstream` +**Strategy:** `take-upstream` **Why:** diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md index c64e777d2..c4a6e2bfd 100644 --- a/.github/skills/upstream-sync/references/reporting.md +++ b/.github/skills/upstream-sync/references/reporting.md @@ -28,28 +28,28 @@ as the PR description (success path) and the issue body (stuck path). ## Picked commits (oldest → newest) -| # | SHA | Subject | Files | Author | -|---|---|---|---|---| -| 1 | | | | | -| ... | | | | | +| # | SHA | Subject | Author | +|---|---|---|---| +| 1 | | | | +| ... | | | | ## Dropped revert pairs -| Original SHA | Original subject | Revert SHA | Detected via | -|---|---|---|---| -| | | | "Revert" prefix / "This reverts commit" body | +| Original SHA | Original subject | Revert SHA | +|---|---|---| +| | | | ## Empty / no-op commits skipped -| SHA | Subject | Reason | -|---|---|---| -| | | upstream empty / already applied | +| SHA | Subject | +|---|---| +| | | ## Tier-0 auto-resolutions -| Commit SHA | File | Strategy | -|---|---|---| -| | `.github/workflows/spelling2.yml` | take-upstream | +| Commit SHA | File | +|---|---| +| | `.github/workflows/spelling2.yml` | ## (Stuck only) Conflict diagnostics diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index c0f561998..b55edb6d3 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -156,7 +156,7 @@ write `.github/upstream-sync/reports/YYYY-MM-DDTHHmm[-suffix].md` with: - Run metadata (start, end, duration, host, status) - Counts: picked / dropped-pair / empty / known-conflict-resolved / stuck-at -- For each picked commit: SHA, subject, author, files-touched count +- For each picked commit: SHA, subject, author - For dropped pairs: the two SHAs and their subjects - If stuck (Tier-3): the conflicting commit, the conflicting paths, what was attempted, the exact resume command - If stuck (Tier-4): the validation findings, the build log tail, the exact resume command diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 3f058b306..5ebb5f36a 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -163,8 +163,9 @@ if ($Status -like 'stuck-*') { $lines.Add("|---|---|---|---|") foreach ($f in $blocking) { $where = if ($f.path) { "``$($f.path)``" } else { '—' } + $findingKind = if ($f.check) { $f.check } elseif ($f.kind) { $f.kind } else { 'unknown' } $detail = ($f | ConvertTo-Json -Compress -Depth 4) - $lines.Add("| $($f.severity) | $($f.kind) | $where | ``$detail`` |") + $lines.Add("| $($f.severity) | $findingKind | $where | ``$detail`` |") } $lines.Add("") } From 67bada95f67e5bdea67a5e6642aee312f1692ca9 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:01:07 +0800 Subject: [PATCH 08/82] fix(upstream-sync): guard cleanup and build logging Serialize asynchronous build log writes, require a clean worktree before clearing stuck locks, and document upstream state fields as provenance rather than runtime configuration. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/references/state-schema.md | 3 +++ .github/skills/upstream-sync/scripts/10-try-build.ps1 | 6 ++++-- .github/skills/upstream-sync/scripts/clear-stuck.ps1 | 1 + 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md index 272fa8a3d..fab3d8c30 100644 --- a/.github/skills/upstream-sync/references/state-schema.md +++ b/.github/skills/upstream-sync/references/state-schema.md @@ -5,6 +5,9 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). ```jsonc { "version": 1, + + // Provenance for the baseline below. v1 scripts intentionally sync + // microsoft/terminal main; these fields are not runtime configuration. "upstream_remote_url": "https://github.com/microsoft/terminal.git", "upstream_branch": "main", diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index c8d2baf25..7d7432528 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -65,8 +65,10 @@ try { $proc = [System.Diagnostics.Process]::Start($psi) - # Tee stdout/stderr into the log file as the build runs. - $writer = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) + # Tee stdout/stderr into the log file as the build runs. The synchronized + # wrapper serializes concurrent stdout/stderr DataReceived callbacks. + $baseWriter = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) + $writer = [System.IO.TextWriter]::Synchronized($baseWriter) $proc.add_OutputDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) $proc.add_ErrorDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) $proc.BeginOutputReadLine() diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index 6613bd1d4..c2728dd4d 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -41,6 +41,7 @@ if (-not ($tier3 -or $tier4)) { return } +Assert-CleanWorktree Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed; refusing to clear stuck-lock against stale refs." } From a4f06122f78e72805356dfa45c9bcfa0d7feeb25 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:06:42 +0800 Subject: [PATCH 09/82] fix(upstream-sync): pull origin main explicitly Fast-forward main from origin/main in the batch and stuck-clear paths so local branch tracking configuration cannot change the sync baseline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/04-run-batch.ps1 | 4 ++-- .github/skills/upstream-sync/scripts/clear-stuck.ps1 | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index 555cb42a5..99b4d13d0 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -128,8 +128,8 @@ try { Assert-CleanWorktree git switch main 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } - git pull --ff-only 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only main failed." } + git pull --ff-only origin main 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only origin main failed." } # --- 1. Fetch upstream --- $toSha = (& "$PSScriptRoot/01-fetch-upstream.ps1").Trim() diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index c2728dd4d..6bf4bb9af 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -48,8 +48,8 @@ if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed; refusing to cl git switch main | Out-Null if ($LASTEXITCODE -ne 0) { throw "git switch main failed; refusing to clear stuck-lock from the wrong branch." } -git pull --ff-only | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only failed; refusing to clear stuck-lock until main is current." } +git pull --ff-only origin main | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed; refusing to clear stuck-lock until main is current." } if ($tier3) { if (-not $ResolvedThroughSha) { From ad61c937308ca70b51742a42925e24334d0fa4ea Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:13:25 +0800 Subject: [PATCH 10/82] fix(upstream-sync): make dry-run reports actionable Fail fast on a mismatched upstream remote, emit pending commits in dry-run reports, trim native git output before persistence, and keep report documentation aligned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/references/reporting.md | 9 ++++++++- .../upstream-sync/references/workflow.md | 2 +- .../upstream-sync/scripts/04-run-batch.ps1 | 3 ++- .../upstream-sync/scripts/05-write-report.ps1 | 18 +++++++++++++++++- .../scripts/07-open-stuck-issue.ps1 | 2 +- .../07b-open-validation-stuck-issue.ps1 | 6 ++++-- .../skills/upstream-sync/scripts/Common.ps1 | 3 ++- 7 files changed, 35 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md index c4a6e2bfd..4e7534e29 100644 --- a/.github/skills/upstream-sync/references/reporting.md +++ b/.github/skills/upstream-sync/references/reporting.md @@ -9,7 +9,7 @@ as the PR description (success path) and the issue body (stuck path). ```markdown # Upstream sync — — -**Status:** +**Status:** **Host:** **Duration:** **Baseline (before run):** `` () — @@ -26,6 +26,13 @@ as the PR description (success path) and the issue body (stuck path). - Tier-2 LLM resolutions: **** (only when -TryTier2) - Tier-3 escalation (stuck at): +## Pending commits (dry-run only, oldest → newest) + +| # | SHA | Subject | Author | +|---|---|---|---| +| 1 | | | | +| ... | | | | + ## Picked commits (oldest → newest) | # | SHA | Subject | Author | diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index b55edb6d3..9b58d6358 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -150,7 +150,7 @@ Skipped when `-SkipBuild`. Logs land in `.github/upstream-sync/build-logs/` ### 8. Write report (always) -Regardless of outcome (ok / no-op / stuck / stuck-static-scan / +Regardless of outcome (ok / no-op / dry-run / stuck / stuck-static-scan / stuck-build-failed / stuck-build-inconclusive / stuck-toolchain-missing), write `.github/upstream-sync/reports/YYYY-MM-DDTHHmm[-suffix].md` with: diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index 99b4d13d0..bb1069fbe 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -147,6 +147,7 @@ try { $pending = $pendingJson | ConvertFrom-Json Write-Host ("Pending: {0} commits, {1} revert pairs dropped, {2} empties dropped." -f $pending.pending.Count, $pending.dropped_pairs.Count, $pending.skipped_empty.Count) + $ctx.Pending = @($pending.pending) $ctx.DroppedPairs = @($pending.dropped_pairs) $ctx.SkippedEmpty = @($pending.skipped_empty) @@ -159,7 +160,7 @@ try { if ($DryRun) { Write-Host "DryRun: skipping branch creation and cherry-picks." -ForegroundColor Cyan - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'dry-run' Write-Host "DryRun report: $reportPath" exit 0 } diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 5ebb5f36a..eb45c1c3f 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -22,7 +22,7 @@ param( [Parameter(Mandatory)] $Ctx, [Parameter(Mandatory)] [string] $From, [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [ValidateSet('ok','no-op','stuck','skipped-locked','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status + [Parameter(Mandatory)] [ValidateSet('ok','no-op','dry-run','stuck','skipped-locked','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status ) . "$PSScriptRoot/Common.ps1" @@ -62,6 +62,21 @@ if ($Ctx.StuckSha) { } $lines.Add("") +if ($Status -eq 'dry-run' -and $Ctx.Pending.Count -gt 0) { + $lines.Add("## Pending commits (oldest → newest)") + $lines.Add("") + $lines.Add("| # | SHA | Subject | Author |") + $lines.Add("|---|---|---|---|") + $i = 0 + foreach ($sha in $Ctx.Pending) { + $i++ + $s = (git log -1 --format='%s' $sha) -replace '\|','\|' + $a = git log -1 --format='%an' $sha + $lines.Add("| $i | ``$($sha.Substring(0,9))`` | $s | $a |") + } + $lines.Add("") +} + if ($Ctx.Picked.Count -gt 0) { $lines.Add("## Picked commits (oldest → newest)") $lines.Add("") @@ -218,6 +233,7 @@ $lines.Add("") $lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``._") $suffix = if ($Status -eq 'skipped-locked') { 'skipped' } + elseif ($Status -eq 'dry-run') { 'dry-run' } elseif ($Status -like 'stuck-*') { $Status } elseif ($Status -eq 'stuck') { 'stuck' } elseif ($Status -eq 'no-op') { 'noop' } diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index 520203d99..1fc69767d 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -27,7 +27,7 @@ git push -u origin $Ctx.Branch 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } $shortSha = $Ctx.StuckSha.Substring(0,9) -$subj = git log -1 --format='%s' $Ctx.StuckSha +$subj = (git log -1 --format='%s' $Ctx.StuckSha).Trim() $title = "Upstream sync stuck at ${shortSha}: $subj" # Header prepended to report content. diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 7bd779cc8..645dbd88a 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -46,8 +46,10 @@ $findingsForHash = switch ($Kind) { $findingsHash = Get-FindingsHash $findingsForHash # Establish base/head for this batch (best-effort; tolerate detached states). -$base = git rev-parse origin/main 2>$null -$head = git rev-parse HEAD 2>$null +$baseRaw = git rev-parse origin/main 2>$null +$base = if ($LASTEXITCODE -eq 0 -and $baseRaw) { $baseRaw.Trim() } else { $null } +$headRaw = git rev-parse HEAD 2>$null +$head = if ($LASTEXITCODE -eq 0 -and $headRaw) { $headRaw.Trim() } else { $null } # Push the sync branch so the human can resume on it (even toolchain-missing — # the picks are still useful artifacts for whoever owns the host). diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 111ff05d5..bb9638b6a 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -50,7 +50,7 @@ function Ensure-UpstreamRemote { git remote add $Name $Url | Out-Null if ($LASTEXITCODE -ne 0) { throw "Failed to add remote $Name." } } elseif ($existing.Trim() -ne $Url) { - Write-Warning "Remote '$Name' exists but points at '$existing' (expected '$Url'). Leaving as-is." + throw "Remote '$Name' points at '$($existing.Trim())' (expected '$Url'). Fix the remote before running upstream-sync." } } @@ -86,6 +86,7 @@ function New-RunContext { Host = $env:COMPUTERNAME Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))" Picked = @() + Pending = @() DroppedPairs = @() SkippedEmpty = @() Tier0 = @() From 9bda34435b8114437bf7939e7cb8c1edd55e0958 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:19:11 +0800 Subject: [PATCH 11/82] fix(upstream-sync): validate cherry-pick cleanup Check native cherry-pick abort/skip cleanup exit codes and document the Windows Visual Studio build host prerequisite for default validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 1 + .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 | 3 +++ 2 files changed, 4 insertions(+) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 199150f9f..6590ad6eb 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -29,6 +29,7 @@ conflict appears. - `git` 2.30+ and `gh` CLI authenticated against `microsoft/intelligent-terminal`. - PowerShell 7+ (`pwsh`) on PATH. +- Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). - Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` (the scripts create it if missing). - `state.json` initialized once (see [references/bootstrap.md](./references/bootstrap.md)). diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 8614eeb6f..ec74e7b6d 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -124,6 +124,7 @@ foreach ($p in $conflicts) { if ($unhandled.Count -gt 0) { git cherry-pick --abort | Out-Null + if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after unhandled conflicts; repository may still be mid-cherry-pick." } $result.status = 'stuck' $result.conflict_paths = $unhandled $result | ConvertTo-Json -Compress @@ -137,11 +138,13 @@ if ($LASTEXITCODE -ne 0) { $staged = git diff --cached --name-only if (-not $staged) { git cherry-pick --skip | Out-Null + if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --skip failed after an empty Tier-0 continuation." } $result.status = 'skipped-empty' $result | ConvertTo-Json -Compress return } git cherry-pick --abort | Out-Null + if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after Tier-0 continuation failed; repository may still be mid-cherry-pick." } $result.status = 'stuck' $result | ConvertTo-Json -Compress return From b12cf07c63eba3c05af7579caa6bbb240b21c044 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:24:51 +0800 Subject: [PATCH 12/82] fix(upstream-sync): stabilize state serialization Preserve JSON property order when reading state and remove a stale finalize comment about the squash warning banner. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/06-finalize-pr.ps1 | 4 +--- .github/skills/upstream-sync/scripts/Common.ps1 | 2 +- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 50be9f7c0..10dea6840 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -26,9 +26,7 @@ param( . "$PSScriptRoot/Common.ps1" # Prepend the squash-warning banner to the report so it lands as the -# first thing reviewers see in the PR body. The same banner is also -# rendered into the report file in scripts/05-write-report.ps1 — keep -# both consistent. +# first thing reviewers see in the PR body. $banner = @" > ⚠️ **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked > upstream commit into one, destroying per-commit attribution, original diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index bb9638b6a..fee37405f 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -29,7 +29,7 @@ function Read-State { if (-not (Test-Path $p)) { throw "state.json not found at $p. Run scripts/00-bootstrap.ps1 first — see references/bootstrap.md." } - return Get-Content -Raw -LiteralPath $p | ConvertFrom-Json -AsHashtable + return Get-Content -Raw -LiteralPath $p | ConvertFrom-Json } function Write-State { From e14b928dbf9ff3fcfb13593206e9e604baefc08d Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:30:49 +0800 Subject: [PATCH 13/82] fix(upstream-sync): canonicalize operator SHAs Resolve bootstrap and clear-stuck SHA inputs to full commit IDs before validation, persistence, and short-label formatting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/00-bootstrap.ps1 | 3 ++- .../skills/upstream-sync/scripts/Common.ps1 | 7 +++++ .../upstream-sync/scripts/clear-stuck.ps1 | 26 ++++++++++--------- 3 files changed, 23 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 index 213003d5f..59d519b8f 100644 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -29,7 +29,8 @@ Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } -# Verify the SHA exists on upstream/main. +# Verify the SHA exists on upstream/main and persist the canonical 40-hex form. +$BaselineSha = Resolve-FullCommitSha $BaselineSha $null = git merge-base --is-ancestor $BaselineSha upstream/main if ($LASTEXITCODE -ne 0) { throw "Baseline SHA $BaselineSha is not an ancestor of upstream/main. Refusing to write state.json." diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index fee37405f..f2014c980 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -62,6 +62,13 @@ function Assert-CleanWorktree { } } +function Resolve-FullCommitSha { + param([Parameter(Mandatory)] [string] $Sha) + $full = (git rev-parse "$Sha^{commit}" 2>$null).Trim() + if ($LASTEXITCODE -ne 0 -or -not $full) { throw "Could not resolve commit SHA '$Sha'." } + return $full +} + function Get-GhUserLogin { $login = gh api user --jq '.login' 2>$null if ($LASTEXITCODE -ne 0 -or -not $login) { throw "gh CLI is not authenticated. Run 'gh auth login'." } diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index 6bf4bb9af..e8481b34d 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -51,19 +51,21 @@ if ($LASTEXITCODE -ne 0) { throw "git switch main failed; refusing to clear stuc git pull --ff-only origin main | Out-Null if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed; refusing to clear stuck-lock until main is current." } +$resolvedFullSha = if ($ResolvedThroughSha) { Resolve-FullCommitSha $ResolvedThroughSha } else { $null } + if ($tier3) { - if (-not $ResolvedThroughSha) { + if (-not $resolvedFullSha) { throw "Tier-3 stuck_on_sha is set ($($state.stuck_on_sha)) — -ResolvedThroughSha is required to clear it." } - $null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main + $null = git merge-base --is-ancestor $resolvedFullSha upstream/main if ($LASTEXITCODE -ne 0) { - throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." + throw "ResolvedThroughSha $resolvedFullSha is not on upstream/main. Refusing to clear lock." } - $null = git merge-base --is-ancestor $state.stuck_on_sha $ResolvedThroughSha + $null = git merge-base --is-ancestor $state.stuck_on_sha $resolvedFullSha if ($LASTEXITCODE -ne 0) { - throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $ResolvedThroughSha. Refusing — pass the same SHA or a later one." + throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $resolvedFullSha. Refusing — pass the same SHA or a later one." } - $state.last_synced_upstream_sha = $ResolvedThroughSha + $state.last_synced_upstream_sha = $resolvedFullSha $state.stuck_on_sha = $null $state.stuck_branch = $null $state.stuck_at = $null @@ -71,12 +73,12 @@ if ($tier3) { } if ($tier4) { - if ($ResolvedThroughSha) { - $null = git merge-base --is-ancestor $ResolvedThroughSha upstream/main + if ($resolvedFullSha) { + $null = git merge-base --is-ancestor $resolvedFullSha upstream/main if ($LASTEXITCODE -ne 0) { - throw "ResolvedThroughSha $ResolvedThroughSha is not on upstream/main. Refusing to clear lock." + throw "ResolvedThroughSha $resolvedFullSha is not on upstream/main. Refusing to clear lock." } - $state.last_synced_upstream_sha = $ResolvedThroughSha + $state.last_synced_upstream_sha = $resolvedFullSha } $state.stuck_validation = $null } @@ -88,14 +90,14 @@ $entry = [ordered] @{ status = if ($tier3 -and $tier4) { 'cleared-stuck (tier3+tier4)' } elseif ($tier3) { 'cleared-stuck (tier3)' } else { 'cleared-stuck (tier4)' } - advanced_to = $ResolvedThroughSha + advanced_to = $resolvedFullSha reason = $Reason } $state.history = @($entry) + @($state.history) | Select-Object -First 20 Write-State $state git add -- (Get-StatePath) | Out-Null -$shortLabel = if ($ResolvedThroughSha) { $ResolvedThroughSha.Substring(0,9) } else { 'no-advance' } +$shortLabel = if ($resolvedFullSha) { $resolvedFullSha.Substring(0,9) } else { 'no-advance' } git commit -m "chore(upstream-sync): clear stuck-lock ($shortLabel)" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } git push origin main | Out-Host From c9baf1458f1017f59314b405a2e464294e1d16e1 Mon Sep 17 00:00:00 2001 From: yeelam Date: Thu, 4 Jun 2026 18:43:55 +0800 Subject: [PATCH 14/82] fix(upstream-sync): align spelling patterns Cover singular hashtable and date-format tokens in spelling patterns, and use the repository spelling judgment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/patterns/patterns.txt | 4 ++-- .github/skills/upstream-sync/SKILL.md | 2 +- .github/skills/upstream-sync/references/known-conflicts.md | 2 +- .github/skills/upstream-sync/references/reporting.md | 2 +- .github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index 688ab76c0..cf8585548 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -323,5 +323,5 @@ env_remove\("[A-Z_]+"\) # upstream-sync skill: PowerShell variable names, git/config tokens, toolchain labels, and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables|(?:[a-z]+_)?[Tt]oolsets)\b -\b(?:DDTH(?:mmss)?|Hmmss|sszzz)\b +\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables?|(?:[a-z]+_)?[Tt]oolsets)\b +\b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 6590ad6eb..2865838ed 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -8,7 +8,7 @@ license: MIT Cherry-pick commit-by-commit from `https://github.com/microsoft/terminal` into this fork, preserving per-commit attribution, skipping commits that -cancel each other out, and stopping cleanly the moment a human-judgement +cancel each other out, and stopping cleanly the moment a human-judgment conflict appears. ## When to Use This Skill diff --git a/.github/skills/upstream-sync/references/known-conflicts.md b/.github/skills/upstream-sync/references/known-conflicts.md index f56d645e4..029048eff 100644 --- a/.github/skills/upstream-sync/references/known-conflicts.md +++ b/.github/skills/upstream-sync/references/known-conflicts.md @@ -46,7 +46,7 @@ auto-resolve under Tier 0. ## What does NOT belong here - Files that conflict but where the correct resolution **depends on the - commit** (those need Tier-2 or human judgement, not a fixed rule). + commit** (those need Tier-2 or human judgment, not a fixed rule). - Fork-only files (`tools/wta/**`, `src/cascadia/TerminalProtocol/**`) — these shouldn't conflict at all; if they do, the cause is structural and a Tier-0 entry would mask it. diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md index 4e7534e29..e4bba7710 100644 --- a/.github/skills/upstream-sync/references/reporting.md +++ b/.github/skills/upstream-sync/references/reporting.md @@ -101,7 +101,7 @@ The issue body is the report itself, plus a short header explaining urgency: ```markdown -🛑 **Upstream sync stopped at a conflict that needs human judgement.** +🛑 **Upstream sync stopped at a conflict that needs human judgment.** The scheduler will keep skipping its runs until this issue is resolved and the stuck-lock is cleared. No alarm — the lock is intentional. diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index 1fc69767d..cdc3cff4a 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -32,7 +32,7 @@ $title = "Upstream sync stuck at ${shortSha}: $subj" # Header prepended to report content. $header = @" -🛑 **Upstream sync stopped at a conflict that needs human judgement.** +🛑 **Upstream sync stopped at a conflict that needs human judgment.** The scheduler will keep skipping its runs until this issue is resolved and the stuck-lock is cleared. No alarm — the lock is intentional. From fd76e668a413d4707d3f5049bfd608a559630a11 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:17:24 +0800 Subject: [PATCH 15/82] docs(upstream-sync): add after-PR review handling step Codifies the fix-in-PR vs. follow-up-PR policy learned on PR #220 / #226: build-blocking failures get one extra commit on the sync branch, but all substantive review feedback (Copilot or human) goes into a separate follow-up PR based on the sync branch's HEAD. Preserves the cherry-pick PR's 'faithful to upstream' audit property. - SKILL.md: new 'After-PR review handling' subsection + Gotcha bullet. - references/follow-up-pr.md: new doc with full rubric, worktree mechanics, reply+resolve flow, and rebase-onto-main fallback. - references/workflow.md: new step 10 cross-linking the policy. - scripts/06-finalize-pr.ps1: PR banner now spells the policy out so the first reviewer doesn't push back on deferred fixes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 49 ++++++ .../upstream-sync/references/follow-up-pr.md | 151 ++++++++++++++++++ .../upstream-sync/references/workflow.md | 43 +++++ .../upstream-sync/scripts/06-finalize-pr.ps1 | 8 + 4 files changed, 251 insertions(+) create mode 100644 .github/skills/upstream-sync/references/follow-up-pr.md diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 2865838ed..64c7fe72d 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -113,6 +113,46 @@ Resumability is built into the state file — re-running after a successful run is a fast no-op (nothing pending), and re-running while the stuck-lock is set exits early without touching the branch. +### After-PR review handling — fix-in-PR vs. follow-up PR + +Once the sync PR is open, Copilot and human reviewers will leave +comments. The cherry-pick PR's commits must stay reviewable as +"per-commit, faithful to upstream + minimal must-merge delta" so the +reviewer can spot-check each upstream commit was applied correctly. +That constraint shapes the response policy: + +| Comment type | Where to fix | +|---|---| +| **Build-blocking** (compile error, dedup of resw/manifest collisions exposed only at build time, CI gate failure on the sync PR itself) | **One** focused extra commit on the sync branch. The cherry-pick PR is what's broken; the cherry-pick PR is what gets the fix. | +| **Everything else** — code-quality findings, logic-bug suggestions, translation corrections, spelling-allowlist migrations, typo fixes, doc nits, design pushback | **Follow-up PR** on top of the cherry-pick PR (see [references/follow-up-pr.md](./references/follow-up-pr.md)) | + +**Why split.** A reviewer scanning the cherry-pick PR is auditing +"did the sync engine faithfully apply the upstream batch?" Mixing +substantive review feedback into the same PR forces them to mentally +subtract those commits from every upstream-comparison check. Worse, +amending or squashing in review fixes destroys the per-commit +attribution the cherry-pick approach was chosen to preserve. + +**Follow-up PR shape** (template in +[references/follow-up-pr.md](./references/follow-up-pr.md)): + +- New worktree + branch `dev//sync--review-fixes` + off the sync PR's HEAD (see + [branch-worktree-workflow](./references/follow-up-pr.md#worktree-setup)). +- Base = the sync branch (e.g. `upstream-sync/2026-06-04`), **not** + `main`. The follow-up rides along with the sync PR. +- One focused commit per concern (code-bugs / translations / + spelling-cleanup / etc.) — same "audit trail per finding" rule as the + Copilot PR review loop skill. +- Reply + resolve every original thread on the sync PR pointing to the + follow-up PR number. +- If the sync PR merges first, rebase the follow-up onto `main` before + it merges. + +The orchestrator's PR banner ([scripts/06-finalize-pr.ps1](./scripts/06-finalize-pr.ps1)) +spells this policy out to the first reviewer so they don't push back on +deferred fixes. + ## Gotchas - **Never squash-merge the sync PR.** Squash collapses every cherry-picked @@ -122,6 +162,14 @@ is set exits early without touching the branch. banner reminding the reviewer; `-AutoMergeStrategy rebase` arms GitHub auto-merge with the right strategy so a tired reviewer can't get it wrong. +- **Don't amend review fixes into the sync PR.** Only build-blocking + fixes (compile errors, dedup of conflicts that surface at build time, + CI gate failures on the sync PR itself) get **one** extra commit on + the sync branch. Substantive Copilot/human review feedback — + code-quality, logic, translations, spelling-list migrations, doc nits + — goes into a separate follow-up PR off the sync branch's HEAD. See + [references/follow-up-pr.md](./references/follow-up-pr.md). Mixing + them poisons the "faithful to upstream" audit of the cherry-pick PR. - **Never rebase `upstream/main` onto this fork.** Old "Merge upstream" commits in the fork history replay and cascade conflicts. Use cherry-pick. Verified failure mode on the sister repo `agentic-terminal`. @@ -176,6 +224,7 @@ is set exits early without touching the branch. - [references/static-scan.md](./references/static-scan.md) — post-pick static breakage scan rules. - [references/fork-invariants.json](./references/fork-invariants.json) — fork-specific patterns that must survive any upstream pick. - [references/build-verification.md](./references/build-verification.md) — try-build pipeline + toolchain preflight policy. +- [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow for handling post-PR review. - [references/reporting.md](./references/reporting.md) — report template and stuck-issue template. - [scripts/04-run-batch.ps1](./scripts/04-run-batch.ps1) — the scheduler entrypoint. - [scripts/clear-stuck.ps1](./scripts/clear-stuck.ps1) — clear the stuck-lock after human resolution. diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md new file mode 100644 index 000000000..bd588a65a --- /dev/null +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -0,0 +1,151 @@ +# After-PR Review Handling: fix-in-PR vs. follow-up PR + +Once the sync PR is open, reviewers (GitHub Copilot bot and humans) will +leave comments. This doc is the decision rubric and the mechanics. + +## Why this exists + +The cherry-pick PR's value to the reviewer is "**N small commits, each +faithful to one upstream commit, plus the bare minimum to make CI +green**". Two failure modes break that: + +1. **Squashing fixes into the sync PR** (e.g. amending the cherry-pick + commits or adding many follow-up commits to the same branch) + destroys per-commit attribution, original author/committer dates, and + makes "which upstream commit broke X?" un-bisectable. +2. **Substantive feedback bundled into the sync PR** forces the + reviewer to mentally subtract those commits from every + upstream-comparison check ("is this difference here because the + cherry-pick was wrong, or because someone added a fix?"). + +So we keep the sync PR pristine and route everything else to a +follow-up PR. + +## Decision rubric + +| Comment / failure | Fix where | Why | +|---|---|---| +| Build error introduced by the cherry-pick batch | **Sync PR** — one focused commit | The sync PR is broken; the sync PR must be made buildable before merge. | +| Duplicate `.resw` keys / manifest collisions surfaced by `bz` but missed by the static-scan baseline | **Sync PR** — one focused commit | Same reason — without it the sync PR can't merge. | +| CI gate failure on the sync PR (check-spelling, lint, format) that is genuinely caused by the cherry-picked content | **Sync PR** — one focused commit | Same reason. | +| Copilot reviewer correctness finding (e.g. accumulator drift past clamp floor, missing cleanup on exception path) | **Follow-up PR** | Reviewer added value but the change is a *refactor* of upstream code, not a build fix. Cherry-pick PR stays faithful. | +| Translation error (`zh-TW` 工作模式 vs 工作階段) | **Follow-up PR** | The cherry-pick faithfully imported what upstream shipped. Correcting it is fork-local improvement. | +| Spelling-allowlist migration (move from `expect.txt` → `allow/*.txt`) | **Follow-up PR** | Repo-policy hygiene, not a build blocker. | +| Typo in a comment cherry-picked from upstream | **Follow-up PR** | Same. | +| Human reviewer asks for a design change / API tweak | **Follow-up PR** *or* push upstream | Fork-local refactor or upstream contribution, never bundled into the sync. | + +When in doubt, ask: *"If this commit didn't exist, would the sync PR +still merge cleanly with a green build?"* — if yes, it belongs in the +follow-up PR. + +## Mechanics + +### Worktree setup + +Stay on the [primary-worktree-clean-main +convention](./conflict-triage.md). Open a sibling worktree for the +follow-up so the main worktree stays clean: + +```pwsh +$syncPr = 220 +$syncBranch = 'upstream-sync/2026-06-04' # the sync PR's head +$fixBranch = "dev/$env:USERNAME/sync-$syncPr-review-fixes" +$fixWorktree = "..\it-$syncPr`fix" + +# Sync branch tip must be local for the worktree to land at the right base +git fetch origin $syncBranch +git worktree add $fixWorktree -b $fixBranch "origin/$syncBranch" + +Push-Location $fixWorktree +# … apply fixes, commit, push from here … +Pop-Location +``` + +### Commit shape + +One focused commit per concern, mirroring the +[`copilot-pr-review-loop`](../../copilot-pr-review-loop/SKILL.md) +"one commit per round" rule: + +``` +fix(terminal): # e.g. code-path corrections from Copilot +fix(loc): # e.g. translation corrections +build(spelling): # e.g. allow/expect migrations +``` + +Don't bundle them into a single mega-commit — `git blame` should still +point each line back to the specific finding that justified it. + +### PR base + +```pwsh +gh pr create ` + --repo microsoft/intelligent-terminal ` + --base $syncBranch ` # NOT main + --head $fixBranch ` + --title "fix(sync-): " ` + --body-file .pr-body.txt +``` + +The base is the **sync branch**, not `main`. That way the follow-up +rides along with the sync PR and only its fix commits show in the diff. +If the sync PR merges to `main` before the follow-up does, rebase the +follow-up onto `main` (the merge strategy will collapse it cleanly). + +### Reply + resolve on the sync PR + +After opening the follow-up, walk every review thread on the sync PR +that was deferred to the follow-up and reply + resolve: + +> Addressed in # (branch +> `dev//sync--review-fixes` based on this PR's head). +> Will be merged together with this PR (or rebased onto `main` if this +> lands first). + +Use [`copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1`](../../copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1) +to do this in a loop: + +```pwsh +$ids = @('PRRT_kw...','PRRT_kw...',...) +$script = '.github/skills/copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1' +$body = "Addressed in # ..." +foreach ($id in $ids) { pwsh -NoProfile -File $script -ThreadId $id -Body $body } +``` + +### When the sync PR merges first + +Rebase the follow-up onto `main`: + +```pwsh +cd $fixWorktree +git fetch origin main +git rebase origin/main +git push --force-with-lease +``` + +Then change the follow-up's base on GitHub: + +```pwsh +gh pr edit --repo microsoft/intelligent-terminal --base main +``` + +## Anti-patterns + +- ❌ Amending or rewriting the cherry-pick commits to "fix" a review + finding. Destroys the per-commit attribution that was the whole + reason for cherry-picking. +- ❌ Adding more than one extra commit to the sync PR. If a second + build-fix commit is needed, that's a smell — investigate whether the + first fix was incomplete or whether the second item really is a + follow-up. +- ❌ Opening the follow-up against `main` while the sync PR is still + unmerged. Diff fills up with the cherry-pick commits and the + reviewer can't see what changed. +- ❌ Resolving a sync-PR thread without leaving a pointer to the + follow-up PR. Future-you will have no record of where the fix went. + +## Cross-references + +- [SKILL.md § After-PR review handling](../SKILL.md#after-pr-review-handling--fix-in-pr-vs-follow-up-pr) — short version of this doc. +- [`copilot-pr-review-loop`](../../copilot-pr-review-loop/SKILL.md) — the skill that drives the follow-up PR through its own review rounds. +- [Branch + worktree convention](./conflict-triage.md) — keep `Q:\official\intelligent-terminal` on `main`; use sibling worktrees for PR work. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index 9b58d6358..e26bacb51 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -202,6 +202,49 @@ the lock is set — NO issue is opened. Script: [`07b-open-validation-stuck-issue.ps1`](../scripts/07b-open-validation-stuck-issue.ps1). +### 10. After-PR review handling (post-merge gate) + +Once the sync PR is open and reviewers (the GitHub Copilot bot, then +humans) start leaving comments, route the response by **comment kind**, +not by reviewer: + +| Comment kind | Where to fix | +|---|---| +| Build-blocking on the sync PR — compile errors, dedup of conflicts surfaced only at build time, sync-PR CI gate failures (check-spelling, lint, format) genuinely caused by the cherry-picked content | **Sync PR**, in **one** focused extra commit. Anything more than one extra commit is a smell. | +| Everything else — Copilot correctness findings, logic suggestions, translation corrections, spelling allow/expect migrations, doc/comment typos, design pushback | **Follow-up PR** based on the sync PR's HEAD. | + +The cherry-pick PR's value to a reviewer is "N small commits, each +faithful to one upstream commit, plus the bare minimum to make CI +green". Bundling substantive review fixes destroys that audit +property and forces the reviewer to mentally subtract those commits +from every upstream-comparison check. + +**Follow-up PR mechanics** (full procedure in +[follow-up-pr.md](./follow-up-pr.md)): + +1. Open a sibling worktree on a new branch off the sync PR's head: + ```pwsh + git fetch origin + git worktree add ..\it-fix -b dev//sync--review-fixes "origin/" + ``` +2. Apply fixes as **one focused commit per concern** (code-bugs / + translations / spelling-cleanup / etc.) — same "one commit per + round" rule as + [`copilot-pr-review-loop`](../../copilot-pr-review-loop/SKILL.md). +3. `gh pr create --base --head dev//sync--review-fixes` — + base is the **sync branch**, not `main`, so only the fix commits + show in the diff. +4. Walk every deferred review thread on the sync PR and reply + + resolve pointing to the follow-up PR number, via + [`copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1`](../../copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1). +5. If the sync PR merges first, rebase the follow-up onto `main` and + `gh pr edit --base main`. + +The PR banner emitted by +[`scripts/06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1) spells +this policy out so the first reviewer doesn't push back on deferred +fixes. + ## Stuck-Lock When **either** `state.stuck_on_sha` (Tier-3) **or** `state.stuck_validation` diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 10dea6840..63ebac21f 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -34,6 +34,14 @@ $banner = @" > merge"** (preferred — flat history, all $($Ctx.Picked.Count) commits land > individually) or **"Create a merge commit"** (also preserves per-commit > content). +> +> 📝 **Review-fix policy.** Only build-blocking fixes (compile errors, dedup +> of conflicts surfaced at build time, CI gate failures on this PR itself) +> belong here — as **one** focused extra commit on this branch. All other +> Copilot / human review feedback (code-quality, logic, translation, +> spelling-list migrations, doc nits) goes into a **follow-up PR** based on +> this PR's head, not amended into the cherry-pick commits. Rationale and +> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](../blob/$($Ctx.Branch)/.github/skills/upstream-sync/references/follow-up-pr.md). --- From 3bd31871facf4f8d034bd4801b63f21f37787ee2 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:17:24 +0800 Subject: [PATCH 16/82] build(spelling): allow CamelCase *Toolsets identifiers The existing pattern only covered '_toolsets' (snake_case); extend to '\w*[Tt]oolsets' so PowerShell function names like 'Get-RequiredToolsets' and MSBuild path components like 'PlatformToolsets' are recognized by check-spelling. Per repo policy (PR #45 user guidance) the right place for code identifiers is patterns.txt regex, not expect.txt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/patterns/patterns.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index cf8585548..ea516d852 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -323,5 +323,5 @@ env_remove\("[A-Z_]+"\) # upstream-sync skill: PowerShell variable names, git/config tokens, toolchain labels, and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables?|(?:[a-z]+_)?[Tt]oolsets)\b +\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables?|\w*[Tt]oolsets)\b \b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b From 59cd38f733d4a73d2e701918b135e89d73370623 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:24:37 +0800 Subject: [PATCH 17/82] fix(upstream-sync): reword to satisfy check-spelling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace 'pushback' -> 'feedback' (×2) and 'bisectable' -> 'impossible to git bisect' so the GHAS check-spelling bot has no new alerts. Per repo policy (PR #45 guidance), reword instead of adding plain English words to expect.txt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 2 +- .github/skills/upstream-sync/references/follow-up-pr.md | 2 +- .github/skills/upstream-sync/references/workflow.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 64c7fe72d..3cdd23708 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -124,7 +124,7 @@ That constraint shapes the response policy: | Comment type | Where to fix | |---|---| | **Build-blocking** (compile error, dedup of resw/manifest collisions exposed only at build time, CI gate failure on the sync PR itself) | **One** focused extra commit on the sync branch. The cherry-pick PR is what's broken; the cherry-pick PR is what gets the fix. | -| **Everything else** — code-quality findings, logic-bug suggestions, translation corrections, spelling-allowlist migrations, typo fixes, doc nits, design pushback | **Follow-up PR** on top of the cherry-pick PR (see [references/follow-up-pr.md](./references/follow-up-pr.md)) | +| **Everything else** — code-quality findings, logic-bug suggestions, translation corrections, spelling-allowlist migrations, typo fixes, doc nits, design feedback | **Follow-up PR** on top of the cherry-pick PR (see [references/follow-up-pr.md](./references/follow-up-pr.md)) | **Why split.** A reviewer scanning the cherry-pick PR is auditing "did the sync engine faithfully apply the upstream batch?" Mixing diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index bd588a65a..06c693e42 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -12,7 +12,7 @@ green**". Two failure modes break that: 1. **Squashing fixes into the sync PR** (e.g. amending the cherry-pick commits or adding many follow-up commits to the same branch) destroys per-commit attribution, original author/committer dates, and - makes "which upstream commit broke X?" un-bisectable. + makes "which upstream commit broke X?" impossible to `git bisect`. 2. **Substantive feedback bundled into the sync PR** forces the reviewer to mentally subtract those commits from every upstream-comparison check ("is this difference here because the diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index e26bacb51..f9d476746 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -211,7 +211,7 @@ not by reviewer: | Comment kind | Where to fix | |---|---| | Build-blocking on the sync PR — compile errors, dedup of conflicts surfaced only at build time, sync-PR CI gate failures (check-spelling, lint, format) genuinely caused by the cherry-picked content | **Sync PR**, in **one** focused extra commit. Anything more than one extra commit is a smell. | -| Everything else — Copilot correctness findings, logic suggestions, translation corrections, spelling allow/expect migrations, doc/comment typos, design pushback | **Follow-up PR** based on the sync PR's HEAD. | +| Everything else — Copilot correctness findings, logic suggestions, translation corrections, spelling allow/expect migrations, doc/comment typos, design feedback | **Follow-up PR** based on the sync PR's HEAD. | The cherry-pick PR's value to a reviewer is "N small commits, each faithful to one upstream commit, plus the bare minimum to make CI From 2c821c512dbda6e46670f829a19c0f7f3bf46bd4 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:33:29 +0800 Subject: [PATCH 18/82] fix(upstream-sync): replace 'exit' with 'throw' in helper scripts Helper scripts 08-static-scan, 09-toolchain-preflight, 10-try-build are invoked from 04-run-batch via the call operator (& ./script.ps1). In PowerShell, 'exit N' from a script invoked this way terminates the calling script (the orchestrator), so an 'exit 20' failure path in any helper would bypass the orchestrator's try/catch, the Tier-4 stuck handling, the report write, and the proper exit-code mapping (10 vs 20). Replace 'exit 20' with 'throw' so failures propagate as exceptions the orchestrator already catches; drop the redundant 'exit 0' at the success path so the captured JSON is the only stdout. Also backfill state.history[0].pr_url alongside state.last_run.pr_url in 06-finalize-pr, since both views share the same run-summary object and 'sessions' reports / bug-reports should find the PR link from either field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/06-finalize-pr.ps1 | 14 ++++++++++---- .../upstream-sync/scripts/08-static-scan.ps1 | 3 +-- .../scripts/09-toolchain-preflight.ps1 | 3 +-- .../skills/upstream-sync/scripts/10-try-build.ps1 | 3 +-- 4 files changed, 13 insertions(+), 10 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 63ebac21f..6c9a63086 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -115,11 +115,17 @@ if ($AutoMergeStrategy -ne 'none') { } } -# Backfill PR URL into state.last_run on the branch (best-effort follow-up -# commit). If this push fails the PR is still open and the baseline is still -# advanced on the branch — the only loss is the pr_url field in state.history, -# which is recoverable from the PR itself. +# Backfill PR URL into state.last_run AND state.history[0] (best-effort +# follow-up commit). The same run summary object was prepended to history +# earlier in this script — keep both views in sync so 'sessions' reports +# and bug-reports can find the PR link from either field. If this push +# fails the PR is still open and the baseline is still advanced on the +# branch — the only loss is the pr_url field in state, which is +# recoverable from the PR itself. $state.last_run.pr_url = $Ctx.PrUrl +if ($state.history -and $state.history.Count -gt 0) { + $state.history[0].pr_url = $Ctx.PrUrl +} Write-State $state git add -- (Get-StatePath) | Out-Null git commit -m "chore(upstream-sync): record PR url" | Out-Host diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 index 3e922a28f..b6dc65c77 100644 --- a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 +++ b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 @@ -175,10 +175,9 @@ try { blocking = $blocking } $doc | ConvertTo-Json -Depth 8 - exit 0 } catch { Write-Error $_.Exception.Message Write-Error $_.ScriptStackTrace - exit 20 + throw } diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index f838faaff..6f278bca8 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -97,10 +97,9 @@ try { ok = $ok } $doc | ConvertTo-Json -Depth 4 - exit 0 } catch { Write-Error $_.Exception.Message Write-Error $_.ScriptStackTrace - exit 20 + throw } diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index 7d7432528..530ce5e01 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -110,10 +110,9 @@ try { log_tail = $tailLines } $doc | ConvertTo-Json -Depth 4 - exit 0 } catch { Write-Error $_.Exception.Message Write-Error $_.ScriptStackTrace - exit 20 + throw } From d29b82269fce052428589f8dca7fbe880e58a40d Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:42:38 +0800 Subject: [PATCH 19/82] docs(upstream-sync): align help/retention text with throw-not-exit contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous round changed the helper scripts (08-static-scan, 09-toolchain-preflight, 10-try-build) from 'exit 20' to 'throw' on wrapper error, but the comment-based help still documented 'exit 20'. Update each script's .OUTPUTS block to describe the actual contract: throws on error so the orchestrator's outer try/catch routes it through the proper Tier-4 path; standalone callers see PowerShell's default exit (1) plus a stack trace. Spell out WHY 'exit 20' was intentionally removed so future maintainers don't 'fix' it back. Also clarify reporting.md retention statement — 'ok' and 'stuck' reports are committed in-repo, but 'no-op' and 'skipped-locked' reports are intentionally local-only to avoid churning main on every scheduler tick. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/upstream-sync/references/reporting.md | 10 +++++++++- .../skills/upstream-sync/scripts/08-static-scan.ps1 | 13 ++++++++++--- .../scripts/09-toolchain-preflight.ps1 | 11 ++++++++--- .../skills/upstream-sync/scripts/10-try-build.ps1 | 13 ++++++++++--- 4 files changed, 37 insertions(+), 10 deletions(-) diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md index e4bba7710..7c5a24573 100644 --- a/.github/skills/upstream-sync/references/reporting.md +++ b/.github/skills/upstream-sync/references/reporting.md @@ -126,4 +126,12 @@ Label: `upstream-sync-stuck` (apply via `gh issue create --label`). - The "skipped-locked" report shows the lock did its job (no duplicate destruction). -Retention: keep all reports indefinitely. They're small markdown. +Retention: the **"ok"** and **"stuck"** reports are committed in-repo +(by `06-finalize-pr.ps1` onto the sync branch, and by +`07-open-stuck-issue.ps1` / `07b-open-validation-stuck-issue.ps1` onto +`main` next to the lock). The **"no-op"** and **"skipped-locked"** +reports are written **locally only** and intentionally NOT committed — +they exist for "did the scheduler tick?" debugging on the host that ran +the script, and committing them would churn `main` on every cadence +even when nothing happened. They're small markdown either way; keep +committed reports indefinitely. diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 index b6dc65c77..94e34c021 100644 --- a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 +++ b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 @@ -21,9 +21,16 @@ Post-pick worktree ref (default: HEAD). .OUTPUTS - Emits a single JSON document on stdout. Exit code: - 0 = scan ran cleanly (findings may still be present — inspect JSON) - 20 = scan itself errored out (broken script, missing files, etc.) + Emits a single JSON document on stdout. + + Error model: + Throws on wrapper error (broken script, missing files, etc.). The + orchestrator (`04-run-batch.ps1`) catches and routes through its + own exit-code mapping (0 ok / 10 stuck / 20 error). Run standalone, + an uncaught throw exits with PowerShell's default code (1) plus a + stack trace. `exit 20` is intentionally NOT used here because this + script is invoked via `&` from the orchestrator — `exit` in that + context would terminate the orchestrator mid-pipeline. #> [CmdletBinding()] param( diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index 6f278bca8..a9d0151da 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -22,9 +22,14 @@ "ok": true } - Exit codes: - 0 = preflight ran (inspect JSON for ok=true/false) - 20 = preflight itself errored out + Error model: + Throws on wrapper error. The orchestrator (`04-run-batch.ps1`) + catches and routes through its own exit-code mapping (0 ok / 10 + stuck / 20 error). Run standalone, an uncaught throw exits with + PowerShell's default code (1) plus a stack trace. `exit 20` is + intentionally NOT used here because this script is invoked via + `&` from the orchestrator — `exit` in that context would terminate + the orchestrator mid-pipeline. #> [CmdletBinding()] param() diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index 530ce5e01..e8b3d4396 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -27,9 +27,16 @@ "log_tail": "" } - Exit codes: - 0 = wrapper ran (inspect JSON for build status) - 20 = wrapper itself errored out (couldn't start the build at all) + Exit / error model: + Stdout JSON on success (orchestrator path). + Throws on wrapper error (couldn't start the build at all). The + orchestrator (`04-run-batch.ps1`) catches and routes through its + own exit-code mapping (0 ok / 10 stuck / 20 error). When the script + is run standalone for debugging, an uncaught throw exits with + PowerShell's default code (1) and prints the stack trace. + `exit 20` is intentionally NOT used here: the script is invoked via + `& "$PSScriptRoot/10-try-build.ps1"`, and `exit` in that context + would terminate the orchestrator mid-pipeline. #> [CmdletBinding()] param( From 98c4bd2dcb2c80563147089ca7181a8214bfe717 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 07:50:32 +0800 Subject: [PATCH 20/82] fix(upstream-sync): tighten revert-pair fallback + broaden toolset probe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two correctness fixes from Copilot review: 1. 02-compute-pending.ps1 revert-pair fallback — when matching by quoted subject (not 'This reverts commit '), the old code searched the whole pending range and took the first match. If subjects repeated, it would pair the revert with a *later* unrelated commit and silently drop legitimate work. Now searches only the prefix of \ up to the current revert (the original must precede its revert in oldest-first order) and requires exactly one match — repeated subjects fall through and the revert lands as a normal pick. 2. 09-toolchain-preflight.ps1 PlatformToolset probe — was looking for src/common.openconsole.props which doesn't exist in this fork (the file lives at the repo root). Added root-level paths and the wap-common.build.* / common.build.post variants so a future relocation doesn't silently leave the preflight reading a stale file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/02-compute-pending.ps1 | 20 +++++++++++++++---- .../scripts/09-toolchain-preflight.ps1 | 13 ++++++++++-- 2 files changed, 27 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 2d853f25c..83de0afb5 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -64,12 +64,24 @@ foreach ($sha in $all) { if ($body -match 'This reverts commit ([0-9a-f]{40})\b') { $targetSha = $Matches[1] } elseif ($subj -match '^Revert "') { - # Best-effort: try to find the original by matching the quoted subject. + # Best-effort fallback: match the quoted original subject. To + # avoid pairing the revert with a *later* unrelated commit + # that happens to share the subject, search only the prefix of + # $all up to (but not including) the current revert — the + # original must precede its revert in oldest-first order. + # Also require exactly one match; if subjects repeat, fall + # through and let the revert land as a normal pick (safer + # than dropping the wrong commit). $origSubject = $subj -replace '^Revert "', '' -replace '"\s*$', '' -replace '"\.?\s*$','' - $candidate = $all | Where-Object { + $prefix = @() + foreach ($candidateSha in $all) { + if ($candidateSha -eq $sha) { break } + $prefix += $candidateSha + } + $candidates = @($prefix | Where-Object { $info[$_].subject -eq $origSubject -and -not $dropped.Contains($_) - } | Select-Object -First 1 - if ($candidate) { $targetSha = $candidate } + }) + if ($candidates.Count -eq 1) { $targetSha = $candidates[0] } } if ($targetSha -and $info.ContainsKey($targetSha) -and -not $dropped.Contains($targetSha)) { diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index a9d0151da..8d32ba4e9 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -38,10 +38,19 @@ param() function Get-RequiredToolsets { $root = Get-RepoRoot + # PlatformToolset can live at either the repo root (e.g. fork-local + # common.openconsole.props) or under src/ (the upstream cascadia + # layout). Probe both so a future relocation doesn't silently make + # the preflight read a stale file. Tests/older layouts may differ — + # missing files are skipped. $candidates = @( - 'src/common.build.pre.props', + 'common.openconsole.props', 'src/common.openconsole.props', - 'src/common.build.tests.props' + 'src/common.build.pre.props', + 'src/common.build.post.props', + 'src/common.build.tests.props', + 'src/wap-common.build.pre.props', + 'src/wap-common.build.post.props' ) $found = [System.Collections.Generic.HashSet[string]]::new() foreach ($rel in $candidates) { From 3531230ba753bc56d1beadadb448aea405203d32 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 08:03:04 +0800 Subject: [PATCH 21/82] fix(upstream-sync): existing-PR guard + case-sensitive resw dedup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two correctness fixes from Copilot review on PR #218: 1. 04-run-batch.ps1 — add early gate so an unattended scheduler that re-runs while a previous upstream-sync PR is still open does not collide on branch creation, double-open a PR, or fail in 06-finalize-pr.ps1. Skips with a 'skipped-pr-open' report and exit 0; -Force overrides. Wired the new status through 05-write-report.ps1 ValidateSet and suffix map. 2. 08-static-scan.ps1 — Group-Object now uses -CaseSensitive when detecting duplicate keys in .resw files. XAML resource lookups are case-sensitive, so 'Foo' and 'foo' are two different keys; without -CaseSensitive a fork that legitimately has both would be Tier-4 stuck on a false positive, and an actual case-only collision would be invisible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/04-run-batch.ps1 | 23 +++++++++++++++++++ .../upstream-sync/scripts/05-write-report.ps1 | 3 ++- .../upstream-sync/scripts/08-static-scan.ps1 | 7 +++++- 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index bb1069fbe..8ed6ed92e 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -125,6 +125,29 @@ try { exit 0 } + # --- Existing-PR gate --- + # Schedulers run unattended. If a prior run already opened an + # upstream-sync PR that hasn't merged yet, our baseline (origin/main) + # is unchanged, so we'd recompute the SAME pending range and then + # either (a) collide on branch creation or (b) 06-finalize-pr.ps1 + # would fail because the PR already exists for that branch. Worse, + # under a per-day branch-name scheme the second run would open a + # NEW PR with identical content. Bail early with a no-op report + # instead, unless -Force is given. + if (-not $Force) { + $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --search 'head:upstream-sync/' --json number,headRefName,url 2>$null + if ($LASTEXITCODE -eq 0 -and $existingJson) { + $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } + if ($existing.Count -gt 0) { + $first = $existing[0] + Write-Host "An upstream-sync PR is already open: #$($first.number) ($($first.headRefName)) -> $($first.url). Skipping until it merges or is closed (use -Force to override)." -ForegroundColor Yellow + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $state.last_synced_upstream_sha -To $state.last_synced_upstream_sha -Status 'skipped-pr-open' + Write-Host "Skip report: $reportPath" + exit 0 + } + } + } + Assert-CleanWorktree git switch main 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index eb45c1c3f..29a0d7e01 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -22,7 +22,7 @@ param( [Parameter(Mandatory)] $Ctx, [Parameter(Mandatory)] [string] $From, [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [ValidateSet('ok','no-op','dry-run','stuck','skipped-locked','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status + [Parameter(Mandatory)] [ValidateSet('ok','no-op','dry-run','stuck','skipped-locked','skipped-pr-open','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status ) . "$PSScriptRoot/Common.ps1" @@ -233,6 +233,7 @@ $lines.Add("") $lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``._") $suffix = if ($Status -eq 'skipped-locked') { 'skipped' } + elseif ($Status -eq 'skipped-pr-open') { 'skipped' } elseif ($Status -eq 'dry-run') { 'dry-run' } elseif ($Status -like 'stuck-*') { $Status } elseif ($Status -eq 'stuck') { 'stuck' } diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 index 94e34c021..8f3fce73f 100644 --- a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 +++ b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 @@ -46,7 +46,12 @@ function Get-ReswDuplicateNames { $names = [System.Collections.Generic.List[string]]::new() $re = [regex]' Date: Fri, 5 Jun 2026 08:11:03 +0800 Subject: [PATCH 22/82] fix(upstream-sync): gitignore local-only report suffixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Without this, the first no-op / dry-run / skipped-* run would leave an unstaged report file in .github/upstream-sync/reports/, and the next scheduled run would fail at Assert-CleanWorktree (which is what guarantees the cherry-pick base is the real origin/main, not a dirty one). Mirrors the retention model already documented in references/reporting.md: ok + stuck* reports are committed; no-op / dry-run / skipped-* are diagnostic-only and stay local. Verified via `git check-ignore` against the exact filenames Format-ReportFilename produces (Common.ps1:83-88) — only the local-only suffixes match, base + stuck still tracked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/upstream-sync/.gitignore | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/upstream-sync/.gitignore b/.github/upstream-sync/.gitignore index 2d32d9453..405032b17 100644 --- a/.github/upstream-sync/.gitignore +++ b/.github/upstream-sync/.gitignore @@ -1 +1,10 @@ build-logs/ + +# Local-only reports — see references/reporting.md for retention model. +# - 'ok' and 'stuck*' reports get committed (orchestrator stages them). +# - no-op / dry-run / skipped-* reports are diagnostic-only and stay +# local. Without ignoring them, the next scheduled run would fail at +# Assert-CleanWorktree because the worktree would be dirty. +reports/*-noop.md +reports/*-dry-run.md +reports/*-skipped.md From 2b14ccf65f0811c384596075c70174940858e1e5 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 08:25:13 +0800 Subject: [PATCH 23/82] fix(upstream-sync): repo-relative git add + bootstrap safety + pr-list limit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five correctness fixes from Copilot review on PR #218 (round 5): 1. Common.ps1 — new ConvertTo-RepoRelativePath helper. `git add` pathspec semantics with absolute paths are inconsistent across platforms / git versions and can silently no-op or refuse with "outside repository" even when the path IS inside the worktree (case-insensitive drive letters on Windows in particular). Normalize to forward-slash, repo-relative form before every `git add`. 2. Apply ConvertTo-RepoRelativePath to all 7 `git add` call sites that were passing absolute paths: - 00-bootstrap.ps1 - 06-finalize-pr.ps1 (initial state+report + backfill of state) - 06b-finalize-direct.ps1 (initial state+report + backfill) - 07-open-stuck-issue.ps1 (Tier-3 lock + report) - 07b-open-validation-stuck-issue.ps1 (Tier-4 lock + report) - clear-stuck.ps1 (post-unlock state write) Without this, a failed `git add` would silently leave state.json / the stuck-lock unwritten and the scheduler would loop re-running the same broken batch instead of honoring the lock. 3. 00-bootstrap.ps1 — assert on `main` + clean worktree + ff-only pull before creating the bootstrap branch. Previously, running bootstrap from a feature branch or a dirty worktree would have piggy-backed unrelated diffs onto the bootstrap commit/PR. 4. 04-run-batch.ps1 — existing-PR gate now passes `--limit 200` to `gh pr list`. The default is 30, so a repo with >30 open PRs could miss the upstream-sync PR and double-open. Declined: clear-stuck.ps1 watermark-advances-on-ancestry-only is intentional — the operator passing `-ResolvedThroughSha` IS the proof that the human picked the right SHA, and `-x` footer scanning would couple this recovery script to commit-message conventions that aren't guaranteed (cherry-pick `-x` is optional; squash-merge strips it). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/00-bootstrap.ps1 | 16 +++++++++++++++- .../upstream-sync/scripts/04-run-batch.ps1 | 5 ++++- .../upstream-sync/scripts/06-finalize-pr.ps1 | 4 ++-- .../scripts/06b-finalize-direct.ps1 | 4 ++-- .../scripts/07-open-stuck-issue.ps1 | 2 +- .../07b-open-validation-stuck-issue.ps1 | 2 +- .../skills/upstream-sync/scripts/Common.ps1 | 18 ++++++++++++++++++ .../upstream-sync/scripts/clear-stuck.ps1 | 2 +- 8 files changed, 44 insertions(+), 9 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 index 59d519b8f..b19cbbcf0 100644 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -25,6 +25,20 @@ param( . "$PSScriptRoot/Common.ps1" +# Safety: a bootstrap PR must contain *only* state.json. Refuse if the +# worktree is dirty or HEAD isn't main, otherwise unrelated diffs (or a +# feature branch's tip) could ride along on the bootstrap commit/PR. +$currentBranch = (git rev-parse --abbrev-ref HEAD).Trim() +if ($LASTEXITCODE -ne 0) { throw "git rev-parse failed (is this a git repo?)." } +if ($currentBranch -ne 'main') { + throw "Bootstrap must be run from 'main'. Currently on '$currentBranch'. git switch main first." +} +$dirty = git status --porcelain +if ($LASTEXITCODE -ne 0) { throw "git status failed." } +if ($dirty) { throw "Worktree is dirty. Bootstrap refuses to commit on a dirty tree:`n$dirty" } +git pull --ff-only origin main | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed. Resolve and retry." } + Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } @@ -64,7 +78,7 @@ if ($LASTEXITCODE -ne 0) { if ($LASTEXITCODE -ne 0) { throw "Could not create or switch to bootstrap branch '$branch'. Refusing to commit state.json on the current HEAD." } } -git add -- (Get-StatePath) +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed." } git commit -m "chore(upstream-sync): bootstrap baseline at $($BaselineSha.Substring(0,9))" | Out-Host diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index 8ed6ed92e..e01f4ab2d 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -135,7 +135,10 @@ try { # NEW PR with identical content. Bail early with a no-op report # instead, unless -Force is given. if (-not $Force) { - $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --search 'head:upstream-sync/' --json number,headRefName,url 2>$null + # --limit 200: `gh pr list` defaults to 30. If a repo somehow has + # 30+ open PRs and the upstream-sync one is older, the default + # would miss it and we'd duplicate the branch / PR. + $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --search 'head:upstream-sync/' --limit 200 --json number,headRefName,url 2>$null if ($LASTEXITCODE -eq 0 -and $existingJson) { $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } if ($existing.Count -gt 0) { diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 6c9a63086..765e22b8e 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -71,7 +71,7 @@ $state.last_run = $runSummary $state.history = @($runSummary) + @($state.history) | Select-Object -First 20 Write-State $state -git add -- (Get-StatePath) $ReportPath +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $ReportPath) if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting without push so baseline is not lost." } @@ -127,7 +127,7 @@ if ($state.history -and $state.history.Count -gt 0) { $state.history[0].pr_url = $Ctx.PrUrl } Write-State $state -git add -- (Get-StatePath) | Out-Null +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null git commit -m "chore(upstream-sync): record PR url" | Out-Host if ($LASTEXITCODE -eq 0) { git push origin $branch | Out-Host diff --git a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 index 99ef9672e..70feccaaf 100644 --- a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 +++ b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 @@ -59,7 +59,7 @@ $state.last_run = $runSummary $state.history = @($runSummary) + @($state.history) | Select-Object -First 20 Write-State $state -git add -- (Get-StatePath) $ReportPath +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $ReportPath) if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting before touching main." } @@ -96,7 +96,7 @@ if ($state.last_run -and $state.history -and $state.history.Count -gt 0) { $state.last_run.main_head_sha = $mainHead $state.history[0].main_head_sha = $mainHead Write-State $state - git add -- (Get-StatePath) | Out-Null + git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null git commit -m "chore(upstream-sync): record main head $($mainHead.Substring(0,9))" | Out-Host if ($LASTEXITCODE -eq 0) { git push origin main | Out-Host diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index cdc3cff4a..710b1a774 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -89,7 +89,7 @@ if ($ReportPath -ne $reportOnMain) { Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force } -git add -- (Get-StatePath) $reportOnMain +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $reportOnMain) if ($LASTEXITCODE -ne 0) { throw "git add of stuck state failed." } git commit -m "chore(upstream-sync): stuck at $shortSha (#$($Ctx.IssueUrl -replace '.*/',''))" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit of stuck-lock failed — lock NOT set on origin/main. The next scheduled run will not see the lock; resolve manually." } diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 645dbd88a..688c93e4a 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -138,7 +138,7 @@ $reportName = Split-Path -Leaf $ReportPath $reportOnMain = Join-Path (Get-ReportsDir) $reportName if ($ReportPath -ne $reportOnMain) { Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force } -git add -- (Get-StatePath) $reportOnMain | Out-Null +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $reportOnMain) | Out-Null if ($LASTEXITCODE -ne 0) { throw "git add failed for Tier-4 stuck state." } $msgIssueRef = if ($validation.issue_url) { " (#$($validation.issue_url -replace '.*/',''))" } else { ' (no issue — infra)' } diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index f2014c980..360dab157 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -24,6 +24,24 @@ function Get-ReportsDir { return $d } +function ConvertTo-RepoRelativePath { + # `git add` pathspec semantics differ between absolute and relative paths + # depending on the worktree-root vs. cwd interaction. Normalize every + # path our scripts hand to `git add` to a forward-slash, repo-relative + # form so the call is portable and won't silently no-op (or worse, + # leak a path-outside-tree error) on platforms where git treats + # absolute paths strictly. + param([Parameter(Mandatory)] [string] $Path) + $root = (Get-RepoRoot) -replace '\\','/' + $abs = $Path -replace '\\','/' + if ($abs.StartsWith($root, [System.StringComparison]::OrdinalIgnoreCase)) { + $rel = $abs.Substring($root.Length).TrimStart('/') + if (-not $rel) { throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" } + return $rel + } + throw "ConvertTo-RepoRelativePath: '$Path' is not under repo root '$root'." +} + function Read-State { $p = Get-StatePath if (-not (Test-Path $p)) { diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index e8481b34d..7557d1cd8 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -96,7 +96,7 @@ $entry = [ordered] @{ $state.history = @($entry) + @($state.history) | Select-Object -First 20 Write-State $state -git add -- (Get-StatePath) | Out-Null +git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null $shortLabel = if ($resolvedFullSha) { $resolvedFullSha.Substring(0,9) } else { 'no-advance' } git commit -m "chore(upstream-sync): clear stuck-lock ($shortLabel)" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } From a9b4075bc58540459b20b3d39fab35ee4d633b5f Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 08:36:17 +0800 Subject: [PATCH 24/82] fix(upstream-sync): prefix-boundary check + ordered state + docs drift MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five fixes from Copilot review on PR #218 (round 6) plus 2 GHAS check-spelling resolutions: 1. Common.ps1 ConvertTo-RepoRelativePath — require a path-segment boundary after the repo-root prefix so 'C:/repo' does not match 'C:/repo-old/file.txt'. Previously the StartsWith+Substring pair would accept the sibling and silently produce a garbage relative path like '-old/file.txt' which would then misroute `git add`. Also handles the exact 'path == root' case as an explicit throw. Verified with 3 sibling-prefix tests + the equality case. 2. 00-bootstrap.ps1 — write state.json from `[ordered] @{}` so ConvertTo-Json emits properties in a deterministic order. Hashtable key order is implementation-defined and the bootstrap commit/PR would otherwise diff against itself on different hosts. 3. references/workflow.md — corrected the no-op exit path description: writes a local report and exits 0 without touching state.json (matches actual orchestrator behavior — state writes are gated to ok and stuck* runs). 4. references/state-schema.md — removed 'no-op' and 'skipped-locked' from state.last_run.status enum (those statuses don't write state). Also removed the no-op entry from the history example and added a comment spelling out that only ok/stuck* runs land in history. 5. check-spelling — rewrote two flagged comments without resorting to allowlist entries: replaced ", otherwise unrelated diffs" with "; otherwise unrelated diffs" (forbidden-pattern rule) and reworded the ConvertTo-RepoRelativePath header to avoid the unrecognized 'pathspec' git jargon. Per repo convention: rework prose over allowlist for plain-English fixes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/references/state-schema.md | 8 ++++-- .../upstream-sync/references/workflow.md | 4 +-- .../upstream-sync/scripts/00-bootstrap.ps1 | 8 +++--- .../skills/upstream-sync/scripts/Common.ps1 | 28 +++++++++++-------- 4 files changed, 28 insertions(+), 20 deletions(-) diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md index fab3d8c30..cbbea9338 100644 --- a/.github/skills/upstream-sync/references/state-schema.md +++ b/.github/skills/upstream-sync/references/state-schema.md @@ -44,7 +44,7 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). "last_run": { "at": "2026-06-04T13:41:45+08:00", "host": "SH-YEELAM-D11S", - "status": "ok", // "ok" | "no-op" | "stuck" | "stuck-static-scan" | "stuck-build-failed" | "stuck-build-inconclusive" | "stuck-toolchain-missing" | "skipped-locked" + "status": "ok", // "ok" | "stuck" | "stuck-static-scan" | "stuck-build-failed" | "stuck-build-inconclusive" | "stuck-toolchain-missing" "branch": "upstream-sync/2026-06-04", "pr_url": "https://github.com/microsoft/intelligent-terminal/pull/999", "picked_count": 7, @@ -53,10 +53,12 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). "tier0_resolutions": 1 }, - // Rolling history — keep last 20 runs. + // Rolling history — keep last 20 runs. Only `ok` and `stuck*` runs + // write state (and therefore appear here); `no-op`, `dry-run`, and + // `skipped-*` runs produce a local-only report and leave state.json + // unchanged. "history": [ { "at": "...", "status": "ok", "picked_count": 7, "pr_url": "..." }, - { "at": "...", "status": "no-op", "picked_count": 0 }, { "at": "...", "status": "stuck", "stuck_on_sha": "abc...", "issue_url": "..." } ] } diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index f9d476746..2f1cd7683 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -22,8 +22,8 @@ git fetch upstream main --no-tags Script: [`01-fetch-upstream.ps1`](../scripts/01-fetch-upstream.ps1). -Exits with `state.last_run.status = "no-op"` and writes a "nothing to do" -report if `git rev-parse upstream/main` equals `state.last_synced_upstream_sha`. +Writes a local "no-op" report and exits 0 (without updating `state.json`) if +`git rev-parse upstream/main` equals `state.last_synced_upstream_sha`. ### 2. Compute pending range diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 index b19cbbcf0..a43fcf2fb 100644 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -25,13 +25,13 @@ param( . "$PSScriptRoot/Common.ps1" -# Safety: a bootstrap PR must contain *only* state.json. Refuse if the -# worktree is dirty or HEAD isn't main, otherwise unrelated diffs (or a +# Safety: a bootstrap PR must contain only state.json. Refuse if the +# worktree is dirty or HEAD is not main; otherwise unrelated diffs (or a # feature branch's tip) could ride along on the bootstrap commit/PR. $currentBranch = (git rev-parse --abbrev-ref HEAD).Trim() if ($LASTEXITCODE -ne 0) { throw "git rev-parse failed (is this a git repo?)." } if ($currentBranch -ne 'main') { - throw "Bootstrap must be run from 'main'. Currently on '$currentBranch'. git switch main first." + throw "Bootstrap must be run from 'main'. Currently on '$currentBranch'; git switch main first." } $dirty = git status --porcelain if ($LASTEXITCODE -ne 0) { throw "git status failed." } @@ -55,7 +55,7 @@ if ((Test-Path $statePath) -and -not $Force) { throw "state.json already exists at $statePath. Pass -Force to overwrite (rewinding the baseline can cause re-picks)." } -$state = @{ +$state = [ordered] @{ version = 1 upstream_remote_url = 'https://github.com/microsoft/terminal.git' upstream_branch = 'main' diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 360dab157..8829eccc9 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -25,19 +25,25 @@ function Get-ReportsDir { } function ConvertTo-RepoRelativePath { - # `git add` pathspec semantics differ between absolute and relative paths - # depending on the worktree-root vs. cwd interaction. Normalize every - # path our scripts hand to `git add` to a forward-slash, repo-relative - # form so the call is portable and won't silently no-op (or worse, - # leak a path-outside-tree error) on platforms where git treats - # absolute paths strictly. + # `git add` path arguments behave differently when absolute vs. + # repo-relative, depending on the worktree-root vs. cwd interaction. + # Normalize every path our scripts hand to `git add` to a + # forward-slash, repo-relative form so the call is portable and + # will not silently no-op (or worse, leak a path-outside-tree + # error) on platforms where git treats absolute paths strictly. param([Parameter(Mandatory)] [string] $Path) - $root = (Get-RepoRoot) -replace '\\','/' + $root = ((Get-RepoRoot) -replace '\\','/').TrimEnd('/') $abs = $Path -replace '\\','/' - if ($abs.StartsWith($root, [System.StringComparison]::OrdinalIgnoreCase)) { - $rel = $abs.Substring($root.Length).TrimStart('/') - if (-not $rel) { throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" } - return $rel + if ($abs.Equals($root, [System.StringComparison]::OrdinalIgnoreCase)) { + throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" + } + # Require a path-segment boundary after the prefix so 'C:/repo' does not + # match 'C:/repo-old/file' (prior implementation took the substring after + # the root length and only trimmed leading '/', which silently mangled + # sibling paths into garbage relative ones). + $prefix = "$root/" + if ($abs.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { + return $abs.Substring($prefix.Length) } throw "ConvertTo-RepoRelativePath: '$Path' is not under repo root '$root'." } From af259813655cc2f79f6482959ed2a1e43a1a4976 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 08:46:10 +0800 Subject: [PATCH 25/82] fix(upstream-sync): stable doc link + bypass PR gate on direct-push MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 7): 1. 06-finalize-pr.ps1 — the review-fix policy note in the PR banner linked to the file via `../blob/$Branch/...`. Auto-merge with `--delete-branch` deletes that ref on merge, leaving the link in the merged PR body permanently broken. Point at `main` instead so the guidance stays accessible long-term. 2. 04-run-batch.ps1 — the existing-PR gate now also short-circuits when `-PushDirectToMain` is set. Direct-push never opens a PR, so calling `gh pr list` is gratuitous and would block runs on hosts without `gh` auth even though no PR is created. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/04-run-batch.ps1 | 6 ++++-- .github/skills/upstream-sync/scripts/06-finalize-pr.ps1 | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index e01f4ab2d..04c790aa6 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -133,8 +133,10 @@ try { # would fail because the PR already exists for that branch. Worse, # under a per-day branch-name scheme the second run would open a # NEW PR with identical content. Bail early with a no-op report - # instead, unless -Force is given. - if (-not $Force) { + # instead, unless -Force is given. Skipped entirely under + # -PushDirectToMain, which never opens a PR and shouldn't require + # `gh` auth on the host. + if (-not $Force -and -not $PushDirectToMain) { # --limit 200: `gh pr list` defaults to 30. If a repo somehow has # 30+ open PRs and the upstream-sync one is older, the default # would miss it and we'd duplicate the branch / PR. diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 765e22b8e..6f0573531 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -41,7 +41,7 @@ $banner = @" > Copilot / human review feedback (code-quality, logic, translation, > spelling-list migrations, doc nits) goes into a **follow-up PR** based on > this PR's head, not amended into the cherry-pick commits. Rationale and -> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](../blob/$($Ctx.Branch)/.github/skills/upstream-sync/references/follow-up-pr.md). +> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](../blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). --- From 21849be4093863d9f3bdd15f4eb5d58e2ee22c45 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 08:58:20 +0800 Subject: [PATCH 26/82] fix(upstream-sync): doc tweaks (retry count + gitignore link path) - 06-finalize-pr.ps1: comment said 'Retry once' but loop is 3 attempts. - .github/upstream-sync/.gitignore: corrected relative path to reporting.md (it lives under .github/skills/upstream-sync/references/, not next to the .gitignore). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/06-finalize-pr.ps1 | 2 +- .github/upstream-sync/.gitignore | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 6f0573531..43b406258 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -85,7 +85,7 @@ $title = "chore(upstream): sync microsoft/terminal up to $shortTo" # so --head takes the bare branch name. `--head OWNER:BRANCH` would tell gh to # look on a fork owned by OWNER, which is wrong for this scheduler. # -# Retry once after a short delay: `gh pr create` on Windows occasionally fails +# Retry up to 3 times with a short delay: `gh pr create` on Windows occasionally fails # with "Head sha can't be blank" right after a push (see SKILL.md gotcha). $prUrl = $null for ($attempt = 1; $attempt -le 3; $attempt++) { diff --git a/.github/upstream-sync/.gitignore b/.github/upstream-sync/.gitignore index 405032b17..727f42cca 100644 --- a/.github/upstream-sync/.gitignore +++ b/.github/upstream-sync/.gitignore @@ -1,6 +1,7 @@ build-logs/ -# Local-only reports — see references/reporting.md for retention model. +# Local-only reports — see .github/skills/upstream-sync/references/reporting.md +# for the retention model. # - 'ok' and 'stuck*' reports get committed (orchestrator stages them). # - no-op / dry-run / skipped-* reports are diagnostic-only and stay # local. Without ignoring them, the next scheduled run would fail at From c509cbb88130ac0bc1f8757ac3965c17939df362 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 09:08:06 +0800 Subject: [PATCH 27/82] fix(upstream-sync): absolute follow-up-pr link + try/finally around build process - 06-finalize-pr.ps1: relative '../blob/main/...' link in PR body resolves against /pull/ and 404s. Switched to absolute https://github.com/microsoft/intelligent-terminal/blob/main/... URL. - 10-try-build.ps1: wrap the writer/process lifecycle in try/finally so the log StreamWriter is always closed and the Process is always Disposed. Without this, an exception between Start() and the success-path Close() would leak a file handle and jam the next scheduler run's log write. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/06-finalize-pr.ps1 | 2 +- .../upstream-sync/scripts/10-try-build.ps1 | 65 +++++++++++-------- 2 files changed, 39 insertions(+), 28 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 43b406258..7ff146f07 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -41,7 +41,7 @@ $banner = @" > Copilot / human review feedback (code-quality, logic, translation, > spelling-list migrations, doc nits) goes into a **follow-up PR** based on > this PR's head, not amended into the cherry-pick commits. Rationale and -> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](../blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). +> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). --- diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index e8b3d4396..49815eb1e 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -71,35 +71,46 @@ try { $psi.CreateNoWindow = $true $proc = [System.Diagnostics.Process]::Start($psi) - - # Tee stdout/stderr into the log file as the build runs. The synchronized - # wrapper serializes concurrent stdout/stderr DataReceived callbacks. - $baseWriter = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) - $writer = [System.IO.TextWriter]::Synchronized($baseWriter) - $proc.add_OutputDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) - $proc.add_ErrorDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) - $proc.BeginOutputReadLine() - $proc.BeginErrorReadLine() - - $timeoutMs = $TimeoutMinutes * 60 * 1000 - $exited = $proc.WaitForExit($timeoutMs) - $kind = $null - $exitCode = $null - - if (-not $exited) { - try { $proc.Kill($true) } catch { } - $proc.WaitForExit() - $kind = 'build-inconclusive' - $exitCode = -1 - } else { - # WaitForExit(timeout) can return before async output callbacks drain. - # The parameterless wait completes only after redirected output events finish. - $proc.WaitForExit() - $exitCode = $proc.ExitCode - $kind = if ($exitCode -eq 0) { 'build-ok' } else { 'build-failed' } + $baseWriter = $null + $writer = $null + + try { + # Tee stdout/stderr into the log file as the build runs. The synchronized + # wrapper serializes concurrent stdout/stderr DataReceived callbacks. + $baseWriter = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) + $writer = [System.IO.TextWriter]::Synchronized($baseWriter) + $proc.add_OutputDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) + $proc.add_ErrorDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) + $proc.BeginOutputReadLine() + $proc.BeginErrorReadLine() + + $timeoutMs = $TimeoutMinutes * 60 * 1000 + $exited = $proc.WaitForExit($timeoutMs) + $kind = $null + $exitCode = $null + + if (-not $exited) { + try { $proc.Kill($true) } catch { } + $proc.WaitForExit() + $kind = 'build-inconclusive' + $exitCode = -1 + } else { + # WaitForExit(timeout) can return before async output callbacks drain. + # The parameterless wait completes only after redirected output events finish. + $proc.WaitForExit() + $exitCode = $proc.ExitCode + $kind = if ($exitCode -eq 0) { 'build-ok' } else { 'build-failed' } + } + } + finally { + # Always release the log file handle and process — scheduler runs are + # unattended and a leaked handle would jam the next run's log write. + if ($writer) { try { $writer.Flush() } catch {} + try { $writer.Close() } catch {} } + if ($baseWriter) { try { $baseWriter.Dispose() } catch {} } + if ($proc) { try { $proc.Dispose() } catch {} } } - $writer.Flush(); $writer.Close() $ended = Get-Date $durationMs = [int]($ended - $started).TotalMilliseconds From 6aae4044f4feafd299c81f26ab014f965f165e79 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 09:18:05 +0800 Subject: [PATCH 28/82] fix(upstream-sync): dispose SHA256 instance in Get-FindingsHash Wrap the hash computation in try/finally so the SHA256 instance is always Disposed. Schedulers re-invoke these scripts on a regular cadence and unmanaged-handle leaks accumulate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/Common.ps1 | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 8829eccc9..1ad2ec932 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -143,6 +143,11 @@ function Get-FindingsHash { # so repeat-runs of the same broken batch can be detected (and not re-issued). $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) $sha = [System.Security.Cryptography.SHA256]::Create() - $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) - return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) + try { + $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) + return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) + } + finally { + $sha.Dispose() + } } From 2654a147afbfc761084be2d319fc5990ebf848f5 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 09:30:42 +0800 Subject: [PATCH 29/82] fix(upstream-sync): drop broken head: search qualifier + move real words to allow.txt MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 9): 1. 04-run-batch.ps1 existing-PR gate — the github 'head:' search qualifier matches exact branch names, not prefixes, so 'head:upstream-sync/' would silently return nothing even when an upstream-sync/2026-06-04 PR is already open. Dropped the search filter and rely on the client-side 'headRefName -like upstream-sync/*' filter that was already in place. The --limit 200 cap still covers repos with more than the default 30 open PRs. 2. spelling allow/patterns split — moved 'forwardable', 'hashtable', and 'hashtables' from patterns.txt to allow.txt (alphabetically inserted), per repo convention: stable real English/tech words go in allow/, only identifier-like tokens (ACMR, MFC, PRRT_*, timestamp regex placeholders, *Toolsets) stay in patterns.txt where broad regexes can reduce signal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/allow/allow.txt | 3 +++ .github/actions/spelling/patterns/patterns.txt | 4 ++-- .github/skills/upstream-sync/scripts/04-run-batch.ps1 | 11 +++++++---- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/.github/actions/spelling/allow/allow.txt b/.github/actions/spelling/allow/allow.txt index 0faae3da1..2a4186ba9 100644 --- a/.github/actions/spelling/allow/allow.txt +++ b/.github/actions/spelling/allow/allow.txt @@ -69,6 +69,7 @@ foob fooba footgun formedness +forwardable FTCS gantt gfm @@ -84,6 +85,8 @@ greenfield greppable haikus handover +hashtable +hashtables historicals hstrings https diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index ea516d852..df21db803 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -321,7 +321,7 @@ env_remove\("[A-Z_]+"\) # GitHub GraphQL node ID prefix for PullRequestReviewThread \bPRRT_[A-Za-z0-9_-]+\b -# upstream-sync skill: PowerShell variable names, git/config tokens, toolchain labels, and timestamp placeholders +# upstream-sync skill: identifiers and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|forwardable|hashtables?|\w*[Tt]oolsets)\b +\b(?:u(?:sha|pid)|quotepath|\w*[Tt]oolsets)\b \b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index 04c790aa6..a44f5dd2c 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -137,10 +137,13 @@ try { # -PushDirectToMain, which never opens a PR and shouldn't require # `gh` auth on the host. if (-not $Force -and -not $PushDirectToMain) { - # --limit 200: `gh pr list` defaults to 30. If a repo somehow has - # 30+ open PRs and the upstream-sync one is older, the default - # would miss it and we'd duplicate the branch / PR. - $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --search 'head:upstream-sync/' --limit 200 --json number,headRefName,url 2>$null + # Drop the GitHub `head:` search qualifier — it matches exact + # branch names, not prefixes, so `head:upstream-sync/` would + # return nothing even when an `upstream-sync/2026-06-04` PR is + # open. List all open PRs (--limit 200 covers the corner case + # where a repo has more than the default 30 open) and filter + # client-side by headRefName. + $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>$null if ($LASTEXITCODE -eq 0 -and $existingJson) { $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } if ($existing.Count -gt 0) { From df7db3e2dd5c6a86e35122864068905181c4afd2 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 09:42:18 +0800 Subject: [PATCH 30/82] fix(upstream-sync): preserve caller env + stale-probe guard + missing ProgramFiles(x86) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four fixes from Copilot review on PR #218 (round 10): 1. 03-cherry-pick-one.ps1 — capture caller's GIT_AUTHOR_* / GIT_COMMITTER_* env vars before overwriting them, and restore (not delete) in the finally block. Higher-level automation may have intentionally set these and our pick should not silently wipe them. Verified with a unit script that sets GIT_AUTHOR_NAME externally and confirms it survives the capture/overwrite/restore cycle. 2. 09-toolchain-preflight.ps1 Get-RequiredToolsets — return both the matched toolset set AND the list of candidate files that actually existed, so the caller can detect a stale probe list (none of the candidate files matched). 3. 09-toolchain-preflight.ps1 ok rollup — fail closed when the probe list is stale (probed_files count == 0). Previously the gate waved through whenever VS was installed and no required toolsets were found, masking a relocated build-props file. Added probed_files and probe_stale to the output JSON for diagnostics. 4. 09-toolchain-preflight.ps1 Get-VsInstalls — guard ${env:ProgramFiles(x86)} being unset (nonstandard hosts, 32-bit containers). Return empty rather than throwing on Join-Path of a null prefix; orchestrator treats that as Tier-4 infra stuck. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/03-cherry-pick-one.ps1 | 29 +++++++++++---- .../scripts/09-toolchain-preflight.ps1 | 36 +++++++++++++++---- 2 files changed, 52 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index ec74e7b6d..aa7f6bc36 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -73,6 +73,19 @@ $prePickHead = (git rev-parse HEAD).Trim() if ($LASTEXITCODE -ne 0) { throw "Could not record pre-pick HEAD before cherry-picking $Sha." } $info = (git log -1 --format='%an%x09%ae%x09%aI%x09%cn%x09%ce%x09%cI' $fullSha) -split "`t" + +# Capture caller's existing GIT_AUTHOR_* / GIT_COMMITTER_* env so the finally +# block can restore them. Higher-level automation may have set these +# intentionally — we don't want to silently wipe them out after our pick. +$prevEnv = @{ + GIT_AUTHOR_NAME = $env:GIT_AUTHOR_NAME + GIT_AUTHOR_EMAIL = $env:GIT_AUTHOR_EMAIL + GIT_AUTHOR_DATE = $env:GIT_AUTHOR_DATE + GIT_COMMITTER_NAME = $env:GIT_COMMITTER_NAME + GIT_COMMITTER_EMAIL = $env:GIT_COMMITTER_EMAIL + GIT_COMMITTER_DATE = $env:GIT_COMMITTER_DATE +} + $env:GIT_AUTHOR_NAME = $info[0] $env:GIT_AUTHOR_EMAIL = $info[1] $env:GIT_AUTHOR_DATE = $info[2] @@ -154,10 +167,14 @@ $result.status = 'picked' $result | ConvertTo-Json -Compress } finally { - Remove-Item Env:GIT_AUTHOR_NAME -ErrorAction SilentlyContinue - Remove-Item Env:GIT_AUTHOR_EMAIL -ErrorAction SilentlyContinue - Remove-Item Env:GIT_AUTHOR_DATE -ErrorAction SilentlyContinue - Remove-Item Env:GIT_COMMITTER_NAME -ErrorAction SilentlyContinue - Remove-Item Env:GIT_COMMITTER_EMAIL -ErrorAction SilentlyContinue - Remove-Item Env:GIT_COMMITTER_DATE -ErrorAction SilentlyContinue + # Restore — not delete — so callers that intentionally set these + # env vars don't see them wiped after our pick. + foreach ($k in $prevEnv.Keys) { + $prev = $prevEnv[$k] + if ($null -eq $prev) { + Remove-Item "Env:$k" -ErrorAction SilentlyContinue + } else { + Set-Item "Env:$k" -Value $prev + } + } } diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index 8d32ba4e9..94491f473 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -52,10 +52,12 @@ function Get-RequiredToolsets { 'src/wap-common.build.pre.props', 'src/wap-common.build.post.props' ) - $found = [System.Collections.Generic.HashSet[string]]::new() + $found = [System.Collections.Generic.HashSet[string]]::new() + $probed = [System.Collections.Generic.List[string]]::new() foreach ($rel in $candidates) { $p = Join-Path $root $rel if (-not (Test-Path -LiteralPath $p)) { continue } + $probed.Add($rel) | Out-Null $text = [System.IO.File]::ReadAllText($p) foreach ($m in ([regex]']*>([^<]+)').Matches($text)) { $val = $m.Groups[1].Value.Trim() @@ -63,11 +65,20 @@ function Get-RequiredToolsets { if ($val -and $val -notmatch '^\$\(') { [void]$found.Add($val) } } } - return ,@($found) + return [pscustomobject] @{ + Toolsets = ,@($found) + ProbedFiles = ,@($probed) + } } function Get-VsInstalls { - $vswhere = Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio\Installer\vswhere.exe' + # ${env:ProgramFiles(x86)} can be unset on nonstandard hosts (e.g. a + # 32-bit-only container, or a misconfigured runner). Treat the + # absence as "no VS detected" rather than throwing — the orchestrator + # treats VS-missing as a Tier-4 infra stuck the same way. + $pf86 = ${env:ProgramFiles(x86)} + if (-not $pf86) { return ,@() } + $vswhere = Join-Path $pf86 'Microsoft Visual Studio\Installer\vswhere.exe' if (-not (Test-Path -LiteralPath $vswhere)) { return ,@() } $out = & $vswhere -all -products * -property installationPath 2>$null if ($LASTEXITCODE -ne 0) { return ,@() } @@ -97,17 +108,28 @@ function Get-AvailableToolsets { } try { - $required = Get-RequiredToolsets + $req = Get-RequiredToolsets + $required = $req.Toolsets + $probed = $req.ProbedFiles $vsInstalls = Get-VsInstalls - $available = Get-AvailableToolsets -VsInstalls $vsInstalls - $missing = @($required | Where-Object { $available -notcontains $_ }) - $ok = ($missing.Count -eq 0) -and ($required.Count -gt 0 -or $vsInstalls.Count -gt 0) + $available = Get-AvailableToolsets -VsInstalls $vsInstalls + $missing = @($required | Where-Object { $available -notcontains $_ }) + + # Stale-probe guard: if none of the candidate build files even + # existed, the probe list itself is wrong (paths moved). That's a + # silent gap — fail closed so the orchestrator surfaces it as a + # Tier-4 stuck instead of waving the run through on an empty + # required-toolsets list. + $probeStale = ($probed.Count -eq 0) + $ok = (-not $probeStale) -and ($missing.Count -eq 0) -and ($vsInstalls.Count -gt 0) $doc = [ordered] @{ required_toolsets = @($required) available_toolsets = @($available) missing = @($missing) vs_installs = @($vsInstalls) + probed_files = @($probed) + probe_stale = $probeStale ok = $ok } $doc | ConvertTo-Json -Depth 4 From 19c2bd7a01bff318b054b9afadf80025aa847213 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 09:53:29 +0800 Subject: [PATCH 31/82] fix(upstream-sync): validate Tier-0 git ops + stable findings hash MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 14): 1. 03-cherry-pick-one.ps1 Tier-0 — check $LASTEXITCODE on each `git checkout --theirs/--ours` and the follow-up `git add`. If any step fails, push the path onto $unhandled and fall through to the existing stuck path (which aborts the cherry-pick cleanly) instead of marching into `cherry-pick --continue` with missing resolutions. 2. 07b-open-validation-stuck-issue.ps1 Get-FindingsHash inputs — switch the per-kind `@{}` hashtables to `[ordered] @{}` so JSON property order (and therefore the SHA) is stable across runs, and normalize the toolchain-missing `missing` array via Sort-Object so list order doesn't perturb the hash either. Verified two structurally identical inputs produce the same digest. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/03-cherry-pick-one.ps1 | 29 ++++++++++++++++--- .../07b-open-validation-stuck-issue.ps1 | 8 +++-- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index aa7f6bc36..532a1ea60 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -126,12 +126,33 @@ $unhandled = @() foreach ($p in $conflicts) { $e = $known | Where-Object { $_.Path -eq $p } | Select-Object -First 1 if (-not $e) { $unhandled += $p; continue } + $checkoutOk = $true switch ($e.Strategy) { - 'take-upstream' { git checkout --theirs -- $p | Out-Null } - 'take-ours' { git checkout --ours -- $p | Out-Null } - 'union' { Write-Warning "union strategy not implemented yet for $p"; $unhandled += $p; continue } + 'take-upstream' { + git checkout --theirs -- $p 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { $checkoutOk = $false } + } + 'take-ours' { + git checkout --ours -- $p 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { $checkoutOk = $false } + } + 'union' { + Write-Warning "union strategy not implemented yet for $p" + $unhandled += $p + continue + } + } + if (-not $checkoutOk) { + Write-Warning "git checkout for Tier-0 strategy '$($e.Strategy)' failed on $p; falling back to stuck." + $unhandled += $p + continue + } + git add -- $p 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Warning "git add failed for Tier-0 path $p; falling back to stuck." + $unhandled += $p + continue } - git add -- $p | Out-Null $result.tier0_paths += $p } diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 688c93e4a..52e5dc722 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -37,11 +37,13 @@ param( . "$PSScriptRoot/Common.ps1" # Compute a findings hash so re-runs of the same broken batch are detectable. +# Use [ordered] hashtables so JSON serialization (and therefore the hash) is +# stable across runs — plain @{} property order is not guaranteed. $findingsForHash = switch ($Kind) { 'static-scan' { $Ctx.Scan.findings } - 'build-failed' { @(@{ exit_code = $Ctx.Build.exit_code; tail_excerpt = ($Ctx.Build.log_tail -split "`n" | Select-Object -Last 20) -join "`n" }) } - 'build-inconclusive' { @(@{ kind = 'inconclusive'; duration_ms = $Ctx.Build.duration_ms }) } - 'toolchain-missing' { @(@{ missing = $Ctx.Preflight.missing }) } + 'build-failed' { @([ordered] @{ exit_code = $Ctx.Build.exit_code; tail_excerpt = ($Ctx.Build.log_tail -split "`n" | Select-Object -Last 20) -join "`n" }) } + 'build-inconclusive' { @([ordered] @{ kind = 'inconclusive'; duration_ms = $Ctx.Build.duration_ms }) } + 'toolchain-missing' { @([ordered] @{ missing = @($Ctx.Preflight.missing | Sort-Object) }) } } $findingsHash = Get-FindingsHash $findingsForHash From a257d13c1b45e0a361e9e38a683a99b08431eaf8 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:04:14 +0800 Subject: [PATCH 32/82] fix(upstream-sync): single active stuck lock at a time MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two paired fixes from Copilot review on PR #218 (round 15): 1. 07-open-stuck-issue.ps1 — when promoting a Tier-3 (cherry-pick conflict) stuck lock, also clear `state.stuck_validation` so the scheduler and humans see one current stuck reason instead of one stale Tier-4 + one live Tier-3. 2. 07b-open-validation-stuck-issue.ps1 — when setting a Tier-4 (validation) stuck lock, also clear the Tier-3 fields (`stuck_on_sha`/`stuck_branch`/`stuck_at`/`stuck_issue_url`) for the same reason. Both guard with `PSObject.Properties.Name -contains` so older state.json files without the sibling field don't break. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/upstream-sync/scripts/07-open-stuck-issue.ps1 | 5 +++++ .../scripts/07b-open-validation-stuck-issue.ps1 | 7 +++++++ 2 files changed, 12 insertions(+) diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index 710b1a774..34b8bcd81 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -69,6 +69,11 @@ $state.stuck_on_sha = $Ctx.StuckSha $state.stuck_branch = $Ctx.Branch $state.stuck_at = Format-Iso8601 $Ctx.StartedAt $state.stuck_issue_url = $Ctx.IssueUrl +# Single active lock: clear the Tier-4 fields when promoting a Tier-3 lock so +# the scheduler and humans see one stuck reason, not two. +if ($state.PSObject.Properties.Name -contains 'stuck_validation') { + $state.stuck_validation = $null +} $runSummary = [ordered] @{ at = Format-Iso8601 $Ctx.StartedAt host = $Ctx.Host diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 52e5dc722..15aba65ea 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -122,6 +122,13 @@ if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stu $state = Read-State $state.stuck_validation = $validation +# Single active lock: clear the Tier-3 fields when setting a Tier-4 lock so +# state.json reflects one stuck reason, not two stale ones. +foreach ($f in 'stuck_on_sha','stuck_branch','stuck_at','stuck_issue_url') { + if ($state.PSObject.Properties.Name -contains $f) { + $state.$f = $null + } +} $runSummary = [ordered] @{ at = Format-Iso8601 $Ctx.StartedAt host = $Ctx.Host From 5ba470a8f48a1ba5d38c5f4e3e46ad63086922d5 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:14:42 +0800 Subject: [PATCH 33/82] fix(upstream-sync): toolchain-missing resume guidance + correct stuck base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 16): 1. 05-write-report.ps1 — split the "How to resume" steps by Kind. For Tier-4 `toolchain-missing` (infra-only, no code change needed on the sync branch), tell the reader to provision the host (or re-run from a provisioned one) and clear the lock; don't push them down a "manual validation fix PR" path that doesn't apply. 2. 07b-open-validation-stuck-issue.ps1 — capture `stuck_validation.base` as `git merge-base HEAD origin/main`, not the current `origin/main` tip. Otherwise an `origin/main` that advanced mid-run records the wrong base. Falls back to the tip only if merge-base fails (e.g. no upstream). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/05-write-report.ps1 | 34 +++++++++++++------ .../07b-open-validation-stuck-issue.ps1 | 16 +++++++-- 2 files changed, 37 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 29a0d7e01..bdda1e7b9 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -215,17 +215,29 @@ if ($Status -like 'stuck-*') { } $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") $lines.Add("") - $lines.Add("**How to resume:**") - $lines.Add("") - $lines.Add("1. ``git switch $($Ctx.Branch)``") - $lines.Add("2. Fix the issue above (e.g. resw dedup, restored fork invariant, build fix, host provisioning).") - $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it.") - $lines.Add("4. Clear the lock:") - $lines.Add(" ``````") - $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") - $lines.Add(" ``````") - $lines.Add("5. The next scheduled sync runs the same range — validation must pass before any PR is opened.") - $lines.Add("") + if ($Kind -eq 'toolchain-missing') { + $lines.Add("**How to resume (infra-only — no PR needed):**") + $lines.Add("") + $lines.Add("1. Provision the host with the missing toolset(s) above, **or** rerun the sync from a correctly provisioned host.") + $lines.Add("2. Clear the lock:") + $lines.Add(" ``````") + $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") + $lines.Add(" ``````") + $lines.Add("3. The next scheduled sync re-runs the same range; nothing on ``$($Ctx.Branch)`` needs editing.") + $lines.Add("") + } else { + $lines.Add("**How to resume:**") + $lines.Add("") + $lines.Add("1. ``git switch $($Ctx.Branch)``") + $lines.Add("2. Fix the issue above (e.g. resw dedup, restored fork invariant, build fix).") + $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it.") + $lines.Add("4. Clear the lock:") + $lines.Add(" ``````") + $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") + $lines.Add(" ``````") + $lines.Add("5. The next scheduled sync runs the same range — validation must pass before any PR is opened.") + $lines.Add("") + } } $lines.Add("---") diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 15aba65ea..0d54d17a1 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -48,10 +48,22 @@ $findingsForHash = switch ($Kind) { $findingsHash = Get-FindingsHash $findingsForHash # Establish base/head for this batch (best-effort; tolerate detached states). -$baseRaw = git rev-parse origin/main 2>$null -$base = if ($LASTEXITCODE -eq 0 -and $baseRaw) { $baseRaw.Trim() } else { $null } +# `base` must be the *pre-pick* origin/main, not its current tip, because +# origin/main may have advanced while the run was in progress. The merge-base +# of the sync tip and origin/main recovers the branch's original starting +# point reliably. $headRaw = git rev-parse HEAD 2>$null $head = if ($LASTEXITCODE -eq 0 -and $headRaw) { $headRaw.Trim() } else { $null } +$base = $null +if ($head) { + $baseRaw = git merge-base HEAD origin/main 2>$null + if ($LASTEXITCODE -eq 0 -and $baseRaw) { $base = $baseRaw.Trim() } +} +if (-not $base) { + # Fallback: use origin/main tip if merge-base failed (e.g. no upstream ref). + $baseRaw = git rev-parse origin/main 2>$null + if ($LASTEXITCODE -eq 0 -and $baseRaw) { $base = $baseRaw.Trim() } +} # Push the sync branch so the human can resume on it (even toolchain-missing — # the picks are still useful artifacts for whoever owns the host). From 7d07f56d7b8712cd7920078b120aeffd1aeec727 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:25:49 +0800 Subject: [PATCH 34/82] fix(upstream-sync): consistent kind casing + backfill before auto-merge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 17): 1. 05-write-report.ps1 — normalize `$Kind` → `$kind` for consistency with the local var derived from `$Status` two lines above. (PowerShell variable names are case-insensitive so this never actually threw, but the casing drift was an unintentional artifact of round 17.) 2. 06-finalize-pr.ps1 — move the pr_url backfill commit + push BEFORE arming `gh pr merge --auto --delete-branch`. If auto-merge is immediately satisfied (all checks green, approvals in place) it can merge and delete the remote branch fast enough that the backfill push would otherwise recreate a deleted `upstream-sync/` branch as an orphan carrying just the pr_url commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/05-write-report.ps1 | 2 +- .../upstream-sync/scripts/06-finalize-pr.ps1 | 42 ++++++++++--------- 2 files changed, 24 insertions(+), 20 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index bdda1e7b9..285fb2e89 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -215,7 +215,7 @@ if ($Status -like 'stuck-*') { } $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") $lines.Add("") - if ($Kind -eq 'toolchain-missing') { + if ($kind -eq 'toolchain-missing') { $lines.Add("**How to resume (infra-only — no PR needed):**") $lines.Add("") $lines.Add("1. Provision the host with the missing toolset(s) above, **or** rerun the sync from a correctly provisioned host.") diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 7ff146f07..a24fee59e 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -102,26 +102,17 @@ $Ctx.PrUrl = $prUrl.Trim() Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue -# Optional: arm GitHub auto-merge with the strategy that preserves per-commit -# history. 'rebase' is the recommended default when auto-merge is enabled — -# it lands all N commits flatly on main once CI + approvals pass. -if ($AutoMergeStrategy -ne 'none') { - $strategyFlag = "--$AutoMergeStrategy" - gh pr merge -R microsoft/intelligent-terminal $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host - if ($LASTEXITCODE -ne 0) { - Write-Warning "gh pr merge --auto failed. PR is open at $($Ctx.PrUrl); merge manually with '$AutoMergeStrategy' strategy (NOT squash)." - } else { - Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green - } -} - # Backfill PR URL into state.last_run AND state.history[0] (best-effort -# follow-up commit). The same run summary object was prepended to history -# earlier in this script — keep both views in sync so 'sessions' reports -# and bug-reports can find the PR link from either field. If this push -# fails the PR is still open and the baseline is still advanced on the -# branch — the only loss is the pr_url field in state, which is -# recoverable from the PR itself. +# follow-up commit) BEFORE arming auto-merge. If auto-merge is already +# satisfied (all checks green, approvals in place) it can merge and delete +# the remote branch immediately, after which `git push origin $branch` +# would recreate a deleted upstream-sync/ branch as an orphan with +# the pr_url commit on top. Keeping the same run summary object in +# last_run and history[0] in sync so 'sessions' reports and bug-reports +# can find the PR link from either field. If this push fails the PR is +# still open and the baseline is still advanced on the branch — the only +# loss is the pr_url field in state, which is recoverable from the PR +# itself. $state.last_run.pr_url = $Ctx.PrUrl if ($state.history -and $state.history.Count -gt 0) { $state.history[0].pr_url = $Ctx.PrUrl @@ -134,4 +125,17 @@ if ($LASTEXITCODE -eq 0) { if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push pr_url backfill; PR is still open at $($Ctx.PrUrl)." } } +# Optional: arm GitHub auto-merge with the strategy that preserves per-commit +# history. 'rebase' is the recommended default when auto-merge is enabled — +# it lands all N commits flatly on main once CI + approvals pass. +if ($AutoMergeStrategy -ne 'none') { + $strategyFlag = "--$AutoMergeStrategy" + gh pr merge -R microsoft/intelligent-terminal $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host + if ($LASTEXITCODE -ne 0) { + Write-Warning "gh pr merge --auto failed. PR is open at $($Ctx.PrUrl); merge manually with '$AutoMergeStrategy' strategy (NOT squash)." + } else { + Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green + } +} + return $Ctx.PrUrl From 0b678ad6737c56a466bb11a6de19fe2b7a15f28c Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:37:05 +0800 Subject: [PATCH 35/82] fix(upstream-sync): handle non-conflict cherry-pick failures + fail-fast gh gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 18): 1. 03-cherry-pick-one.ps1 — when `git cherry-pick` exits non-zero, check whether Get-ConflictPaths actually returned any unmerged paths. An empty list means this was a non-conflict failure (cherry-picking a merge commit without `-m`, hook failure, etc.), not a Tier-0 case. Abort the cherry-pick and emit a controlled `stuck` result with the pick exit code in `error`, instead of falling through to `cherry-pick --continue` (which would explode). 2. 04-run-batch.ps1 existing-PR gate — - Capture `gh pr list` stderr and `Exit-Hard` on non-zero exit instead of silently continuing into expensive pick+scan+build. Tells the user how to bypass (-Force, -DryRun, -PushDirectToMain). - Skip the gate entirely under `-DryRun` (no PR is opened so no gh is needed), matching the existing carve-out for `-PushDirectToMain`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/03-cherry-pick-one.ps1 | 17 ++++++++++++++++- .../upstream-sync/scripts/04-run-batch.ps1 | 15 ++++++++++++--- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 532a1ea60..f180d9a8d 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -118,8 +118,23 @@ if ($pickCode -eq 0) { return } -# Conflict. Try Tier-0. +# Conflict — or some other non-zero exit (e.g. trying to cherry-pick a +# merge commit without -m, hook failures, etc.). For *real* conflicts +# Get-ConflictPaths returns the unmerged paths and we try Tier-0; for +# non-conflict failures the list is empty and we must NOT fall through +# to `cherry-pick --continue` (which would explode), nor leave the repo +# mid-cherry-pick. Abort and surface a controlled stuck result. $conflicts = Get-ConflictPaths +if (-not $conflicts -or $conflicts.Count -eq 0) { + Write-Warning "git cherry-pick exited $pickCode with no conflict paths — likely a non-conflict failure (e.g. merge commit without -m). Aborting and marking stuck." + git cherry-pick --abort 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after a non-conflict failure; repository may still be mid-cherry-pick." } + $result.status = 'stuck' + $result.conflict_paths = @() + $result.error = "git cherry-pick exited $pickCode with no conflict paths" + $result | ConvertTo-Json -Compress + return +} $result.conflict_paths = $conflicts $known = Get-KnownConflicts $unhandled = @() diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index a44f5dd2c..5ca769c73 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -136,15 +136,24 @@ try { # instead, unless -Force is given. Skipped entirely under # -PushDirectToMain, which never opens a PR and shouldn't require # `gh` auth on the host. - if (-not $Force -and -not $PushDirectToMain) { + # Skip the existing-PR gate when there's nothing to publish anyway: + # -PushDirectToMain never opens a PR, and -DryRun stops well before + # 06-finalize-pr.ps1 — neither should require `gh` auth on the host. + if (-not $Force -and -not $PushDirectToMain -and -not $DryRun) { # Drop the GitHub `head:` search qualifier — it matches exact # branch names, not prefixes, so `head:upstream-sync/` would # return nothing even when an `upstream-sync/2026-06-04` PR is # open. List all open PRs (--limit 200 covers the corner case # where a repo has more than the default 30 open) and filter # client-side by headRefName. - $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>$null - if ($LASTEXITCODE -eq 0 -and $existingJson) { + $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>&1 + if ($LASTEXITCODE -ne 0) { + # `gh` missing / not authenticated / network-blocked. Don't + # silently continue and waste a full pick + scan + build + # only to fail later in 06-finalize-pr.ps1 — fail fast. + Exit-Hard "gh pr list failed (exit $LASTEXITCODE): $existingJson. The existing-PR gate requires gh to be installed and authenticated. Re-run with -Force to bypass (at your own risk), or with -DryRun / -PushDirectToMain to skip the gate." + } + if ($existingJson) { $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } if ($existing.Count -gt 0) { $first = $existing[0] From 179892fb5d60a0eecb58906ab17a00945325babe Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:47:56 +0800 Subject: [PATCH 36/82] fix(upstream-sync): array-wrap git log output + preserve upstream identity in resume guidance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 19): 1. 03-cherry-pick-one.ps1 empty-commit footer check — wrap `git log -1 --format='%B' HEAD` in `@(...)` before `-join "`n"`. When the external command returns a single string PowerShell can treat the result as a char array and insert newlines between every character, false-negating the cherry-pick footer regex on single-line commit messages. Array-wrap forces a string array. 2. 05-write-report.ps1 Tier-3 manual-resume — the resume snippet previously ran `git cherry-pick -x ` without pinning identity, so a human-resolved commit would land with the resolver's name as committer (and possibly author), breaking the per-commit upstream attribution that 03-cherry-pick-one.ps1 carefully preserves for the automated picks in the rest of the batch. Resume snippet now copies the upstream commit's author+committer name/email/date into env vars before the pick, and clears them afterwards so they don't leak. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/03-cherry-pick-one.ps1 | 2 +- .../skills/upstream-sync/scripts/05-write-report.ps1 | 11 ++++++++--- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index f180d9a8d..73abfb8dd 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -103,7 +103,7 @@ if ($pickCode -eq 0) { # Tier-1 check: did we just create an empty commit (allowed by --keep-redundant-commits)? $changed = git diff-tree --no-commit-id --name-only -r HEAD if (-not $changed) { - $commitMessage = (git log -1 --format='%B' HEAD) -join "`n" + $commitMessage = @(git log -1 --format='%B' HEAD) -join "`n" $expectedFooter = "\(cherry picked from commit $([regex]::Escape($fullSha))\)" if ($commitMessage -notmatch $expectedFooter) { throw "Refusing to reset --hard ${prePickHead}: HEAD does not contain the cherry-pick footer for $fullSha. Investigate before retrying." diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 285fb2e89..870524a2c 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -144,11 +144,16 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { $lines.Add("**How to resume:**") $lines.Add("") $lines.Add("1. ``git switch $($Ctx.Branch)``") - $lines.Add("2. Manually cherry-pick the stuck commit and resolve:") - $lines.Add(" ``````") + $lines.Add("2. Manually cherry-pick the stuck commit and resolve. Pin upstream identity/dates so the resolved commit matches the rest of this batch (otherwise the resolved commit's committer would be you, not the original upstream author — breaking the per-commit attribution the rest of the sync preserves):") + $lines.Add(" ``````pwsh") + $lines.Add(" `$info = (git -C `"`$((git -C . rev-parse --show-toplevel))`" log -1 --pretty=format:'%an%x00%ae%x00%aI%x00%cn%x00%ce%x00%cI' $($Ctx.StuckSha)) -split [char]0") + $lines.Add(" `$env:GIT_AUTHOR_NAME=`$info[0]; `$env:GIT_AUTHOR_EMAIL=`$info[1]; `$env:GIT_AUTHOR_DATE=`$info[2]") + $lines.Add(" `$env:GIT_COMMITTER_NAME=`$info[3]; `$env:GIT_COMMITTER_EMAIL=`$info[4]; `$env:GIT_COMMITTER_DATE=`$info[5]") $lines.Add(" git cherry-pick -x $($Ctx.StuckSha)") $lines.Add(" # resolve conflicts, then:") - $lines.Add(" git add -A && git cherry-pick --continue") + $lines.Add(" git add -A; git cherry-pick --continue --no-edit") + $lines.Add(" # finally, clear the env vars so they don't leak into your next commit:") + $lines.Add(" Remove-Item Env:GIT_AUTHOR_NAME,Env:GIT_AUTHOR_EMAIL,Env:GIT_AUTHOR_DATE,Env:GIT_COMMITTER_NAME,Env:GIT_COMMITTER_EMAIL,Env:GIT_COMMITTER_DATE -ErrorAction SilentlyContinue") $lines.Add(" ``````") $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual resolution for $($Ctx.StuckSha.Substring(0,9))``, merge it.") $lines.Add("4. Clear the lock:") From dbbf346071ab978a59f138997400ff7fd15de9f8 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 10:58:37 +0800 Subject: [PATCH 37/82] docs(upstream-sync): correct 05 help block + 02 return-shape doc drift MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three doc-only fixes from Copilot review on PR #218 (round 20): 1. 05-write-report.ps1 .PARAMETER Status — expand the help comment to list the full ValidateSet (dry-run, skipped-pr-open, and the four stuck-* Tier-4 variants were missing). 2. workflow.md step 2 — said 02-compute-pending.ps1 emits a "JSON array"; it actually emits an object. Updated and cross-referenced step 3 for the full shape. 3. workflow.md step 3 — the abbreviated return-shape `{ pending, dropped_pairs }` was missing `from`, `to`, and `skipped_empty`, which a parser would silently drop. Replaced with the full shape. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/references/workflow.md | 5 +++-- .github/skills/upstream-sync/scripts/05-write-report.ps1 | 4 +++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index 2f1cd7683..9b1affd0e 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -35,7 +35,7 @@ Oldest-first ordering is mandatory. Cherry-picking newest-first inverts dependencies and creates spurious conflicts. Script: [`02-compute-pending.ps1`](../scripts/02-compute-pending.ps1) emits -a JSON array on stdout. +a JSON object on stdout — see step 3 below for the full shape. ### 3. Detect & drop revert pairs @@ -47,7 +47,8 @@ its body contains `This reverts commit <40-hex>`. - If `<40-hex>` is **outside** the pending range (already synced earlier) → keep the revert; it must land as a normal pick. -Script: same `02-compute-pending.ps1` (returns `{ pending: [...], dropped_pairs: [[A,B],...] }`). +Script: same `02-compute-pending.ps1`. Full return shape: +`{ from: "", to: "", pending: [...], dropped_pairs: [[A,B],...], skipped_empty: [...] }`. ### 4. Drop upstream-empty commits diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 870524a2c..213647e11 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -12,7 +12,9 @@ Upstream HEAD SHA at fetch time. .PARAMETER Status - ok | no-op | stuck | skipped-locked + ok | no-op | dry-run | stuck | skipped-locked | skipped-pr-open + | stuck-static-scan | stuck-build-failed | stuck-build-inconclusive + | stuck-toolchain-missing .OUTPUTS Absolute path to the written report file. From 548a29e3a1fd3832e76e2a495911045d6fe072fc Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 11:09:21 +0800 Subject: [PATCH 38/82] fix(upstream-sync): use repo-root path in bootstrap hint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Common.ps1 Read-State — the "run 00-bootstrap.ps1 first" message used the relative path `scripts/00-bootstrap.ps1`, but documentation elsewhere in this PR consistently invokes it from the repo root (`.github/skills/upstream-sync/scripts/00-bootstrap.ps1`). Updated the error message to match the documented invocation path so users don't get a "file not found" trying the relative form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/Common.ps1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 1ad2ec932..41ce46ea8 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -51,7 +51,7 @@ function ConvertTo-RepoRelativePath { function Read-State { $p = Get-StatePath if (-not (Test-Path $p)) { - throw "state.json not found at $p. Run scripts/00-bootstrap.ps1 first — see references/bootstrap.md." + throw "state.json not found at $p. Run .github/skills/upstream-sync/scripts/00-bootstrap.ps1 (from the repo root) first — see references/bootstrap.md." } return Get-Content -Raw -LiteralPath $p | ConvertFrom-Json } From e7c0216dbdba7db2627ff60245fb67d5c797cbbb Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 11:21:01 +0800 Subject: [PATCH 39/82] fix(upstream-sync): bootstrap example + direct-push schema + toolsets spelling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three fixes from Copilot review on PR #218 (round 22): 1. 00-bootstrap.ps1 .EXAMPLE — change `pwsh scripts/00-bootstrap.ps1` to the documented repo-root path `pwsh .github/skills/upstream-sync/scripts/00-bootstrap.ps1`. Copy-pasted by users; was failing unless they happened to be in the scripts/ folder. 2. state-schema.md — document the optional `merge_mode` and `main_head_sha` fields written by `06b-finalize-direct.ps1` under `-PushDirectToMain`. Previously the schema doc and on-disk state.json could diverge silently when direct-push was used. 3. spelling allow/patterns — moved `toolsets` to allow.txt (real English word, plural of existing `toolset`), and narrowed the patterns.txt regex from `\w*[Tt]oolsets` to `\w+_[Tt]oolsets` so the pattern only matches identifier-like forms (e.g. `required_toolsets`, `available_toolsets`) and no longer over-suppresses the plain word "toolsets" in prose. Matches the repo convention (allow = stable real words, patterns = identifier regexes only). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/allow/allow.txt | 1 + .github/actions/spelling/patterns/patterns.txt | 2 +- .github/skills/upstream-sync/references/state-schema.md | 7 ++++++- .github/skills/upstream-sync/scripts/00-bootstrap.ps1 | 2 +- 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/.github/actions/spelling/allow/allow.txt b/.github/actions/spelling/allow/allow.txt index 2a4186ba9..5c01c78fd 100644 --- a/.github/actions/spelling/allow/allow.txt +++ b/.github/actions/spelling/allow/allow.txt @@ -236,6 +236,7 @@ tokio tombstoned toolpath toolset +toolsets trn TUs txs diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index df21db803..7d269b106 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -323,5 +323,5 @@ env_remove\("[A-Z_]+"\) # upstream-sync skill: identifiers and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|\w*[Tt]oolsets)\b +\b(?:u(?:sha|pid)|quotepath|\w+_[Tt]oolsets)\b \b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md index cbbea9338..67293f1e4 100644 --- a/.github/skills/upstream-sync/references/state-schema.md +++ b/.github/skills/upstream-sync/references/state-schema.md @@ -50,7 +50,12 @@ Path: `.github/upstream-sync/state.json` (committed on `main`). "picked_count": 7, "dropped_pair_count": 1, "empty_count": 2, - "tier0_resolutions": 1 + "tier0_resolutions": 1, + + // OPTIONAL — only set by the -PushDirectToMain code path + // (06b-finalize-direct.ps1). Omitted on normal PR runs. + "merge_mode": "direct-push", // "pr" (default, implicit) | "direct-push" + "main_head_sha": "1234567890abcdef..." // SHA of origin/main after the direct push }, // Rolling history — keep last 20 runs. Only `ok` and `stuck*` runs diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 index a43fcf2fb..1f659a7e2 100644 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 @@ -15,7 +15,7 @@ accidentally rewinding the baseline. .EXAMPLE - pwsh scripts/00-bootstrap.ps1 -BaselineSha 93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f + pwsh .github/skills/upstream-sync/scripts/00-bootstrap.ps1 -BaselineSha 93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f #> [CmdletBinding()] param( From 8e23d749b4d2bffa9dff9ed8deb571b2cec1c9da Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 11:32:11 +0800 Subject: [PATCH 40/82] fix(upstream-sync): bump git prereq to 2.38 + narrow toolsets pattern MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 23): 1. SKILL.md prerequisites — bump documented git from 2.30+ to 2.38+ because `scripts/03-cherry-pick-one.ps1` uses `git cherry-pick --keep-redundant-commits`, which was introduced in Git 2.38. Runs on 2.30–2.37 would fail immediately. 2. spelling patterns.txt — narrow `\w+_[Tt]oolsets` to the specific identifiers actually introduced by upstream-sync (`required_toolsets`, `available_toolsets`, `missing_toolsets`). The broad form would mask real misspellings in any future `_toolsets`-suffixed snake_case token elsewhere in the repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/patterns/patterns.txt | 2 +- .github/skills/upstream-sync/SKILL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index 7d269b106..e2dfb2b8b 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -323,5 +323,5 @@ env_remove\("[A-Z_]+"\) # upstream-sync skill: identifiers and timestamp placeholders \b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|\w+_[Tt]oolsets)\b +\b(?:u(?:sha|pid)|quotepath|(?:required|available|missing)_[Tt]oolsets)\b \b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 3cdd23708..9727fd889 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -27,7 +27,7 @@ conflict appears. ## Prerequisites -- `git` 2.30+ and `gh` CLI authenticated against `microsoft/intelligent-terminal`. +- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). - Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` From d758d670af974e698032d2e664b92f6d6f9a8c24 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 11:43:11 +0800 Subject: [PATCH 41/82] fix(upstream-sync): preserve existing BOM in Tier-2 line-ending snippet MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit conflict-triage.md line-ending snippet — replaced the unconditional `UTF8Encoding($true)` (always-BOM) write with a small `$hasBom` probe that detects the file's existing BOM and re-emits it the same way. UTF-8-with-BOM is right for .resw / .csproj on this repo, but UTF-8-without-BOM is right for many .yml / .md files; the old snippet would silently add a BOM and either pollute the diff or break tooling that expects bare UTF-8. Note: the companion review thread about the PR description counts ("6 references + 11 scripts" → actually 10 references + 15 scripts) was fixed by editing the PR description on GitHub directly, not via a file change in the repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/upstream-sync/references/conflict-triage.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/conflict-triage.md index 8920c78fd..becd63f9f 100644 --- a/.github/skills/upstream-sync/references/conflict-triage.md +++ b/.github/skills/upstream-sync/references/conflict-triage.md @@ -152,8 +152,13 @@ re-normalize before staging: ```pwsh # Inside Tier-2, after writing the resolved content: $bytes = [System.IO.File]::ReadAllBytes($p) -$text = [System.Text.Encoding]::UTF8.GetString($bytes) -replace "`r?`n", "`r`n" -[System.IO.File]::WriteAllText($p, $text, (New-Object System.Text.UTF8Encoding($true))) # BOM +# Preserve the file's original BOM presence — UTF-8-with-BOM is right +# for .resw / .csproj on this repo, but UTF-8-without-BOM is right for +# many .yml / .md files. Adding a BOM where one wasn't there before +# introduces unrelated encoding diffs and can break tooling. +$hasBom = $bytes.Length -ge 3 -and $bytes[0] -eq 0xEF -and $bytes[1] -eq 0xBB -and $bytes[2] -eq 0xBF +$text = [System.Text.Encoding]::UTF8.GetString($bytes) -replace "`r?`n", "`r`n" +[System.IO.File]::WriteAllText($p, $text, (New-Object System.Text.UTF8Encoding($hasBom))) ``` (Skipping this is how the winget-pkgs submission broke last time — From 532463b0b105a62f41f853234846e639bfd82a57 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 11:54:08 +0800 Subject: [PATCH 42/82] fix(upstream-sync): document push perms + propagate non-conflict cherry-pick error MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three fixes from Copilot review on PR #218 (round 25): 1. SKILL.md prerequisites — add an explicit note that the gh/git credential must be able to push directly to `main` (bypass branch protection if the branch is protected) because the stuck-handling scripts (07-open-stuck-issue.ps1, 07b-open-validation-stuck-issue.ps1, clear-stuck.ps1) commit + push the lock to main. Without that perm, unattended runs failed mid-stuck-handling with a generic git error. 2. 04-run-batch.ps1 stuck case — capture the `error` field that 03-cherry-pick-one.ps1 emits for non-conflict failures (merge commit without -m, hook failure) into `Ctx.StuckError`, and split the warning so empty conflict_paths no longer prints "Stuck at on paths: " with a trailing empty list. 3. 05-write-report.ps1 stuck section — when `Ctx.StuckPaths` is empty, emit an explicit "No unmerged paths" block with the captured error instead of an empty bullet list. Resume guidance below still applies once the human addresses the underlying error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 2 +- .../upstream-sync/scripts/04-run-batch.ps1 | 8 ++++++- .../upstream-sync/scripts/05-write-report.ps1 | 21 +++++++++++++++---- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 9727fd889..7a61f8180 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -27,7 +27,7 @@ conflict appears. ## Prerequisites -- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. +- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential **must also be able to push directly to `main`** (i.e. bypass branch protection if the branch is protected): `07-open-stuck-issue.ps1`, `07b-open-validation-stuck-issue.ps1`, and `clear-stuck.ps1` write the stuck-lock + report into `state.json` on `main` via a direct push. Without that permission, unattended runs will fail mid-stuck-handling with a generic git push error. - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). - Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index 5ca769c73..aefbdb990 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -239,8 +239,14 @@ try { 'stuck' { $ctx.StuckSha = $sha $ctx.StuckPaths = @($res.conflict_paths) + $ctx.StuckError = if ($res.PSObject.Properties.Name -contains 'error') { [string]$res.error } else { $null } $ctx.Status = 'stuck' - Write-Warning "Stuck at $sha on paths: $($res.conflict_paths -join ', ')" + $errSuffix = if ($ctx.StuckError) { " (error: $($ctx.StuckError))" } else { '' } + if ($ctx.StuckPaths.Count -gt 0) { + Write-Warning "Stuck at $sha on paths: $($ctx.StuckPaths -join ', ')$errSuffix" + } else { + Write-Warning "Stuck at $sha — no conflict paths reported$errSuffix" + } break } default { Exit-Hard "Unknown cherry-pick-one status: $($res.status)" } diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 213647e11..1bd7e53fa 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -137,10 +137,23 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { $lines.Add("**Conflicting commit:** [`$($Ctx.StuckSha)`](https://github.com/microsoft/terminal/commit/$($Ctx.StuckSha)) — $stuckSubj ") $lines.Add("**Author:** $stuckAuthor") $lines.Add("") - $lines.Add("**Files in conflict:**") - $lines.Add("") - foreach ($p in $Ctx.StuckPaths) { $lines.Add("- ``$p``") } - $lines.Add("") + $stuckError = if ($Ctx.PSObject.Properties.Name -contains 'StuckError') { $Ctx.StuckError } else { $null } + if ($Ctx.StuckPaths -and $Ctx.StuckPaths.Count -gt 0) { + $lines.Add("**Files in conflict:**") + $lines.Add("") + foreach ($p in $Ctx.StuckPaths) { $lines.Add("- ``$p``") } + $lines.Add("") + } else { + # Non-conflict cherry-pick failure (e.g. merge commit picked without -m, + # hook failure). The resume guidance below still applies after the human + # addresses the underlying error. + $lines.Add("**No unmerged paths** — ``git cherry-pick`` failed for a reason other than a merge conflict (e.g. attempting to pick a merge commit without ``-m``, hook failure, or another non-conflict error).") + if ($stuckError) { + $lines.Add("") + $lines.Add("**Reported error:** ``$stuckError``") + } + $lines.Add("") + } $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") $lines.Add("") $lines.Add("**How to resume:**") From ac2aa281d67b667610ac8c22b1506b575fc2554d Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 12:05:05 +0800 Subject: [PATCH 43/82] fix(upstream-sync): trim git rev-parse output for scan baseline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 04-run-batch.ps1 — `$preBase = git rev-parse origin/main` is used later in `"$preBase..HEAD"` ranges; trim defensively in case the underlying git shim emits a trailing \r. Matches the pattern already used elsewhere in the codebase (e.g. 07b-open-validation-stuck-issue.ps1 calls `.Trim()` on the same git output). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/04-run-batch.ps1 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index aefbdb990..bbeba530a 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -206,7 +206,9 @@ try { } # Capture pre-pick base SHA (origin/main) — used as static-scan baseline. - $preBase = git rev-parse origin/main + # Trim defensively: native git output occasionally carries a trailing \r + # depending on shim, and an untrimmed SHA breaks `"$Base..$Head"` ranges. + $preBase = (git rev-parse origin/main).Trim() if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not resolve origin/main for scan baseline." } # --- 3. Create / switch to sync branch --- From ccb0126441cabeff89cc1bc08a1c195d9aba1ed0 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 12:16:16 +0800 Subject: [PATCH 44/82] fix(upstream-sync): separate stderr from JSON + finally-cleanup PR body temp MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from Copilot review on PR #218 (round 27): 1. 04-run-batch.ps1 existing-PR gate — `gh pr list --json` was captured with `2>&1`, so a stray `gh` stderr message (deprecation notice, version-update banner) at exit 0 would break the `ConvertFrom-Json` parse. Now stdout is captured directly (clean JSON) and stderr is redirected to a temp file; the temp file is read only when surfacing the failure message and is cleaned up in `finally`. 2. 06-finalize-pr.ps1 PR-create — wrap the 3-retry loop in `try { ... } finally { Remove-Item $bodyPath }` so the temp PR body file is cleaned up even when `gh pr create` exhausts all retries and the script throws. Previously the temp file leaked into `%TEMP%` on any final-attempt failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/04-run-batch.ps1 | 26 ++++++++++++++----- .../upstream-sync/scripts/06-finalize-pr.ps1 | 23 +++++++++------- 2 files changed, 34 insertions(+), 15 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index bbeba530a..d256ad1d6 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -146,12 +146,26 @@ try { # open. List all open PRs (--limit 200 covers the corner case # where a repo has more than the default 30 open) and filter # client-side by headRefName. - $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>&1 - if ($LASTEXITCODE -ne 0) { - # `gh` missing / not authenticated / network-blocked. Don't - # silently continue and waste a full pick + scan + build - # only to fail later in 06-finalize-pr.ps1 — fail fast. - Exit-Hard "gh pr list failed (exit $LASTEXITCODE): $existingJson. The existing-PR gate requires gh to be installed and authenticated. Re-run with -Force to bypass (at your own risk), or with -DryRun / -PushDirectToMain to skip the gate." + # + # Capture stdout (JSON) and stderr separately: merging them with + # `2>&1` breaks ConvertFrom-Json when `gh` emits any warning or + # progress text on stderr even at exit 0 (e.g. version-update + # notice, deprecation warning). stderr is only used for the + # failure message. + $errFile = [System.IO.Path]::GetTempFileName() + try { + $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>$errFile + $ghExit = $LASTEXITCODE + if ($ghExit -ne 0) { + # `gh` missing / not authenticated / network-blocked. Don't + # silently continue and waste a full pick + scan + build + # only to fail later in 06-finalize-pr.ps1 — fail fast. + $errText = if (Test-Path $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + Exit-Hard "gh pr list failed (exit $ghExit): $errText. The existing-PR gate requires gh to be installed and authenticated. Re-run with -Force to bypass (at your own risk), or with -DryRun / -PushDirectToMain to skip the gate." + } + } + finally { + Remove-Item -LiteralPath $errFile -ErrorAction SilentlyContinue } if ($existingJson) { $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index a24fee59e..9e5d79358 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -88,20 +88,25 @@ $title = "chore(upstream): sync microsoft/terminal up to $shortTo" # Retry up to 3 times with a short delay: `gh pr create` on Windows occasionally fails # with "Head sha can't be blank" right after a push (see SKILL.md gotcha). $prUrl = $null -for ($attempt = 1; $attempt -le 3; $attempt++) { - $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 - if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } - Write-Warning "gh pr create attempt $attempt failed: $prUrl" - Start-Sleep -Seconds 5 +try { + for ($attempt = 1; $attempt -le 3; $attempt++) { + $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 + if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } + Write-Warning "gh pr create attempt $attempt failed: $prUrl" + Start-Sleep -Seconds 5 + } + if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { + throw "gh pr create did not return a PR URL after 3 attempts. Last output: $prUrl" + } } -if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { - throw "gh pr create did not return a PR URL after 3 attempts. Last output: $prUrl" +finally { + # Always clean up the temp PR body file — even if `gh pr create` failed + # after all retries, the temp file should not leak in %TEMP%. + Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue } $Ctx.PrUrl = $prUrl.Trim() -Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue - # Backfill PR URL into state.last_run AND state.history[0] (best-effort # follow-up commit) BEFORE arming auto-merge. If auto-merge is already # satisfied (all checks green, approvals in place) it can merge and delete From d4278bca76e5ad92727b3ae33cfff260ea0995b3 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 12:27:45 +0800 Subject: [PATCH 45/82] fix(upstream-sync): emit repo-relative log_path in 10-try-build report The build report JSON included an absolute log_path (e.g. C:\Users\\repo\.github\upstream-sync\build-logs\.log) which leaks machine-specific details (username, drive letter, clone location) into the GitHub issues / reports the skill posts. Normalize via ConvertTo-RepoRelativePath when the log dir is under the repo root (the default), and fall back to the absolute path only when the caller passed a custom -LogDir outside the repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/10-try-build.ps1 | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index 49815eb1e..3c1dcb71c 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -119,12 +119,18 @@ try { @(Get-Content -LiteralPath $logPath -Tail 200) -join "`n" } else { '' } + # Emit a repo-relative log_path when the log lives inside the repo + # (the common case). Absolute paths leak machine-specific details + # like username + drive letter into GitHub issues/reports. Fall back + # to the absolute path when the user passed a custom -LogDir that + # sits outside the repo root. + $logPathForReport = try { ConvertTo-RepoRelativePath $logPath } catch { $logPath } $doc = [ordered] @{ kind = $kind exit_code = $exitCode duration_ms = $durationMs command = $BuildCommand - log_path = $logPath + log_path = $logPathForReport log_tail = $tailLines } $doc | ConvertTo-Json -Depth 4 From 05c3114dde8e1a3a2744bf290b99fcdeff2a1d61 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 12:40:01 +0800 Subject: [PATCH 46/82] fix(upstream-sync): guard git add exit code and tighten invariants MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five small robustness fixes from round-30 Copilot review: * clear-stuck.ps1: throw if git add of state.json fails BEFORE the git commit (so a staging failure surfaces immediately instead of producing a confusing "nothing to commit" later). * 06b-finalize-direct.ps1 / 06-finalize-pr.ps1: same git add exit-code check, but warn-and-skip (these backfill commits are intentionally best-effort cosmetic — they must not abort an otherwise-successful sync). * 09-toolchain-preflight.ps1: sort the toolset/probed arrays before emitting JSON. They were built from HashSet, whose enumeration order is non-deterministic, so two runs on the same machine produced different JSON and noisy diffs in embedded reports / stuck-issue payloads. * fork-invariants.json (C4459): tighten the must_contain_regex to require the digits inside ... . The previous \b4459\b would spuriously satisfy the invariant if 4459 showed up in a comment after a take-upstream cherry-pick stripped the real suppression. Verified the tightened regex still matches the real props line and rejects a comment-only mention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/references/fork-invariants.json | 2 +- .../skills/upstream-sync/scripts/06-finalize-pr.ps1 | 12 ++++++++---- .../upstream-sync/scripts/06b-finalize-direct.ps1 | 12 ++++++++---- .../upstream-sync/scripts/09-toolchain-preflight.ps1 | 8 ++++---- .github/skills/upstream-sync/scripts/clear-stuck.ps1 | 1 + 5 files changed, 22 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/references/fork-invariants.json b/.github/skills/upstream-sync/references/fork-invariants.json index a78ac35c5..cda885126 100644 --- a/.github/skills/upstream-sync/references/fork-invariants.json +++ b/.github/skills/upstream-sync/references/fork-invariants.json @@ -5,7 +5,7 @@ { "id": "common-build-c4459-suppression", "path": "src/common.build.pre.props", - "must_contain_regex": "\\b4459\\b", + "must_contain_regex": "[^<]*\\b4459\\b[^<]*", "severity": "high", "reason": "Fork added C4459 (declaration of 'X' hides class member) suppression because TreatWarningAsError=true and fork-specific code triggers it. Upstream removed C4459 in the ATL/MFC cleanup; a take-upstream resolution drops it and the next build fails. See PR #220 audit (2026-06-04).", "introduced_in_fork_sha": "4bf2b6a45", diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 9e5d79358..fab5a1c35 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -124,10 +124,14 @@ if ($state.history -and $state.history.Count -gt 0) { } Write-State $state git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null -git commit -m "chore(upstream-sync): record PR url" | Out-Host -if ($LASTEXITCODE -eq 0) { - git push origin $branch | Out-Host - if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push pr_url backfill; PR is still open at $($Ctx.PrUrl)." } +if ($LASTEXITCODE -ne 0) { + Write-Warning "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); skipping pr_url backfill commit. PR is still open at $($Ctx.PrUrl)." +} else { + git commit -m "chore(upstream-sync): record PR url" | Out-Host + if ($LASTEXITCODE -eq 0) { + git push origin $branch | Out-Host + if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push pr_url backfill; PR is still open at $($Ctx.PrUrl)." } + } } # Optional: arm GitHub auto-merge with the strategy that preserves per-commit diff --git a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 index 70feccaaf..3682d7fe5 100644 --- a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 +++ b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 @@ -97,10 +97,14 @@ if ($state.last_run -and $state.history -and $state.history.Count -gt 0) { $state.history[0].main_head_sha = $mainHead Write-State $state git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null - git commit -m "chore(upstream-sync): record main head $($mainHead.Substring(0,9))" | Out-Host - if ($LASTEXITCODE -eq 0) { - git push origin main | Out-Host - if ($LASTEXITCODE -ne 0) { Write-Warning "Backfill push failed (cosmetic only; sync content already on main)." } + if ($LASTEXITCODE -ne 0) { + Write-Warning "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); skipping main_head_sha backfill (cosmetic only; sync content already on main)." + } else { + git commit -m "chore(upstream-sync): record main head $($mainHead.Substring(0,9))" | Out-Host + if ($LASTEXITCODE -eq 0) { + git push origin main | Out-Host + if ($LASTEXITCODE -ne 0) { Write-Warning "Backfill push failed (cosmetic only; sync content already on main)." } + } } } diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index 94491f473..c1ce44996 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -124,11 +124,11 @@ try { $ok = (-not $probeStale) -and ($missing.Count -eq 0) -and ($vsInstalls.Count -gt 0) $doc = [ordered] @{ - required_toolsets = @($required) - available_toolsets = @($available) - missing = @($missing) + required_toolsets = @($required | Sort-Object) + available_toolsets = @($available | Sort-Object) + missing = @($missing | Sort-Object) vs_installs = @($vsInstalls) - probed_files = @($probed) + probed_files = @($probed | Sort-Object) probe_stale = $probeStale ok = $ok } diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index 7557d1cd8..c1946b20c 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -97,6 +97,7 @@ $state.history = @($entry) + @($state.history) | Select-Object -First 20 Write-State $state git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null +if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); lock is NOT cleared on origin/main." } $shortLabel = if ($resolvedFullSha) { $resolvedFullSha.Substring(0,9) } else { 'no-advance' } git commit -m "chore(upstream-sync): clear stuck-lock ($shortLabel)" | Out-Host if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } From b0aa32d18a7f9dbe927c826a00b1ecb4f8aa63ad Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 12:52:48 +0800 Subject: [PATCH 47/82] fix(upstream-sync): pscustomobject nesting + pin gh label create to repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two real bugs from round-31 Copilot review (six others on the same review were verified false positives — see PR comments): * 09-toolchain-preflight.ps1: `Get-RequiredToolsets` returned a pscustomobject with `Toolsets = ,@($found)` / `ProbedFiles = ,@($probed)`. pscustomobject property assignment does NOT unroll a comma-wrapped array (verified: `Items = ,@('a','b')` has Count=1 + first element of type Object[]), so `\.Toolsets` was actually a 1-element array wrapping the real toolset array. The downstream `\ = @(\ | ... -notcontains \)` happened to "work" by coincidence because the Where pipeline re-unrolled, but `\.Count` and similar were wrong. Drop the leading comma — pscustomobject preserves arrays as-is. * 07 / 07b stuck-issue scripts: `gh label create 'upstream-sync- stuck' ...` was missing `-R microsoft/intelligent-terminal`. Both scripts already note explicitly that the `upstream` remote can trick gh into defaulting to microsoft/terminal — the label call needs the same pinning as the adjacent `gh issue create -R` (where this account has no label-create permission). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/upstream-sync/scripts/07-open-stuck-issue.ps1 | 5 ++++- .../scripts/07b-open-validation-stuck-issue.ps1 | 6 ++++-- .../skills/upstream-sync/scripts/09-toolchain-preflight.ps1 | 4 ++-- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index 34b8bcd81..efd83ecb6 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -48,7 +48,10 @@ $tmp = New-TemporaryFile [System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) # Ensure label exists (best-effort; ignore if already present). -gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' 2>$null | Out-Null +# -R is pinned for the same reason as the issue-create call below: an `upstream` +# remote makes `gh` default to microsoft/terminal, where this account does not +# have label-create permission. +gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null # -R is explicit because the `upstream` remote can make gh default to microsoft/terminal. $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 0d54d17a1..3429a2388 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -112,8 +112,10 @@ Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). $tmp = New-TemporaryFile [System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - # Ensure label exists (best-effort). - gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' 2>$null | Out-Null + # Ensure label exists (best-effort). -R pinned for the same reason as the + # issue-create call below (avoid the `upstream` remote tricking gh into + # microsoft/terminal). + gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 Remove-Item $tmp -Force diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 index c1ce44996..fd31f264d 100644 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 @@ -66,8 +66,8 @@ function Get-RequiredToolsets { } } return [pscustomobject] @{ - Toolsets = ,@($found) - ProbedFiles = ,@($probed) + Toolsets = @($found) + ProbedFiles = @($probed) } } From b14b7a339add9116dc47ce371763bde1ae515714 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 13:04:37 +0800 Subject: [PATCH 48/82] fix(upstream-sync): separate stderr capture for gh pr/issue create MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-32 Copilot review: the three remaining `2>&1 | Select-Object -Last 1` patterns (06-finalize-pr.ps1 gh pr create, 07/07b gh issue create) had the same fragility that round 28 fixed in 04-run-batch.ps1's existing-PR gate: If gh prints any line on stderr (version-upgrade notice, deprecation warning, OneDrive-redirected-credentials notice, etc.), `2>&1` merges it into stdout and `Select-Object -Last 1` returns that warning text instead of the URL. The URL-regex match then fails and the script throws "did not return a URL" — even though the PR / issue was successfully created on GitHub. Apply the same pattern as 04-run-batch.ps1's gate: * Redirect stderr to its own temp file (`2>`). * Parse only stdout for the URL. * Read the stderr capture into the failure message so unattended runs (the scheduler use case) get an actionable diagnosis. * ry/finally cleanup for both the body-file and the err-file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/06-finalize-pr.ps1 | 21 ++++++++++++++----- .../scripts/07-open-stuck-issue.ps1 | 19 +++++++++++++---- .../07b-open-validation-stuck-issue.ps1 | 20 ++++++++++++++---- 3 files changed, 47 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index fab5a1c35..0f468f25f 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -88,21 +88,32 @@ $title = "chore(upstream): sync microsoft/terminal up to $shortTo" # Retry up to 3 times with a short delay: `gh pr create` on Windows occasionally fails # with "Head sha can't be blank" right after a push (see SKILL.md gotcha). $prUrl = $null +$errFile = [System.IO.Path]::GetTempFileName() try { for ($attempt = 1; $attempt -le 3; $attempt++) { - $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>&1 | Select-Object -Last 1 + # Capture stderr separately: merging via `2>&1` can let a `gh` warning + # (version notice, deprecation, etc.) become the last line, after + # which `Select-Object -Last 1` returns the warning text and the URL + # match fails even though the PR was successfully created. The temp + # file is reused across retries (overwritten each call); cleanup runs + # in the outer finally. + Set-Content -LiteralPath $errFile -Value '' -NoNewline + $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>$errFile | Select-Object -Last 1 if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } - Write-Warning "gh pr create attempt $attempt failed: $prUrl" + $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + Write-Warning "gh pr create attempt $attempt failed (exit $LASTEXITCODE): stdout='$prUrl' stderr='$errText'" Start-Sleep -Seconds 5 } if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { - throw "gh pr create did not return a PR URL after 3 attempts. Last output: $prUrl" + $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + throw "gh pr create did not return a PR URL after 3 attempts. Last stdout: '$prUrl'. Last stderr: '$errText'." } } finally { - # Always clean up the temp PR body file — even if `gh pr create` failed - # after all retries, the temp file should not leak in %TEMP%. + # Always clean up the temp PR body file and stderr-capture file — even if + # `gh pr create` failed after all retries, neither temp file should leak. Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue } $Ctx.PrUrl = $prUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index efd83ecb6..f1b451525 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -54,10 +54,21 @@ $tmp = New-TemporaryFile gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null # -R is explicit because the `upstream` remote can make gh default to microsoft/terminal. -$issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 -Remove-Item $tmp -Force -if ($LASTEXITCODE -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed: $issueUrl" +# Capture stderr to a separate temp file so a `gh` warning on stderr (version notice etc.) +# can't displace the URL as the "last line" of merged output. +$errFile = [System.IO.Path]::GetTempFileName() +$errText = '' +try { + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } +} +finally { + Remove-Item $tmp -Force -ErrorAction SilentlyContinue + Remove-Item $errFile -Force -ErrorAction SilentlyContinue +} +if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" } $Ctx.IssueUrl = $issueUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index 3429a2388..d3f9ec0e2 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -117,10 +117,22 @@ Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). # microsoft/terminal). gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>&1 | Select-Object -Last 1 - Remove-Item $tmp -Force - if ($LASTEXITCODE -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed: $issueUrl" + # Capture stderr to a separate temp file so a `gh` warning on stderr (version + # notice etc.) can't displace the URL as the "last line" of merged output — + # `state.stuck_validation.issue_url` must always be either a real URL or unset. + $errFile = [System.IO.Path]::GetTempFileName() + $errText = '' + try { + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } + } + finally { + Remove-Item $tmp -Force -ErrorAction SilentlyContinue + Remove-Item $errFile -Force -ErrorAction SilentlyContinue + } + if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" } $validation.issue_url = $issueUrl.Trim() $Ctx.IssueUrl = $validation.issue_url From 0ca6911516c02b0bf466c7950d8efc08a5f80317 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 13:15:37 +0800 Subject: [PATCH 49/82] fix(upstream-sync): fast-forward main before reading state.json MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-33 Copilot review: both `04-run-batch.ps1` (the scheduler entry point) and `clear-stuck.ps1` (the operator recovery script) read `state.json` and made the stuck-lock decision BEFORE doing `git switch main` + `git pull --ff-only origin main`. The single-active-lock + `last_synced_upstream_sha` invariants live on origin/main, so reading from a stale local clone created two real bugs: * 04-run-batch.ps1 could miss a stuck-lock that was set on origin/main by a concurrent run on another host (or by an operator) — the scheduler would then proceed to fetch/pick/PR, defeating the whole "single active lock" safety model the spec relies on. * clear-stuck.ps1 could falsely conclude "no stuck-lock is set" and early-return when origin/main actually had one set; or, even with a lock locally, could later overwrite a newer `state.json` from origin/main when the clear-stuck commit landed on top. Fix in both scripts: hoist the FF block (`Assert-CleanWorktree` + `git switch main` + `git pull --ff-only origin main`) BEFORE the `Read-State` call so every state-based decision and every state mutation operates on the authoritative origin/main copy. 04-run-batch.ps1 no longer has the duplicate FF block immediately before "--- 1. Fetch upstream ---" (it was redundant after the hoist). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/04-run-batch.ps1 | 23 +++++++++++++++---- .../upstream-sync/scripts/clear-stuck.ps1 | 23 ++++++++++++------- 2 files changed, 33 insertions(+), 13 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index d256ad1d6..d3ddecb15 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -106,9 +106,23 @@ function Invoke-Tier4Stuck { } try { - $state = Read-State $ctx = New-RunContext + # Fast-forward local main from origin BEFORE reading state.json. The + # single-active-lock and last_synced_upstream_sha invariants live on + # origin/main; a stale local clone would let this scheduler proceed + # past a stuck-lock set by a concurrent run on another host or by + # the operator's `clear-stuck.ps1` reverse (defeating the whole + # safety model). Worktree cleanliness is checked first so that an + # unrelated dirty file can't block the FF unexpectedly mid-script. + Assert-CleanWorktree + git switch main 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } + git pull --ff-only origin main 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only origin main failed." } + + $state = Read-State + # --- Stuck-lock gate (Tier-3 OR Tier-4) --- $stuckTier3 = [bool] $state.stuck_on_sha $stuckTier4 = [bool] $state.stuck_validation @@ -180,10 +194,9 @@ try { } Assert-CleanWorktree - git switch main 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } - git pull --ff-only origin main 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only origin main failed." } + # main is already FF'd from origin above (before the stuck-lock + existing-PR + # gates); no need to re-pull here. Re-assert clean state in case the gates + # produced ephemeral artifacts on disk. # --- 1. Fetch upstream --- $toSha = (& "$PSScriptRoot/01-fetch-upstream.ps1").Trim() diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 index c1946b20c..436a8c968 100644 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 @@ -33,14 +33,13 @@ param( . "$PSScriptRoot/Common.ps1" -$state = Read-State -$tier3 = [bool] $state.stuck_on_sha -$tier4 = [bool] $state.stuck_validation -if (-not ($tier3 -or $tier4)) { - Write-Warning "No stuck-lock is set. Nothing to clear." - return -} - +# Fast-forward local main from origin BEFORE reading state.json so the +# stuck-lock decision (and any subsequent mutations) operates on the +# authoritative state from origin/main rather than a stale local copy. +# A stale local main would otherwise let the script falsely conclude +# "no stuck-lock is set" (early-return) when origin/main actually has +# one, or — worse — overwrite a newer state.json from origin/main when +# the clear-stuck commit lands. Assert-CleanWorktree Ensure-UpstreamRemote git fetch upstream main --no-tags | Out-Null @@ -51,6 +50,14 @@ if ($LASTEXITCODE -ne 0) { throw "git switch main failed; refusing to clear stuc git pull --ff-only origin main | Out-Null if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed; refusing to clear stuck-lock until main is current." } +$state = Read-State +$tier3 = [bool] $state.stuck_on_sha +$tier4 = [bool] $state.stuck_validation +if (-not ($tier3 -or $tier4)) { + Write-Warning "No stuck-lock is set on origin/main. Nothing to clear." + return +} + $resolvedFullSha = if ($ResolvedThroughSha) { Resolve-FullCommitSha $ResolvedThroughSha } else { $null } if ($tier3) { From 15c6cbf14750f228547828de4f121c1f21003cd4 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 14:49:55 +0800 Subject: [PATCH 50/82] refactor(upstream-sync): derive state from git+gh, drop state.json MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per PR #218 review feedback: the persisted state.json + reports/ folder duplicates information that already lives in authoritative sources (git trailers on origin/main, open `upstream-sync-stuck` issues), and the reports were never read back into the workflow. This commit removes the duplication and reshapes the skill around derived state. Derived state (new helpers in Common.ps1): - Get-LastSyncedUpstreamSha — most recent `(cherry picked from commit )` trailer on origin/main that is an ancestor of upstream/main; capped at the 5000 most-recent commits for speed. - Get-PendingUpstreamShas — `git log --cherry-pick --right-only` on `origin/main...upstream/main`, then drop ancestors of the watermark. The patch-id filter covers manual cherry- picks landed outside the trailer convention; the watermark only prunes the obviously-old tail for speed. - Get-StuckIssues / Get-StuckMetaFromIssue / Format-StuckYamlBlock — the open labeled issue IS the stuck lock; its body carries a fenced ```yaml # wta-state``` block with tier/kind/findings_hash/etc. Values are single-quoted with `'` -> `''` escaping so colons/newlines/leading- dashes round-trip safely. - Get-GeneratedDir [-Sub] — transient artifacts go to `Generated Files/upstream-sync//` which is gitignored at the repo root. Files removed: - .github/upstream-sync/ (state.json, reports/, gitignore) - scripts/00-bootstrap.ps1 (no state to bootstrap) - scripts/06b-finalize-direct.ps1 (was state-backfill helper) - scripts/clear-stuck.ps1 (replaced by closing the issue) - references/state-schema.md, reporting.md, bootstrap.md Scripts rewritten: - 02-compute-pending.ps1 — now calls Get-PendingUpstreamShas (was a plain `$from..$to` range, which silently missed picked-then-reverted upstream commits). - 04-run-batch.ps1 — Get-StuckIssues replaces Read-State gate; $fromSha from Get-LastSyncedUpstreamSha; no state-update commit on success. - 05-write-report.ps1 — writes to Get-GeneratedDir, never commits; resume hint is "close the labeled issue". - 06-finalize-pr.ps1 — no state commit, no pr_url backfill. - 07 / 07b — open labeled issue with the YAML block as the persistent lock; 07b's toolchain- missing path opens no issue and sets no lock (Tier-4d retries next tick). - 10-try-build.ps1 — build logs under Get-GeneratedDir. Docs: - SKILL.md adds a "State model" section, a "First-time sync" recipe (one `git commit --allow-empty` with the trailer — no scripted bootstrap), a squash-merge recovery playbook, and a single-host concurrency caveat. The Tier-4d wording now matches code (no issue, no lock). - workflow.md / conflict-triage.md / build-verification.md rewritten for the derived-state model and the new build-log location. Backup of pre-redesign state: tag `backup/pre-redesign-pr218` on `origin` at commit 0ca691151. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 114 ++++++-- .../upstream-sync/references/bootstrap.md | 89 ------ .../references/build-verification.md | 23 +- .../references/conflict-triage.md | 23 +- .../upstream-sync/references/reporting.md | 137 --------- .../upstream-sync/references/state-schema.md | 106 ------- .../upstream-sync/references/workflow.md | 122 ++++---- .../upstream-sync/scripts/00-bootstrap.ps1 | 89 ------ .../scripts/02-compute-pending.ps1 | 32 ++- .../upstream-sync/scripts/04-run-batch.ps1 | 165 ++++++----- .../upstream-sync/scripts/05-write-report.ps1 | 80 +++--- .../upstream-sync/scripts/06-finalize-pr.ps1 | 132 +++------ .../scripts/06b-finalize-direct.ps1 | 111 -------- .../scripts/07-open-stuck-issue.ps1 | 115 ++++---- .../07b-open-validation-stuck-issue.ps1 | 216 ++++++-------- .../upstream-sync/scripts/10-try-build.ps1 | 8 +- .../skills/upstream-sync/scripts/Common.ps1 | 265 ++++++++++++++---- .../upstream-sync/scripts/clear-stuck.ps1 | 113 -------- .github/upstream-sync/.gitignore | 11 - .github/upstream-sync/reports/.gitkeep | 0 .github/upstream-sync/state.json | 13 - 21 files changed, 733 insertions(+), 1231 deletions(-) delete mode 100644 .github/skills/upstream-sync/references/bootstrap.md delete mode 100644 .github/skills/upstream-sync/references/reporting.md delete mode 100644 .github/skills/upstream-sync/references/state-schema.md delete mode 100644 .github/skills/upstream-sync/scripts/00-bootstrap.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/clear-stuck.ps1 delete mode 100644 .github/upstream-sync/.gitignore delete mode 100644 .github/upstream-sync/reports/.gitkeep delete mode 100644 .github/upstream-sync/state.json diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 7a61f8180..a36c4b4b1 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -16,25 +16,80 @@ conflict appears. - User asks to "sync upstream", "pull from microsoft/terminal", "catch up to upstream", or "run upstream sync". - A scheduler (Task Scheduler, cron, GitHub Actions) invokes [`scripts/04-run-batch.ps1`](./scripts/04-run-batch.ps1) on a weekly/daily cadence. -- The previous run left a stuck-lock and the human has finished resolving - the conflict — use [`scripts/clear-stuck.ps1`](./scripts/clear-stuck.ps1) and re-run. +- The previous run left an `upstream-sync-stuck` labeled issue open and the + human has finished resolving the conflict — **close the issue** (that IS + the lock-clear signal) and re-run. ## When NOT to Use This Skill - The user wants a **one-shot rebase** of a single feature branch onto upstream — that's a normal `git rebase`, not this skill. -- The fork has never been initialized (`state.json` missing) — first do the one-time bootstrap from [references/bootstrap.md](./references/bootstrap.md). -- A stuck-lock is set — do not re-run; resolve the conflict on the stuck branch first. +- An `upstream-sync-stuck` labeled issue is open on `microsoft/intelligent-terminal` — do not re-run; resolve the conflict on the stuck branch first, then close the issue. ## Prerequisites -- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential **must also be able to push directly to `main`** (i.e. bypass branch protection if the branch is protected): `07-open-stuck-issue.ps1`, `07b-open-validation-stuck-issue.ps1`, and `clear-stuck.ps1` write the stuck-lock + report into `state.json` on `main` via a direct push. Without that permission, unattended runs will fail mid-stuck-handling with a generic git push error. +- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches** (`upstream-sync/`) and **issue + label create** in the same repo (the stuck-lock is an open labeled issue, not a commit on `main`). - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). - Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` (the scripts create it if missing). -- `state.json` initialized once (see [references/bootstrap.md](./references/bootstrap.md)). +- **No `state.json` to bootstrap.** Watermark comes from the + `(cherry picked from commit )` trailers on `origin/main`. If + the fork has never used `cherry-pick -x` (or trailers were stripped), + see "First-time sync" below for the one-time operator step. + +## State model (no `state.json`) + +Every persistent fact lives in the source that owns it: + +| Question | Source of truth | +|---|---| +| What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Computed by `Get-LastSyncedUpstreamSha` in `scripts/Common.ps1`. | +| What's pending? | `git log --cherry-pick --right-only --no-merges ...upstream/main`. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | +| Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | +| What does the lock mean? | A fenced ```yaml # wta-state``` block in the issue body carries `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. (parsed by `Get-StuckMetaFromIssue`). | +| Where do reports + build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | + +### First-time sync + +If the fork has no `(cherry picked from commit )` trailer on +`origin/main` yet, `Get-LastSyncedUpstreamSha` will throw. To seed, +the operator commits an **empty seed commit** carrying the trailer +in its message — the same format `cherry-pick -x` would emit, written +by hand exactly once: + +```pwsh +git remote add upstream https://github.com/microsoft/terminal.git +git fetch upstream main --no-tags +git switch main +# Pick an upstream SHA you consider "already in this fork" (typically the +# commit the fork was originally branched from). Write the trailer EXACTLY +# as cherry-pick -x would so Get-LastSyncedUpstreamSha picks it up. +$seedSha = '<40-char-upstream-sha-already-in-fork>' +git commit --allow-empty -m "chore(upstream): seed upstream-sync watermark" -m "(cherry picked from commit $seedSha)" +git push origin main +``` + +That one merged trailer becomes the watermark. From then on, every +run extends it. No script needed — the bootstrap is just one +`commit --allow-empty` ever. + +### Squash-merge recovery (don't do this, but if you did) + +The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` +locks in the safe path. If a reviewer squash-merges anyway: + +- The squash commit's body usually concatenates every cherry-picked + message, so it contains MANY `(cherry picked from commit )` + trailers. `Get-LastSyncedUpstreamSha` matches the FIRST one it sees + via regex, which is the oldest cherry-pick in the squashed batch. + That means the watermark moves backward, and the next sync run + re-picks everything between the oldest and newest commits of the + squashed batch. +- The recovery is the same as a first-time seed: commit an empty + watermark commit on `main` carrying the trailer for the upstream + HEAD that was actually merged, and push. + -## Why Cherry-Pick (Not Rebase, Not Merge) | Approach | Why rejected / chosen | |---|---| @@ -62,7 +117,7 @@ same names; a take-upstream resolution silently dropping a fork-specific warning suppression; etc. — see PR #220 audit). Before any push or PR, the orchestrator now runs three hard gates: -1. **Toolchain preflight** ([`scripts/09-toolchain-preflight.ps1`](./scripts/09-toolchain-preflight.ps1)) — verifies the host has the `PlatformToolset` versions the repo requires. Missing → **Tier-4d infra-stuck** (lock set, NO GitHub issue — PR review can't fix host provisioning). +1. **Toolchain preflight** ([`scripts/09-toolchain-preflight.ps1`](./scripts/09-toolchain-preflight.ps1)) — verifies the host has the `PlatformToolset` versions the repo requires. Missing → **Tier-4d infra-stuck**: NO GitHub issue is opened and NO lock is set (PR review can't fix host provisioning). The next scheduler tick simply retries; if this host keeps tripping it, run from a properly provisioned host instead. 2. **Static breakage scan** ([`scripts/08-static-scan.ps1`](./scripts/08-static-scan.ps1)) — baseline-diffs `.resw` files for newly-duplicated `` keys, and regex-checks fork invariants from [`references/fork-invariants.json`](./references/fork-invariants.json). Blocking → **Tier-4a stuck**. 3. **Try-build** ([`scripts/10-try-build.ps1`](./scripts/10-try-build.ps1)) — runs `tools\razzle.cmd && bz no_clean` with a 45-minute wall-clock cap. Build failed → **Tier-4b stuck**; timeout → **Tier-4c stuck** (unless `-AllowInconclusiveBuild`). @@ -109,9 +164,10 @@ pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -BuildTimeoutMinutes The cherry-pick loop pins both author and committer identity/dates to the upstream commit so audit timestamps match the original commit-by-commit history. -Resumability is built into the state file — re-running after a successful -run is a fast no-op (nothing pending), and re-running while the stuck-lock -is set exits early without touching the branch. +Resumability is built into the trailer model — re-running after a successful +run is a fast no-op (the new merged commits extend the watermark, so the +pending-list is empty), and re-running while a stuck-issue is open exits +early without touching the branch. ### After-PR review handling — fix-in-PR vs. follow-up PR @@ -182,18 +238,24 @@ deferred fixes. branch is freshly pushed and not yet visible. The same-repo finalize script intentionally uses `--head ` plus a 5s retry — do not "fix" it to `--head :`, which would point `gh` at a fork. -- **Do not run the scheduler twice while stuck.** The lock in - `state.json` makes the second run a no-op, but a human running the - script manually with `-Force` will overwrite the stuck branch and lose - their in-progress resolution. The `-Force` flag is documented but +- **Do not run the scheduler twice while a stuck issue is open.** The + open labeled issue makes the second run a no-op, but a human running + the script manually with `-Force` will overwrite the stuck branch and + lose their in-progress resolution. The `-Force` flag is documented but intentionally not the default. - **Cherry-pick over `git revert` style commits is intentional, not skipped.** We only skip revert-pairs where **both** sides are inside the pending range. A revert of a commit we already merged last week must land as a normal pick — otherwise the fork diverges silently. -- **Always commit `state.json` and `reports/`** so the next scheduler - invocation (possibly on a different machine) starts from the right - checkpoint. The finalize PR includes the state update. +- **Never strip the `(cherry picked from commit )` trailer** when + hand-resolving a stuck pick. That trailer IS the watermark the next + sync run reads. A squash-merge or `--no-commit` workflow that drops + the trailer breaks the next sync as effectively as deleting a + `state.json` used to. +- **Reports and build logs live under `Generated Files/upstream-sync//`.** + This directory is gitignored at the repo root (`**/Generated Files/`). + Do not check these in — they're transient diagnostics, and the issue + body inlines the parts that matter for review. - **Prefer the PR path.** The default workflow always opens a PR so CI and human review checkpoint the upstream batch. Use `-PushDirectToMain` only for an explicit admin/bypass run where skipping PR latency is intentional. @@ -203,12 +265,19 @@ deferred fixes. manifest, re-normalize before staging — see [references/conflict-triage.md](./references/conflict-triage.md#line-endings). +- **Single-host scheduler.** The stuck-lock (open labeled issue) is a + read-then-check gate, not an atomic lease — two hosts running on the + same tick can both observe "no open issue" and proceed in parallel. + Run the scheduler from ONE host. For multi-host fan-out, layer atomic + locking on top (GitHub Actions `concurrency: upstream-sync` group is + the easiest). + ## Troubleshooting | Issue | Solution | |---|---| -| `state.json` is missing | Run the one-time bootstrap — see [references/bootstrap.md](./references/bootstrap.md). Do not guess the baseline SHA. | -| Stuck-lock prevents new run | Resolve the conflict on the stuck branch, open a PR, merge, then run [`scripts/clear-stuck.ps1`](./scripts/clear-stuck.ps1) and re-run the batch. | +| `Get-LastSyncedUpstreamSha` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [State model — First-time sync](#first-time-sync). | +| Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | | Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; the loop auto-resets and marks them skipped. No action needed. | | Same file conflicts every run | Add it to the Tier-0 list in [references/known-conflicts.md](./references/known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | | `gh pr create` returns "Head sha can't be blank" | Retry — the finalize script already does, but on slow networks may need a manual second run. | @@ -217,14 +286,11 @@ deferred fixes. ## References - [references/workflow.md](./references/workflow.md) — full per-step procedure with exit codes and delegation map. -- [references/state-schema.md](./references/state-schema.md) — `state.json` shape and field semantics. -- [references/bootstrap.md](./references/bootstrap.md) — one-time baseline-SHA discovery and initialization. - [references/conflict-triage.md](./references/conflict-triage.md) — Tier 0/1/2/3/4 resolution rubric with examples. - [references/known-conflicts.md](./references/known-conflicts.md) — files that always need a fixed resolution. - [references/static-scan.md](./references/static-scan.md) — post-pick static breakage scan rules. - [references/fork-invariants.json](./references/fork-invariants.json) — fork-specific patterns that must survive any upstream pick. - [references/build-verification.md](./references/build-verification.md) — try-build pipeline + toolchain preflight policy. - [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow for handling post-PR review. -- [references/reporting.md](./references/reporting.md) — report template and stuck-issue template. - [scripts/04-run-batch.ps1](./scripts/04-run-batch.ps1) — the scheduler entrypoint. -- [scripts/clear-stuck.ps1](./scripts/clear-stuck.ps1) — clear the stuck-lock after human resolution. +- [scripts/Common.ps1](./scripts/Common.ps1) — derived-state helpers (`Get-LastSyncedUpstreamSha`, `Get-PendingUpstreamShas`, `Get-StuckIssues`, `Get-GeneratedDir`). diff --git a/.github/skills/upstream-sync/references/bootstrap.md b/.github/skills/upstream-sync/references/bootstrap.md deleted file mode 100644 index 22ac1cf83..000000000 --- a/.github/skills/upstream-sync/references/bootstrap.md +++ /dev/null @@ -1,89 +0,0 @@ -# One-Time Bootstrap - -The skill is incremental — it needs to know which upstream commit the -fork is "caught up to" before it can compute a pending range. This page -covers establishing that baseline exactly once. - -## When to run - -- `state.json` does not exist yet. -- `state.json` exists but `last_synced_upstream_sha` is missing/`null`. -- You manually merged some upstream commits outside this skill and want - to fast-forward the baseline so the next sync doesn't re-pick them. - -**Do NOT** re-run bootstrap on a working skill. It overwrites the -baseline and can cause the next sync to either re-pick already-synced -commits (creating empties — harmless but noisy) or to skip pending -commits (silently dropping upstream changes — bad). - -## How to find the baseline SHA - -The "baseline" is the most recent upstream commit whose tree is -**fully contained** in the fork's history. Pick one of: - -### Method A — known last manual sync (preferred) - -If you remember the last upstream sync (PR or branch), grab the upstream -SHA mentioned in that PR description / commit message: - -```pwsh -git log --all --grep="upstream" --grep="microsoft/terminal" -i --oneline | head -20 -``` - -Look for messages like `Merge upstream main @ ` or -`Sync upstream up to `. That `` is your baseline. - -### Method B — patch-id scan - -For each recent fork commit (last ~200), get its patch-id and search -upstream for a matching patch-id: - -```pwsh -git fetch upstream main -git log --format='%H' -200 | ForEach-Object { - $pid = git show $_ | git patch-id --stable | ForEach-Object { ($_ -split ' ')[0] } - $match = git log upstream/main --format='%H %s' | ForEach-Object { - $usha = ($_ -split ' ',2)[0] - $upid = git show $usha | git patch-id --stable | ForEach-Object { ($_ -split ' ')[0] } - if ($upid -eq $pid) { $_ } - } | Select-Object -First 1 - if ($match) { "$_ matches $match"; break } -} -``` - -Slow but reliable. The first match (newest fork commit with an upstream -twin) gives you the baseline. - -### Method C — ask the human - -If both above fail, ask the user for the baseline SHA. Do **not** guess. -A wrong baseline silently drops upstream commits. - -## Initialize `state.json` - -Once you have ``: - -```pwsh -pwsh .github/skills/upstream-sync/scripts/00-bootstrap.ps1 -BaselineSha -``` - -This script: - -1. Verifies `` exists on `upstream/main`. -2. Writes a fresh `state.json` with the baseline + empty history. -3. Stages and commits `.github/upstream-sync/state.json` on a branch - `chore/upstream-sync-bootstrap`. -4. Tells you to open a PR — do not push state changes straight to main. - -## Verify - -After the bootstrap PR merges, a dry run should report a non-empty -pending list (the commits upstream has made since baseline) without -actually picking anything: - -```pwsh -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -DryRun -``` - -Inspect the latest `reports/*.md` — it should look sane. **Then** enable -the scheduler. diff --git a/.github/skills/upstream-sync/references/build-verification.md b/.github/skills/upstream-sync/references/build-verification.md index e14baa28b..4cced5ed1 100644 --- a/.github/skills/upstream-sync/references/build-verification.md +++ b/.github/skills/upstream-sync/references/build-verification.md @@ -30,7 +30,7 @@ Outcomes: | Outcome | Behavior | |---------------------|--------------------------------------------------------| | All toolsets found | Continue to static scan + build. | -| Required missing | Tier-4 **infra-stuck** — separate kind from code-stuck. Does NOT open a stuck issue (it's a host problem, not a code problem). | +| Required missing | Tier-4 **infra-stuck** — separate kind from code-stuck. Does NOT open a stuck issue, does NOT set a lock (no labeled issue to gate on). The next scheduler tick simply retries; provision the host before then. | | Skipped (`-SkipBuild`) | Preflight not run. Caller accepts risk. | **The preflight does NOT auto-bump v143→v145.** That recipe is @@ -55,11 +55,13 @@ the generated Tier-4 diagnostics include the build log path and tail. Output: -- Full build log → `.github/upstream-sync/build-logs/.log` - (gitignored — these are big and noisy). +- Full build log → `Generated Files/upstream-sync//build-logs/.log` + (gitignored — these are big and noisy; the gitignore is the repo root's + `**/Generated Files/` rule). - Last ~200 lines → captured into the run report and any Tier-4 stuck - issue. -- Exit code + duration → state.json. + issue body. +- Exit code + duration → embedded in the Tier-4 stuck-issue YAML block + (and the run report) when the build is the failing gate. Timeout: @@ -86,8 +88,8 @@ If a flaky build (unrelated env issue, transient toolchain glitch) trips the gate: 1. The stuck issue gives a clear log tail. -2. A human can `clear-stuck.ps1 -ResolvedThroughSha ` after - re-running the build locally to confirm it's a transient. +2. A human can re-run the build locally to confirm it's a transient, + then **close the stuck issue** to clear the lock. 3. The next scheduler tick will re-attempt the same pick range. Distinguishing transient-build from real-pick-broke-build is left to @@ -96,6 +98,7 @@ of a manual cross-check is small (~once per N runs). ## Build artifacts -`.github/upstream-sync/build-logs/` is **not** committed (added to -`.gitignore` by the skill PR). Build artifacts under `bin/`, `obj/`, -etc. follow the repo's existing `.gitignore`. +`Generated Files/upstream-sync//build-logs/` is **not** +committed — the repo root's `**/Generated Files/` gitignore rule +covers it. Build artifacts under `bin/`, `obj/`, etc. follow the +repo's existing `.gitignore`. diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/conflict-triage.md index becd63f9f..bcd8f694e 100644 --- a/.github/skills/upstream-sync/references/conflict-triage.md +++ b/.github/skills/upstream-sync/references/conflict-triage.md @@ -89,9 +89,9 @@ Anything not resolved by Tier 0–2: ```pwsh git cherry-pick --abort -# Set state.stuck_on_sha = , state.stuck_branch = +# Open the labeled stuck issue (07-open-stuck-issue.ps1) — issue body +# carries the ```yaml # wta-state``` block with stuck_on_sha + branch # Write the report with the conflict diagnostics -# Open the GitHub issue (07-open-stuck-issue.ps1) # Exit with code 10 ``` @@ -101,10 +101,9 @@ The report **must** include: - The list of conflicting paths with a one-line classification each (`semantic-overlap`, `deleted-by-us`, `binary-merge`, etc.). - The exact local branch name where the human picks up. -- The exact resume command the human runs after they merge their fix: - ``` - pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha - ``` +- The exact resume action: resolve on the stuck branch, merge a PR + that keeps the `(cherry picked from commit )` trailer, then + CLOSE the stuck issue (that's the lock-clear signal — no script). ## Tier 4 — Post-pick validation failed @@ -133,11 +132,13 @@ historical real-world failures with zero false positives: - 4d distinguishes "this code is broken" from "this host can't even try to build it" — the latter must never open a GitHub issue. -Tier-4 state lives in `state.stuck_validation` (separate from -Tier-3's `state.stuck_on_sha`); either being set causes the -scheduler to skip. Clear with [`clear-stuck.ps1`](../scripts/clear-stuck.ps1) -(omit `-ResolvedThroughSha` to keep the watermark and re-attempt the -same range; pass it to advance past the broken upstream batch). +Tier-4 state lives in the body of an open `upstream-sync-stuck` labeled +issue (separate per kind by `findings_hash`); any such open issue causes +the scheduler to skip. Clear by **closing the issue** after the human +resolves the validation failure — either by merging a fix PR (whose +trailers will advance the watermark) or by fixing the underlying defect +on `main` directly (the next run re-attempts the same range and +re-validates). No script needed. The Tier-4 report includes a `findings_hash` (16-hex prefix). Re-runs that produce the same hash mean the underlying defect is unchanged; diff --git a/.github/skills/upstream-sync/references/reporting.md b/.github/skills/upstream-sync/references/reporting.md deleted file mode 100644 index 7c5a24573..000000000 --- a/.github/skills/upstream-sync/references/reporting.md +++ /dev/null @@ -1,137 +0,0 @@ -# Reporting - -Every run writes one markdown report to -`.github/upstream-sync/reports/YYYY-MM-DDTHHmm.md`. The report doubles -as the PR description (success path) and the issue body (stuck path). - -## Report template - -```markdown -# Upstream sync — — - -**Status:** -**Host:** -**Duration:** -**Baseline (before run):** `` () — -**Upstream HEAD:** `` () — - ---- - -## Summary - -- Commits picked: **** -- Revert pairs dropped: **

** (= <2P> commits skipped, net zero) -- Upstream-empty commits skipped:**** -- Tier-0 auto-resolutions: **** (across files) -- Tier-2 LLM resolutions: **** (only when -TryTier2) -- Tier-3 escalation (stuck at): - -## Pending commits (dry-run only, oldest → newest) - -| # | SHA | Subject | Author | -|---|---|---|---| -| 1 | | | | -| ... | | | | - -## Picked commits (oldest → newest) - -| # | SHA | Subject | Author | -|---|---|---|---| -| 1 | | | | -| ... | | | | - -## Dropped revert pairs - -| Original SHA | Original subject | Revert SHA | -|---|---|---| -| | | | - -## Empty / no-op commits skipped - -| SHA | Subject | -|---|---| -| | | - -## Tier-0 auto-resolutions - -| Commit SHA | File | -|---|---| -| | `.github/workflows/spelling2.yml` | - -## (Stuck only) Conflict diagnostics - -**Conflicting commit:** [``](https://github.com/microsoft/terminal/commit/) — -**Author:** -**Files in conflict:** - -| Path | Classification | Notes | -|---|---|---| -| `` | semantic-overlap | both sides changed `` | -| `` | deleted-by-us | upstream modifies a file we removed | - -**Attempted resolutions:** -- Tier 0: -- Tier 1: -- Tier 2: > - -**Pickup branch:** `upstream-sync/` (pushed to origin) - -**How to resume:** - -1. `git switch upstream-sync/` -2. Resolve the conflict in ``. Reference: - - Upstream commit: https://github.com/microsoft/terminal/commit/ - - Fork file at HEAD: `git show HEAD:` -3. `git add ` and `git cherry-pick --continue` -4. Push, open a PR titled `chore(upstream-sync): manual resolution for `, merge it. -5. Run: - ``` - pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha - ``` -6. The next scheduled sync resumes from +1. - ---- - -_Generated by `.github/skills/upstream-sync/scripts/05-write-report.ps1`._ -``` - -## Stuck-issue template - -The issue body is the report itself, plus a short header explaining -urgency: - -```markdown -🛑 **Upstream sync stopped at a conflict that needs human judgment.** - -The scheduler will keep skipping its runs until this issue is resolved -and the stuck-lock is cleared. No alarm — the lock is intentional. - -To unblock: -1. Follow "How to resume" in the report below. -2. Close this issue after `clear-stuck.ps1` runs cleanly. - - -``` - -Label: `upstream-sync-stuck` (apply via `gh issue create --label`). - -## Why always write a report - -- The "ok" report becomes the PR description automatically — reviewers - see what landed and why anything was skipped. -- The "no-op" report proves the scheduler ran (useful for "did it stop - ticking?" debugging) without polluting issue/PR queues. -- The "stuck" report is the issue body — humans don't need to re-derive - what was attempted. -- The "skipped-locked" report shows the lock did its job (no duplicate - destruction). - -Retention: the **"ok"** and **"stuck"** reports are committed in-repo -(by `06-finalize-pr.ps1` onto the sync branch, and by -`07-open-stuck-issue.ps1` / `07b-open-validation-stuck-issue.ps1` onto -`main` next to the lock). The **"no-op"** and **"skipped-locked"** -reports are written **locally only** and intentionally NOT committed — -they exist for "did the scheduler tick?" debugging on the host that ran -the script, and committing them would churn `main` on every cadence -even when nothing happened. They're small markdown either way; keep -committed reports indefinitely. diff --git a/.github/skills/upstream-sync/references/state-schema.md b/.github/skills/upstream-sync/references/state-schema.md deleted file mode 100644 index 67293f1e4..000000000 --- a/.github/skills/upstream-sync/references/state-schema.md +++ /dev/null @@ -1,106 +0,0 @@ -# `state.json` Schema - -Path: `.github/upstream-sync/state.json` (committed on `main`). - -```jsonc -{ - "version": 1, - - // Provenance for the baseline below. v1 scripts intentionally sync - // microsoft/terminal main; these fields are not runtime configuration. - "upstream_remote_url": "https://github.com/microsoft/terminal.git", - "upstream_branch": "main", - - // The most recent upstream commit that has landed in this fork's main. - // Updated only when a sync PR merges (the PR includes the state update). - "last_synced_upstream_sha": "93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f", - - // Stuck-lock (Tier-3 — cherry-pick conflict). When non-null, the - // scheduler exits early without touching any branch. Cleared by - // scripts/clear-stuck.ps1 after a human merges the resolution PR. - "stuck_on_sha": null, - "stuck_branch": null, - "stuck_at": null, // ISO 8601 timestamp; null when not stuck - "stuck_issue_url": null, // populated by 07-open-stuck-issue.ps1 - - // Stuck-lock (Tier-4 — post-pick validation failed). Set when picks - // applied cleanly but static-scan / try-build / toolchain-preflight - // blocked the push. Independent of stuck_on_sha but treated the same - // way by the scheduler gate: either lock present → skip. - // Shape (when non-null): - // { - // "kind": "static-scan" | "build-failed" | "build-inconclusive" | "toolchain-missing", - // "base": "", - // "head": "", - // "branch": "upstream-sync/YYYY-MM-DD", - // "range": ["", "", ...], // picks that landed before validation said no - // "findings_hash": "<16-hex sha256 prefix>", // dedup signal across re-runs - // "at": "ISO 8601", - // "issue_url": "https://..." | null // null for toolchain-missing (infra) - // } - "stuck_validation": null, - - // Last run summary (for fast inspection without grepping reports). - "last_run": { - "at": "2026-06-04T13:41:45+08:00", - "host": "SH-YEELAM-D11S", - "status": "ok", // "ok" | "stuck" | "stuck-static-scan" | "stuck-build-failed" | "stuck-build-inconclusive" | "stuck-toolchain-missing" - "branch": "upstream-sync/2026-06-04", - "pr_url": "https://github.com/microsoft/intelligent-terminal/pull/999", - "picked_count": 7, - "dropped_pair_count": 1, - "empty_count": 2, - "tier0_resolutions": 1, - - // OPTIONAL — only set by the -PushDirectToMain code path - // (06b-finalize-direct.ps1). Omitted on normal PR runs. - "merge_mode": "direct-push", // "pr" (default, implicit) | "direct-push" - "main_head_sha": "1234567890abcdef..." // SHA of origin/main after the direct push - }, - - // Rolling history — keep last 20 runs. Only `ok` and `stuck*` runs - // write state (and therefore appear here); `no-op`, `dry-run`, and - // `skipped-*` runs produce a local-only report and leave state.json - // unchanged. - "history": [ - { "at": "...", "status": "ok", "picked_count": 7, "pr_url": "..." }, - { "at": "...", "status": "stuck", "stuck_on_sha": "abc...", "issue_url": "..." } - ] -} -``` - -## Field rules - -- **`last_synced_upstream_sha`** advances **only** when a sync PR is merged. - The orchestrator updates this in the PR commit itself, so it lands - atomically with the picks. Never edit by hand except via - `clear-stuck.ps1`. -- **`stuck_on_sha`** is the Tier-3 gate. When set, `04-run-batch.ps1` exits 0 - without doing anything. This is intentional — the scheduler will keep - ticking but will not clobber the stuck branch. -- **`stuck_validation`** is the Tier-4 gate, independent of `stuck_on_sha`. - Either one being non-null causes the scheduler to skip. Cleared by - `clear-stuck.ps1` — `-ResolvedThroughSha` is optional for Tier-4 (omit - to keep the watermark and have the next run re-attempt the same range - after the human fixed whatever validation caught). -- **`findings_hash`** in `stuck_validation` is a stable 16-hex prefix of - the SHA-256 of the normalized findings list. If a re-run produces the - same hash, the human knows the underlying fault is unchanged; if it - changes, validation has moved to a new failure mode. -- **`stuck_branch`** must still exist on `origin` until the human merges - it; `clear-stuck.ps1` does not delete it (the PR merge does). -- **`history`** is for the human reading state.json directly. The reports - in `reports/` are the source of truth. - -## Concurrency - -The scheduler should run on a single host. If multiple hosts run -concurrently, the second one's `git push -u origin upstream-sync/` -will collide on the same-day branch name — `git push` will reject with -non-fast-forward and `04-run-batch.ps1` will exit 20 (hard failure). -This is acceptable: the loser's report is still written locally for -inspection, and no state on `main` has been updated. - -If you genuinely need multi-host scheduling, add a per-host suffix to -the branch name and a state-file mutex via `gh api repos/.../contents/...` -GraphQL check-and-set — out of scope for v1. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index 9b1affd0e..330ed68c3 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -4,12 +4,24 @@ This is the authoritative per-step procedure. The orchestrator is [`scripts/04-run-batch.ps1`](../scripts/04-run-batch.ps1); each step below maps to a script or an in-orchestrator function. +## State model — derived, not stored + +There is **no `state.json`**. Every persistent fact is derived from an +authoritative source on demand: + +| Fact | Source | Helper in [`scripts/Common.ps1`](../scripts/Common.ps1) | +|---|---|---| +| Last-synced upstream SHA | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main` | `Get-LastSyncedUpstreamSha` | +| Pending list | `git log --cherry-pick --right-only --no-merges ...upstream/main` (patch-id-aware; reverted picks reappear) | `Get-PendingUpstreamShas -Since ` | +| Stuck-lock | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal` | `Get-StuckIssues` | +| Stuck-lock metadata | Fenced ```yaml # wta-state``` block inside the issue body | `Get-StuckMetaFromIssue` | +| Transient artifacts (reports, build logs) | `Generated Files/upstream-sync//` (gitignored at repo root) | `Get-GeneratedDir [-Sub]` | + ## Entry Conditions -- `state.json` exists (bootstrap done — see [bootstrap.md](./bootstrap.md)). - Working tree is clean (`git status --porcelain` empty). -- We are on `main` (or the script will `git switch main`). -- `state.stuck_on_sha` is `null` AND `state.stuck_validation` is `null` (otherwise exit early — see "Stuck-lock" below). +- We are on `main` (or the script will `git switch main` and `git pull --ff-only origin main`). +- `Get-StuckIssues` returns empty (otherwise exit early — see "Stuck-lock" below). ## Steps @@ -22,17 +34,20 @@ git fetch upstream main --no-tags Script: [`01-fetch-upstream.ps1`](../scripts/01-fetch-upstream.ps1). -Writes a local "no-op" report and exits 0 (without updating `state.json`) if -`git rev-parse upstream/main` equals `state.last_synced_upstream_sha`. +If `git rev-parse upstream/main` equals `Get-LastSyncedUpstreamSha`, the +orchestrator writes a local no-op report and exits 0. ### 2. Compute pending range ```pwsh -git log --reverse --format='%H' "$last_synced..upstream/main" +$since = Get-LastSyncedUpstreamSha +git log --cherry-pick --right-only --no-merges --format='%H' --reverse "$since...upstream/main" ``` Oldest-first ordering is mandatory. Cherry-picking newest-first inverts -dependencies and creates spurious conflicts. +dependencies and creates spurious conflicts. `--cherry-pick` compares +patch IDs, so a commit that was picked then reverted on `origin/main` +correctly re-appears here as pending. Script: [`02-compute-pending.ps1`](../scripts/02-compute-pending.ps1) emits a JSON object on stdout — see step 3 below for the full shape. @@ -74,7 +89,8 @@ git cherry-pick --keep-redundant-commits -x ``` - `-x` adds `(cherry picked from commit )` to the message — critical - for audit trail and for the next-run revert-pair detector. + for audit trail, for the next-run revert-pair detector, **and for + `Get-LastSyncedUpstreamSha` to derive the next watermark.** Never strip it. - `--keep-redundant-commits` lets us preserve no-op picks for traceability (we then `git reset --hard HEAD~1` if Tier-1 fires). @@ -91,9 +107,9 @@ git cherry-pick --keep-redundant-commits -x 3. **Tier 2 — trivial textual (opt-in via `-TryTier2`).** Delegate to a fresh sub-agent with the conflict text. Accept only `high` confidence. See [conflict-triage.md](./conflict-triage.md#tier-2-llm-assisted). -4. **Tier 3 — semantic conflict.** Run `git cherry-pick --abort`. Set - the stuck-lock, write report, exit non-zero. The script that calls - us will then open the stuck issue. +4. **Tier 3 — semantic conflict.** Run `git cherry-pick --abort`. Open + the labeled stuck issue, write report, exit 10. The next scheduler + tick sees the open labeled issue and skips. Script: [`03-cherry-pick-one.ps1`](../scripts/03-cherry-pick-one.ps1) handles one commit, returns a JSON status object. The orchestrator loops. @@ -113,8 +129,9 @@ pwsh .github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 Detects required `` values from `src/common.build.*.props` and checks they exist under `\MSBuild\Microsoft\VC\\Platforms\x64\PlatformToolsets\`. -If `ok=false`, this is **Tier-4d infra-stuck**: lock + NO GitHub issue -(PR review cannot fix host provisioning). Skipped when `-SkipBuild` is set. +If `ok=false`, this is **Tier-4d infra-stuck**: NO GitHub issue, NO lock +(PR review cannot fix host provisioning; the next scheduler tick simply +retries from a properly provisioned host). Skipped when `-SkipBuild` is set. #### 7b. Static breakage scan @@ -132,7 +149,7 @@ loop. The scan: against the post-pick worktree. If `blocking=true` (any `critical` or `high` finding), this is **Tier-4a -stuck**: lock + GitHub issue + exit 10. Skipped when `-SkipStaticScan`. +stuck**: opens labeled issue + exit 10. Skipped when `-SkipStaticScan`. #### 7c. Try-build @@ -146,14 +163,15 @@ pwsh .github/skills/upstream-sync/scripts/10-try-build.ps1 -BuildCommand 'tools\ - `kind = build-inconclusive` (timeout) → **Tier-4c stuck**, unless `-AllowInconclusiveBuild` (dev opt-in; never in a scheduler). -Skipped when `-SkipBuild`. Logs land in `.github/upstream-sync/build-logs/` -(git-ignored). +Skipped when `-SkipBuild`. Logs land in +`Generated Files/upstream-sync//build-logs/` (gitignored). ### 8. Write report (always) Regardless of outcome (ok / no-op / dry-run / stuck / stuck-static-scan / stuck-build-failed / stuck-build-inconclusive / stuck-toolchain-missing), -write `.github/upstream-sync/reports/YYYY-MM-DDTHHmm[-suffix].md` with: +write `Generated Files/upstream-sync//-.md` +with: - Run metadata (start, end, duration, host, status) - Counts: picked / dropped-pair / empty / known-conflict-resolved / stuck-at @@ -162,7 +180,9 @@ write `.github/upstream-sync/reports/YYYY-MM-DDTHHmm[-suffix].md` with: - If stuck (Tier-3): the conflicting commit, the conflicting paths, what was attempted, the exact resume command - If stuck (Tier-4): the validation findings, the build log tail, the exact resume command -Template: [`reporting.md`](./reporting.md). +Reports are **transient** — never committed. The stuck issue body +(step 9b/9c) inlines the parts of the report a reviewer needs without +fetching the local file. Script: [`05-write-report.ps1`](../scripts/05-write-report.ps1). @@ -173,13 +193,13 @@ git push -u origin $branch gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title "chore(upstream): sync up to $shortSha" --body-file $reportPath ``` -Update `state.last_synced_upstream_sha = upstream/main` and commit -`state.json` + the report into the sync branch (amend the last pick or -add a trailing commit titled `chore(upstream-sync): update state`). +No state-file commit. The `(cherry picked from commit )` trailer on +each cherry-pick IS the watermark — once the PR merges, the trailer is +on `origin/main` and the next run's `Get-LastSyncedUpstreamSha` finds it. Script: [`06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1). -### 9b. Stuck path (Tier-3) — open issue + set lock +### 9b. Stuck path (Tier-3) — open labeled issue ```pwsh gh issue create -R microsoft/intelligent-terminal --label upstream-sync-stuck ` @@ -187,19 +207,19 @@ gh issue create -R microsoft/intelligent-terminal --label upstream-sync-stuck ` --body-file $reportPath ``` -Set `state.stuck_on_sha = ` and `state.stuck_branch = $branch`. -Commit `state.json` and the report on `main` (yes, directly — this is the -lock, and the PR path is blocked). The next scheduled run sees the lock -and exits. +The issue body carries a fenced ```yaml # wta-state``` block with +`tier`, `kind=cherry-pick-conflict`, `stuck_on_sha`, `branch`, `at`, +`host` so a future run's `Get-StuckMetaFromIssue` can read the lock +context. Nothing is committed to `main`. Script: [`07-open-stuck-issue.ps1`](../scripts/07-open-stuck-issue.ps1). -### 9c. Stuck path (Tier-4) — open issue + set lock +### 9c. Stuck path (Tier-4) — open labeled issue -For Tier-4a/b/c, the same flow as 9b but the issue title carries the -validation kind and findings hash; `state.stuck_validation` is set -instead of `state.stuck_on_sha`. For Tier-4d (toolchain-missing), only -the lock is set — NO issue is opened. +For Tier-4a/b/c, same flow as 9b — open the labeled issue with a +`# wta-state` block carrying `tier=4`, `kind`, `findings_hash`, +`picked_count`. For Tier-4d (toolchain-missing), NO issue is opened +(infra problem); the next scheduler tick simply retries. Script: [`07b-open-validation-stuck-issue.ps1`](../scripts/07b-open-validation-stuck-issue.ps1). @@ -248,28 +268,36 @@ fixes. ## Stuck-Lock -When **either** `state.stuck_on_sha` (Tier-3) **or** `state.stuck_validation` -(Tier-4) is non-null, the orchestrator: +When `Get-StuckIssues` returns any OPEN `upstream-sync-stuck` labeled +issue, the orchestrator: -1. Logs `"stuck-lock set: ; skipping run"`. -2. Writes a `reports/YYYY-MM-DDTHHmm-skipped.md` noting the skip. -3. Exits 0 (the scheduler should not retry on the same lock). +1. Logs `"stuck-lock set ( at ); skipping run"`. +2. Writes a transient `-skipped.md` under + `Generated Files/upstream-sync//` noting the skip. +3. Exits 0 (the scheduler should not alarm). To clear the lock after the human has resolved the underlying issue: -```pwsh -# Tier-3: -ResolvedThroughSha is REQUIRED and advances the watermark. -pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha - -# Tier-4: -ResolvedThroughSha is OPTIONAL. Omit it to keep the watermark -# and have the next run re-attempt the same range (recommended when the -# fix lands as a separate PR on main — the next sync will pick up the -# upstream batch atop the now-fixed main and re-validate). -pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 +``` +1. Resolve the conflict on the stuck branch (`upstream-sync/`), + keeping every `(cherry picked from commit )` trailer intact. +2. Open a PR for the fix, merge it (rebase or merge — NOT squash). +3. CLOSE the stuck issue. That's the lock-clear signal — no script. ``` -This sets `state.last_synced_upstream_sha` (when advanced), clears the -appropriate lock fields, and commits `state.json` on `main`. +The next scheduler tick: +- `Get-LastSyncedUpstreamSha` re-derives the watermark from the merged + PR's trailers (advancing past the resolved batch — for Tier-3 by the + exact resolved commit; for Tier-4 by whatever extra trailers the fix + PR carried). +- `Get-StuckIssues` returns empty (the issue is closed). +- The run proceeds from the new watermark. + +For Tier-4 where the operator wants to **re-attempt the same range** +(e.g. because the fix landed as a separate PR on `main` that doesn't +itself carry trailers), simply close the issue without merging a sync +fix: the next run will recompute pending against the same watermark and +re-validate. ## Sub-Agent Delegation Map diff --git a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 b/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 deleted file mode 100644 index 1f659a7e2..000000000 --- a/.github/skills/upstream-sync/scripts/00-bootstrap.ps1 +++ /dev/null @@ -1,89 +0,0 @@ -<# -.SYNOPSIS - One-time bootstrap: initialize state.json with the baseline upstream SHA. - -.DESCRIPTION - Use this exactly once when the skill is first installed in the repo. - See references/bootstrap.md for how to discover the right baseline SHA. - -.PARAMETER BaselineSha - The upstream/microsoft/terminal commit SHA the fork is currently - "caught up to". Must be reachable from upstream/main. - -.PARAMETER Force - Overwrite an existing state.json. Refuses by default to prevent - accidentally rewinding the baseline. - -.EXAMPLE - pwsh .github/skills/upstream-sync/scripts/00-bootstrap.ps1 -BaselineSha 93bdbfaa3d62304f4b50b4ca4484da4dd08e4a1f -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] [string] $BaselineSha, - [switch] $Force -) - -. "$PSScriptRoot/Common.ps1" - -# Safety: a bootstrap PR must contain only state.json. Refuse if the -# worktree is dirty or HEAD is not main; otherwise unrelated diffs (or a -# feature branch's tip) could ride along on the bootstrap commit/PR. -$currentBranch = (git rev-parse --abbrev-ref HEAD).Trim() -if ($LASTEXITCODE -ne 0) { throw "git rev-parse failed (is this a git repo?)." } -if ($currentBranch -ne 'main') { - throw "Bootstrap must be run from 'main'. Currently on '$currentBranch'; git switch main first." -} -$dirty = git status --porcelain -if ($LASTEXITCODE -ne 0) { throw "git status failed." } -if ($dirty) { throw "Worktree is dirty. Bootstrap refuses to commit on a dirty tree:`n$dirty" } -git pull --ff-only origin main | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed. Resolve and retry." } - -Ensure-UpstreamRemote -git fetch upstream main --no-tags | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } - -# Verify the SHA exists on upstream/main and persist the canonical 40-hex form. -$BaselineSha = Resolve-FullCommitSha $BaselineSha -$null = git merge-base --is-ancestor $BaselineSha upstream/main -if ($LASTEXITCODE -ne 0) { - throw "Baseline SHA $BaselineSha is not an ancestor of upstream/main. Refusing to write state.json." -} - -$statePath = Get-StatePath -if ((Test-Path $statePath) -and -not $Force) { - throw "state.json already exists at $statePath. Pass -Force to overwrite (rewinding the baseline can cause re-picks)." -} - -$state = [ordered] @{ - version = 1 - upstream_remote_url = 'https://github.com/microsoft/terminal.git' - upstream_branch = 'main' - last_synced_upstream_sha = $BaselineSha - stuck_on_sha = $null - stuck_branch = $null - stuck_at = $null - stuck_issue_url = $null - stuck_validation = $null - last_run = $null - history = @() -} -Write-State $state - -# Stage and commit on a dedicated branch so the human can open the PR. -$branch = 'chore/upstream-sync-bootstrap' -git switch -c $branch 2>$null -if ($LASTEXITCODE -ne 0) { - git switch $branch | Out-Null - if ($LASTEXITCODE -ne 0) { throw "Could not create or switch to bootstrap branch '$branch'. Refusing to commit state.json on the current HEAD." } -} - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) -if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed." } - -git commit -m "chore(upstream-sync): bootstrap baseline at $($BaselineSha.Substring(0,9))" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit failed; bootstrap aborted." } - -Write-Host "" -Write-Host "Bootstrap committed on branch '$branch'." -ForegroundColor Green -Write-Host "Next: git push -u origin $branch && gh pr create" diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 83de0afb5..7e34b12d6 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -3,10 +3,11 @@ Compute the pending cherry-pick list with revert-pair detection. .DESCRIPTION - Reads state.last_synced_upstream_sha, lists commits in - state.last_synced..upstream/main (oldest first), detects revert pairs - within the range and drops them, detects upstream-empty commits and - drops them, and emits the final pending list as JSON. + Reads the last-synced upstream watermark from origin/main's + `cherry picked from commit ` trailers, lists commits in + watermark..upstream/main (oldest first), detects revert pairs within the + range and drops them, detects upstream-empty commits and drops them, and + emits the final pending list as JSON. .OUTPUTS JSON object on stdout: @@ -23,10 +24,11 @@ param() . "$PSScriptRoot/Common.ps1" -$state = Read-State -$from = [string]$state.last_synced_upstream_sha -if (-not $from) { throw "state.last_synced_upstream_sha is empty. Run bootstrap." } - +# `git fetch upstream main` must have been run already (orchestrator calls +# 01-fetch-upstream.ps1 before us). Get-LastSyncedUpstreamSha walks the +# `cherry picked from commit ` trailers on origin/main back to the most +# recent one that resolves to a commit on upstream/main — no state.json. +$from = Get-LastSyncedUpstreamSha $to = (git rev-parse upstream/main).Trim() if ($LASTEXITCODE -ne 0) { throw "git rev-parse upstream/main failed." } @@ -35,10 +37,16 @@ if ($from -eq $to) { return } -# Oldest-first list of full SHAs. -$all = git log --reverse --format='%H' "$from..$to" -if ($LASTEXITCODE -ne 0) { throw "git log failed." } -$all = @($all | Where-Object { $_ }) +# Patch-id-aware list of full SHAs (oldest-first). Uses Get-PendingUpstreamShas +# from Common.ps1, which wraps `git log --cherry-pick --right-only --no-merges`: +# any upstream commit whose patch ID matches a commit already on origin/main is +# excluded (so picked-then-reverted commits stay out unless their patch is no +# longer on origin/main, in which case they correctly re-appear as pending). +# The revert-pair detection below stays as defense-in-depth and as the source +# of the `dropped_pairs` report field; in practice --cherry-pick already drops +# most pairs, but a same-batch original+revert that wasn't yet on origin/main +# at the time of computation is still useful to surface. +$all = @(Get-PendingUpstreamShas -Since $from) # Build sha -> first line and body map (single git invocation per commit is fine for typical batch sizes). $info = @{} diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index d3ddecb15..b45f14d1a 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -4,24 +4,29 @@ scheduler on a weekly/daily cadence. .DESCRIPTION - Reads state.json. If the stuck-lock (Tier-3 stuck_on_sha OR Tier-4 - stuck_validation) is set, writes a skipped-locked report and exits 0. - Otherwise: + No state.json. Everything is derived from authoritative sources: + * last-synced watermark -> Get-LastSyncedUpstreamSha (origin/main trailers) + * pending list -> Get-PendingUpstreamShas (git log --cherry-pick) + * stuck-lock -> Get-StuckIssues (open upstream-sync-stuck labeled issues) + + If any open `upstream-sync-stuck` labeled issue exists, this run skips + with a `skipped-locked` report and exits 0. Otherwise: 1. Fetches upstream/main. 2. Computes pending commits, dropping revert pairs and empties. 3. Creates branch upstream-sync/YYYY-MM-DD. 4. Cherry-picks one-by-one with Tier-0/Tier-1 auto-resolution. - On cherry-pick conflict → Tier-3 stuck path (07). + On cherry-pick conflict -> Tier-3 stuck path (07). 5. Post-batch HARD GATES (in order, before any push/PR): - a. Toolchain preflight (09) — missing toolset = infra stuck. - b. Static breakage scan (08) — duplicate resw / fork invariants. - c. Try-build (10) — razzle + bz no_clean. - Any failure → Tier-4 stuck path (07b). - 6. Writes a report. - 7. On success → pushes branch, opens PR (exit 0). - On Tier-3 → pushes branch, opens issue, sets lock (exit 10). - On Tier-4 → pushes branch, opens issue (except infra), sets lock (exit 10). - On no-op → exits 0 with a "no-op" report. + a. Toolchain preflight (09) - missing toolset = infra stuck. + b. Static breakage scan (08) - duplicate resw / fork invariants. + c. Try-build (10) - razzle + bz no_clean. + Any failure -> Tier-4 stuck path (07b). + 6. Writes a transient report under `Generated Files/upstream-sync//` + (gitignored; never committed). + 7. On success -> pushes branch, opens PR (exit 0). + On Tier-3 -> pushes branch, opens labeled issue (exit 10). + On Tier-4 -> pushes branch, opens labeled issue (except infra), (exit 10). + On no-op -> exits 0 with a "no-op" report. .PARAMETER DryRun Compute & report only; do not create the branch or pick anything. @@ -30,7 +35,7 @@ Reserved: enable LLM-assisted Tier-2 conflict resolution (NOT YET IMPLEMENTED). .PARAMETER Force - Override the stuck-lock (Tier-3 OR Tier-4). DANGEROUS — clobbers the + Override the stuck-lock (Tier-3 OR Tier-4). DANGEROUS - clobbers the in-progress branch. Use only when you know the lock is stale. .PARAMETER MaxPicks @@ -51,7 +56,7 @@ Skip steps 5a + 5c. Default: build. Schedulers MUST build. .PARAMETER AllowInconclusiveBuild - Don't treat a build timeout as Tier-4 stuck — proceed with a warning + Don't treat a build timeout as Tier-4 stuck - proceed with a warning in the report. Dev opt-in only; schedulers should leave it off so hung builds don't escape into unproven PRs. @@ -65,8 +70,8 @@ .OUTPUTS Writes status to stdout. Exit codes: 0 = success (PR opened) OR no-op OR skipped-locked - 10 = stuck (Tier-3 or Tier-4) — NOT an error - 20 = hard failure (git/gh broken) — alarm-worthy + 10 = stuck (Tier-3 or Tier-4) - NOT an error + 20 = hard failure (git/gh broken) - alarm-worthy #> [CmdletBinding()] param( @@ -108,85 +113,71 @@ function Invoke-Tier4Stuck { try { $ctx = New-RunContext - # Fast-forward local main from origin BEFORE reading state.json. The - # single-active-lock and last_synced_upstream_sha invariants live on - # origin/main; a stale local clone would let this scheduler proceed - # past a stuck-lock set by a concurrent run on another host or by - # the operator's `clear-stuck.ps1` reverse (defeating the whole - # safety model). Worktree cleanliness is checked first so that an - # unrelated dirty file can't block the FF unexpectedly mid-script. + # Fast-forward local main from origin BEFORE any state-derivation calls + # so Get-LastSyncedUpstreamSha / Get-PendingUpstreamShas see the + # authoritative refs. A stale local clone would otherwise compute a + # wrong pending list (or repeat picks already on origin/main from a + # concurrent run on another host). Worktree cleanliness is checked + # first so an unrelated dirty file can't block the FF mid-script. Assert-CleanWorktree git switch main 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } git pull --ff-only origin main 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only origin main failed." } - $state = Read-State - - # --- Stuck-lock gate (Tier-3 OR Tier-4) --- - $stuckTier3 = [bool] $state.stuck_on_sha - $stuckTier4 = [bool] $state.stuck_validation - if (($stuckTier3 -or $stuckTier4) -and -not $Force) { - $lockDesc = if ($stuckTier3) { - "Tier-3 at $($state.stuck_on_sha) (issue: $($state.stuck_issue_url))" - } else { - $v = $state.stuck_validation - "Tier-4 $($v.kind) [hash $($v.findings_hash)] (issue: $($v.issue_url))" + # --- Stuck-lock gate --- + # Derived from open `upstream-sync-stuck` labeled issues. Any open + # issue with that label blocks the scheduler until a human closes it + # (the close acts as the "lock cleared" signal - no clear-stuck.ps1 + # needed). The gate ALSO needs `upstream` fetched so that the report's + # range / watermark fields can be computed even when we skip. + Ensure-UpstreamRemote + git fetch upstream main --no-tags 2>&1 | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git fetch upstream main failed." } + + if (-not $Force) { + $stuck = Get-StuckIssues + if ($stuck.Count -gt 0) { + $first = $stuck[0] + $meta = Get-StuckMetaFromIssue -Issue $first + $lockDesc = if ($meta -and ($meta.PSObject.Properties.Name -contains 'tier')) { + "$($meta.tier) at $($first.url)" + } else { + "labeled issue $($first.url)" + } + Write-Host "Stuck-lock set ($lockDesc). Skipping. Close the issue to clear the lock." -ForegroundColor Yellow + $fromSha = try { Get-LastSyncedUpstreamSha } catch { '(unknown)' } + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $fromSha -Status 'skipped-locked' + Write-Host "Skip report: $reportPath" + exit 0 } - Write-Host "Stuck-lock set: $lockDesc. Skipping." -ForegroundColor Yellow - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $state.last_synced_upstream_sha -To $state.last_synced_upstream_sha -Status 'skipped-locked' - Write-Host "Skip report: $reportPath" - exit 0 } # --- Existing-PR gate --- - # Schedulers run unattended. If a prior run already opened an - # upstream-sync PR that hasn't merged yet, our baseline (origin/main) - # is unchanged, so we'd recompute the SAME pending range and then - # either (a) collide on branch creation or (b) 06-finalize-pr.ps1 - # would fail because the PR already exists for that branch. Worse, - # under a per-day branch-name scheme the second run would open a - # NEW PR with identical content. Bail early with a no-op report - # instead, unless -Force is given. Skipped entirely under - # -PushDirectToMain, which never opens a PR and shouldn't require - # `gh` auth on the host. - # Skip the existing-PR gate when there's nothing to publish anyway: - # -PushDirectToMain never opens a PR, and -DryRun stops well before - # 06-finalize-pr.ps1 — neither should require `gh` auth on the host. + # Don't open a second concurrent upstream-sync PR. Same stderr-temp-file + # pattern as everywhere else: a gh banner on stderr must not be merged + # into stdout (would break ConvertFrom-Json on the JSON payload). if (-not $Force -and -not $PushDirectToMain -and -not $DryRun) { - # Drop the GitHub `head:` search qualifier — it matches exact - # branch names, not prefixes, so `head:upstream-sync/` would - # return nothing even when an `upstream-sync/2026-06-04` PR is - # open. List all open PRs (--limit 200 covers the corner case - # where a repo has more than the default 30 open) and filter - # client-side by headRefName. - # - # Capture stdout (JSON) and stderr separately: merging them with - # `2>&1` breaks ConvertFrom-Json when `gh` emits any warning or - # progress text on stderr even at exit 0 (e.g. version-update - # notice, deprecation warning). stderr is only used for the - # failure message. $errFile = [System.IO.Path]::GetTempFileName() + $existingJson = $null try { $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>$errFile $ghExit = $LASTEXITCODE if ($ghExit -ne 0) { - # `gh` missing / not authenticated / network-blocked. Don't - # silently continue and waste a full pick + scan + build - # only to fail later in 06-finalize-pr.ps1 — fail fast. - $errText = if (Test-Path $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } Exit-Hard "gh pr list failed (exit $ghExit): $errText. The existing-PR gate requires gh to be installed and authenticated. Re-run with -Force to bypass (at your own risk), or with -DryRun / -PushDirectToMain to skip the gate." } } finally { - Remove-Item -LiteralPath $errFile -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue } if ($existingJson) { $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } if ($existing.Count -gt 0) { $first = $existing[0] Write-Host "An upstream-sync PR is already open: #$($first.number) ($($first.headRefName)) -> $($first.url). Skipping until it merges or is closed (use -Force to override)." -ForegroundColor Yellow - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $state.last_synced_upstream_sha -To $state.last_synced_upstream_sha -Status 'skipped-pr-open' + $fromSha = try { Get-LastSyncedUpstreamSha } catch { '(unknown)' } + $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $fromSha -Status 'skipped-pr-open' Write-Host "Skip report: $reportPath" exit 0 } @@ -194,13 +185,11 @@ try { } Assert-CleanWorktree - # main is already FF'd from origin above (before the stuck-lock + existing-PR - # gates); no need to re-pull here. Re-assert clean state in case the gates - # produced ephemeral artifacts on disk. - # --- 1. Fetch upstream --- - $toSha = (& "$PSScriptRoot/01-fetch-upstream.ps1").Trim() - $fromSha = $state.last_synced_upstream_sha + # --- 1. Resolve from/to (upstream already fetched above) --- + $toSha = (git rev-parse upstream/main).Trim() + if ($LASTEXITCODE -ne 0) { Exit-Hard "git rev-parse upstream/main failed." } + $fromSha = Get-LastSyncedUpstreamSha if ($toSha -eq $fromSha) { Write-Host "Already at upstream HEAD ($toSha). No-op." -ForegroundColor Green @@ -232,9 +221,7 @@ try { exit 0 } - # Capture pre-pick base SHA (origin/main) — used as static-scan baseline. - # Trim defensively: native git output occasionally carries a trailing \r - # depending on shim, and an untrimmed SHA breaks `"$Base..$Head"` ranges. + # Capture pre-pick base SHA (origin/main) - used as static-scan baseline. $preBase = (git rev-parse origin/main).Trim() if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not resolve origin/main for scan baseline." } @@ -274,7 +261,7 @@ try { if ($ctx.StuckPaths.Count -gt 0) { Write-Warning "Stuck at $sha on paths: $($ctx.StuckPaths -join ', ')$errSuffix" } else { - Write-Warning "Stuck at $sha — no conflict paths reported$errSuffix" + Write-Warning "Stuck at $sha - no conflict paths reported$errSuffix" } break } @@ -301,7 +288,7 @@ try { $ctx.Preflight = $preflightJson | ConvertFrom-Json Write-Host "Required: $($ctx.Preflight.required_toolsets -join ', '); available: $($ctx.Preflight.available_toolsets -join ', ')" if (-not $ctx.Preflight.ok) { - Write-Warning "Toolchain preflight FAILED — missing: $($ctx.Preflight.missing -join ', ')" + Write-Warning "Toolchain preflight FAILED - missing: $($ctx.Preflight.missing -join ', ')" Invoke-Tier4Stuck -Ctx $ctx -Kind 'toolchain-missing' -FromSha $fromSha -ToSha $toSha } } @@ -330,7 +317,7 @@ try { 'build-failed' { Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-failed' -FromSha $fromSha -ToSha $toSha } 'build-inconclusive' { if ($AllowInconclusiveBuild) { - Write-Warning "Build inconclusive — proceeding (--AllowInconclusiveBuild)." + Write-Warning "Build inconclusive - proceeding (--AllowInconclusiveBuild)." } else { Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-inconclusive' -FromSha $fromSha -ToSha $toSha } @@ -346,15 +333,23 @@ try { Write-Host "Report: $reportPath" if ($PushDirectToMain) { - $mainHead = & "$PSScriptRoot/06b-finalize-direct.ps1" -Ctx $ctx -To $toSha -ReportPath $reportPath + # No more state.json -> no backfill commit needed. Just push the + # sync branch's commits directly onto main as a fast-forward. + git switch main | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed before direct-push." } + git merge --ff-only $branch | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git merge --ff-only $branch failed (main moved during the run?)." } + git push origin main | Out-Host + if ($LASTEXITCODE -ne 0) { Exit-Hard "git push origin main failed; sync content is local only." } + $mainHead = (git rev-parse HEAD).Trim() Write-Host "" - Write-Host "✅ Sync fast-forwarded onto main at $($mainHead.Substring(0,9))" -ForegroundColor Green + Write-Host ("[OK] Sync fast-forwarded onto main at " + $mainHead.Substring(0,9)) -ForegroundColor Green exit 0 } $prUrl = & "$PSScriptRoot/06-finalize-pr.ps1" -Ctx $ctx -To $toSha -ReportPath $reportPath -AutoMergeStrategy $AutoMergeStrategy Write-Host "" - Write-Host "✅ Sync PR opened: $prUrl" -ForegroundColor Green + Write-Host "[OK] Sync PR opened: $prUrl" -ForegroundColor Green exit 0 } catch { diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 1bd7e53fa..99b01e855 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -1,12 +1,21 @@ <# .SYNOPSIS - Generate a sync run report markdown file. + Generate a sync run report markdown file under + `Generated Files/upstream-sync//` (gitignored). + +.DESCRIPTION + Reports are TRANSIENT artifacts: they live in the gitignored + `Generated Files/` workspace and are NEVER committed. They exist + so a human running the skill (or reviewing a stuck issue) can read + what happened in a single run. The stuck-issue body (07 / 07b) + inlines the relevant parts of the report so issue readers don't + need to fetch the local file. .PARAMETER Ctx - The run-context hashtable built by 04-run-batch.ps1. + The run-context object built by 04-run-batch.ps1. .PARAMETER From - Baseline upstream SHA before the run. + Baseline upstream SHA before the run (Get-LastSyncedUpstreamSha). .PARAMETER To Upstream HEAD SHA at fetch time. @@ -43,13 +52,13 @@ $fromSubj = Get-Subj $From $toSubj = Get-Subj $To $lines = New-Object System.Collections.Generic.List[string] -$lines.Add("# Upstream sync — $Status — $(Format-Iso8601 $started)") +$lines.Add("# Upstream sync - $Status - $(Format-Iso8601 $started)") $lines.Add("") $lines.Add("**Status:** $Status ") $lines.Add("**Host:** $($Ctx.Host) ") $lines.Add("**Duration:** $durStr ") -$lines.Add("**Baseline (before run):** ``$From`` — $fromSubj ") -$lines.Add("**Upstream HEAD:** ``$To`` — $toSubj ") +$lines.Add("**Baseline (before run):** ``$From`` - $fromSubj ") +$lines.Add("**Upstream HEAD:** ``$To`` - $toSubj ") $lines.Add("**Branch:** ``$($Ctx.Branch)`` ") $lines.Add("") $lines.Add("## Summary") @@ -65,7 +74,7 @@ if ($Ctx.StuckSha) { $lines.Add("") if ($Status -eq 'dry-run' -and $Ctx.Pending.Count -gt 0) { - $lines.Add("## Pending commits (oldest → newest)") + $lines.Add("## Pending commits (oldest -> newest)") $lines.Add("") $lines.Add("| # | SHA | Subject | Author |") $lines.Add("|---|---|---|---|") @@ -80,7 +89,7 @@ if ($Status -eq 'dry-run' -and $Ctx.Pending.Count -gt 0) { } if ($Ctx.Picked.Count -gt 0) { - $lines.Add("## Picked commits (oldest → newest)") + $lines.Add("## Picked commits (oldest -> newest)") $lines.Add("") $lines.Add("| # | SHA | Subject | Author |") $lines.Add("|---|---|---|---|") @@ -134,7 +143,7 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { $stuckAuthor = git log -1 --format='%an <%ae>' $Ctx.StuckSha $lines.Add("## Conflict diagnostics") $lines.Add("") - $lines.Add("**Conflicting commit:** [`$($Ctx.StuckSha)`](https://github.com/microsoft/terminal/commit/$($Ctx.StuckSha)) — $stuckSubj ") + $lines.Add("**Conflicting commit:** [`$($Ctx.StuckSha)`](https://github.com/microsoft/terminal/commit/$($Ctx.StuckSha)) - $stuckSubj ") $lines.Add("**Author:** $stuckAuthor") $lines.Add("") $stuckError = if ($Ctx.PSObject.Properties.Name -contains 'StuckError') { $Ctx.StuckError } else { $null } @@ -144,10 +153,7 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { foreach ($p in $Ctx.StuckPaths) { $lines.Add("- ``$p``") } $lines.Add("") } else { - # Non-conflict cherry-pick failure (e.g. merge commit picked without -m, - # hook failure). The resume guidance below still applies after the human - # addresses the underlying error. - $lines.Add("**No unmerged paths** — ``git cherry-pick`` failed for a reason other than a merge conflict (e.g. attempting to pick a merge commit without ``-m``, hook failure, or another non-conflict error).") + $lines.Add("**No unmerged paths** - ``git cherry-pick`` failed for a reason other than a merge conflict (e.g. attempting to pick a merge commit without ``-m``, hook failure, or another non-conflict error).") if ($stuckError) { $lines.Add("") $lines.Add("**Reported error:** ``$stuckError``") @@ -159,29 +165,25 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { $lines.Add("**How to resume:**") $lines.Add("") $lines.Add("1. ``git switch $($Ctx.Branch)``") - $lines.Add("2. Manually cherry-pick the stuck commit and resolve. Pin upstream identity/dates so the resolved commit matches the rest of this batch (otherwise the resolved commit's committer would be you, not the original upstream author — breaking the per-commit attribution the rest of the sync preserves):") + $lines.Add("2. Manually cherry-pick the stuck commit and resolve. Pin upstream identity/dates so the resolved commit matches the rest of this batch:") $lines.Add(" ``````pwsh") - $lines.Add(" `$info = (git -C `"`$((git -C . rev-parse --show-toplevel))`" log -1 --pretty=format:'%an%x00%ae%x00%aI%x00%cn%x00%ce%x00%cI' $($Ctx.StuckSha)) -split [char]0") + $lines.Add(" `$info = (git log -1 --pretty=format:'%an%x00%ae%x00%aI%x00%cn%x00%ce%x00%cI' $($Ctx.StuckSha)) -split [char]0") $lines.Add(" `$env:GIT_AUTHOR_NAME=`$info[0]; `$env:GIT_AUTHOR_EMAIL=`$info[1]; `$env:GIT_AUTHOR_DATE=`$info[2]") $lines.Add(" `$env:GIT_COMMITTER_NAME=`$info[3]; `$env:GIT_COMMITTER_EMAIL=`$info[4]; `$env:GIT_COMMITTER_DATE=`$info[5]") $lines.Add(" git cherry-pick -x $($Ctx.StuckSha)") $lines.Add(" # resolve conflicts, then:") $lines.Add(" git add -A; git cherry-pick --continue --no-edit") - $lines.Add(" # finally, clear the env vars so they don't leak into your next commit:") $lines.Add(" Remove-Item Env:GIT_AUTHOR_NAME,Env:GIT_AUTHOR_EMAIL,Env:GIT_AUTHOR_DATE,Env:GIT_COMMITTER_NAME,Env:GIT_COMMITTER_EMAIL,Env:GIT_COMMITTER_DATE -ErrorAction SilentlyContinue") $lines.Add(" ``````") - $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual resolution for $($Ctx.StuckSha.Substring(0,9))``, merge it.") - $lines.Add("4. Clear the lock:") - $lines.Add(" ``````") - $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1 -ResolvedThroughSha $($Ctx.StuckSha)") - $lines.Add(" ``````") - $lines.Add("5. The next scheduled sync resumes from the commit after this one.") + $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual resolution for $($Ctx.StuckSha.Substring(0,9))``, merge it. The ``-x`` trailer is what the next sync run reads as the new watermark - **do not strip it.**") + $lines.Add("4. Close the stuck issue. That's the lock-cleared signal; no script needed.") + $lines.Add("5. The next scheduled sync derives the new watermark from your merged PR and resumes from the commit after this one.") $lines.Add("") } if ($Status -like 'stuck-*') { $kind = $Status -replace '^stuck-','' - $lines.Add("## Validation diagnostics — Tier-4 ($kind)") + $lines.Add("## Validation diagnostics - Tier-4 ($kind)") $lines.Add("") $lines.Add("All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch validation step blocked the push.") $lines.Add("") @@ -197,7 +199,7 @@ if ($Status -like 'stuck-*') { $lines.Add("| Severity | Kind | Where | Detail |") $lines.Add("|---|---|---|---|") foreach ($f in $blocking) { - $where = if ($f.path) { "``$($f.path)``" } else { '—' } + $where = if ($f.path) { "``$($f.path)``" } else { '-' } $findingKind = if ($f.check) { $f.check } elseif ($f.kind) { $f.kind } else { 'unknown' } $detail = ($f | ConvertTo-Json -Compress -Depth 4) $lines.Add("| $($f.severity) | $findingKind | $where | ``$detail`` |") @@ -229,20 +231,17 @@ if ($Status -like 'stuck-*') { $lines.Add("**Available toolsets:** $($Ctx.Preflight.available_toolsets -join ', ')") $lines.Add("**Missing:** $($Ctx.Preflight.missing -join ', ')") $lines.Add("") - $lines.Add("This is an **infrastructure** problem — provision the host with the required Visual Studio toolset(s). No GitHub issue was opened because PR review cannot fix it.") + $lines.Add("This is an **infrastructure** problem - provision the host with the required Visual Studio toolset(s). No GitHub issue was opened because PR review cannot fix it.") $lines.Add("") } } $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") $lines.Add("") if ($kind -eq 'toolchain-missing') { - $lines.Add("**How to resume (infra-only — no PR needed):**") + $lines.Add("**How to resume (infra-only - no PR needed):**") $lines.Add("") $lines.Add("1. Provision the host with the missing toolset(s) above, **or** rerun the sync from a correctly provisioned host.") - $lines.Add("2. Clear the lock:") - $lines.Add(" ``````") - $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") - $lines.Add(" ``````") + $lines.Add("2. No issue was opened, so there is no lock to clear - the next scheduler tick just retries.") $lines.Add("3. The next scheduled sync re-runs the same range; nothing on ``$($Ctx.Branch)`` needs editing.") $lines.Add("") } else { @@ -250,19 +249,16 @@ if ($Status -like 'stuck-*') { $lines.Add("") $lines.Add("1. ``git switch $($Ctx.Branch)``") $lines.Add("2. Fix the issue above (e.g. resw dedup, restored fork invariant, build fix).") - $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it.") - $lines.Add("4. Clear the lock:") - $lines.Add(" ``````") - $lines.Add(" pwsh .github/skills/upstream-sync/scripts/clear-stuck.ps1") - $lines.Add(" ``````") - $lines.Add("5. The next scheduled sync runs the same range — validation must pass before any PR is opened.") + $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it (keep the ``-x`` trailers on every cherry-picked commit so the next run's watermark derivation still works).") + $lines.Add("4. Close the stuck issue. That's the lock-cleared signal; no script needed.") + $lines.Add("5. The next scheduled sync runs the same range - validation must pass before any PR is opened.") $lines.Add("") } } $lines.Add("---") $lines.Add("") -$lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``._") +$lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``. This file lives under `Generated Files/` and is gitignored - never committed._") $suffix = if ($Status -eq 'skipped-locked') { 'skipped' } elseif ($Status -eq 'skipped-pr-open') { 'skipped' } @@ -270,8 +266,12 @@ $suffix = if ($Status -eq 'skipped-locked') { 'skipped' } elseif ($Status -like 'stuck-*') { $Status } elseif ($Status -eq 'stuck') { 'stuck' } elseif ($Status -eq 'no-op') { 'noop' } - else { '' } -$name = Format-ReportFilename -When $started -Suffix $suffix -$path = Join-Path (Get-ReportsDir) $name + else { 'ok' } + +# Filenames are sortable by run timestamp + disambiguated by status. The +# parent directory (Generated Files/upstream-sync//) is already per-day. +$stamp = $started.ToString('yyyy-MM-ddTHHmmss') +$name = "$stamp-$suffix.md" +$path = Join-Path (Get-GeneratedDir) $name [System.IO.File]::WriteAllText($path, ($lines -join "`n"), (New-Object System.Text.UTF8Encoding($false))) return $path diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 index 0f468f25f..fe131a287 100644 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 @@ -1,16 +1,26 @@ <# .SYNOPSIS - Push the sync branch and open a PR. Commits state.json + report onto - the branch first so the merge atomically advances last_synced. + Push the sync branch and open a PR. No state file, no extra commits. + +.DESCRIPTION + The branch already carries the cherry-picked commits (each with its + `(cherry picked from commit )` trailer - that IS the watermark + the next run reads). We just push it and open the PR. No state.json + to commit, no pr_url backfill commit, no extra round-trip after PR + creation. .PARAMETER Ctx Run context from 04-run-batch.ps1. .PARAMETER To - Upstream HEAD SHA at fetch time (becomes new last_synced_upstream_sha). + Upstream HEAD SHA at fetch time (used only in the PR title). .PARAMETER ReportPath - Absolute path to the report markdown to use as the PR body. + Absolute path to the report markdown to use as the PR body. The report + itself is NOT committed - just inlined into the PR body text. + +.PARAMETER AutoMergeStrategy + rebase | merge | none. Passed to `gh pr merge --auto`. .OUTPUTS PR URL on stdout (and writes Ctx.PrUrl). @@ -25,23 +35,26 @@ param( . "$PSScriptRoot/Common.ps1" -# Prepend the squash-warning banner to the report so it lands as the -# first thing reviewers see in the PR body. +# Prepend the squash-warning + review-policy banner to the report so it +# lands as the first thing reviewers see. $banner = @" -> ⚠️ **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked +> [!WARNING] +> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked > upstream commit into one, destroying per-commit attribution, original -> author dates, and ``git bisect`` resolution. Merge with **"Rebase and -> merge"** (preferred — flat history, all $($Ctx.Picked.Count) commits land -> individually) or **"Create a merge commit"** (also preserves per-commit -> content). -> -> 📝 **Review-fix policy.** Only build-blocking fixes (compile errors, dedup +> author dates, the ``(cherry picked from commit )`` trailers that the +> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. +> Merge with **"Rebase and merge"** (preferred - flat history, all +> $($Ctx.Picked.Count) commits land individually) or **"Create a merge +> commit"** (also preserves per-commit content). + +> [!NOTE] +> **Review-fix policy.** Only build-blocking fixes (compile errors, dedup > of conflicts surfaced at build time, CI gate failures on this PR itself) -> belong here — as **one** focused extra commit on this branch. All other +> belong here - as **one** focused extra commit on this branch. All other > Copilot / human review feedback (code-quality, logic, translation, > spelling-list migrations, doc nits) goes into a **follow-up PR** based on -> this PR's head, not amended into the cherry-pick commits. Rationale and -> mechanics: [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). +> this PR's head. Rationale and mechanics: +> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). --- @@ -50,53 +63,29 @@ $bodyPath = New-TemporaryFile $bodyContent = $banner + (Get-Content -Raw -LiteralPath $ReportPath) [System.IO.File]::WriteAllText($bodyPath, $bodyContent, (New-Object System.Text.UTF8Encoding($false))) -$branch = $Ctx.Branch +$branch = $Ctx.Branch $shortTo = $To.Substring(0,9) -# Update state.json with new baseline and run summary, plus commit the report. -$state = Read-State -$state.last_synced_upstream_sha = $To -$runSummary = [ordered] @{ - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host - status = 'ok' - branch = $branch - pr_url = $null # filled in after PR creation - picked_count = $Ctx.Picked.Count - dropped_pair_count = $Ctx.DroppedPairs.Count - empty_count = $Ctx.SkippedEmpty.Count - tier0_resolutions = $Ctx.Tier0.Count -} -$state.last_run = $runSummary -$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 -Write-State $state - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $ReportPath) -if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } -git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting without push so baseline is not lost." } - +# Push the sync branch (cherry-pick commits already have their `-x` +# trailers; those trailers ARE the watermark - nothing else to commit). git push -u origin $branch | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push failed for $branch." } +if ($LASTEXITCODE -ne 0) { + Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue + throw "git push failed for $branch." +} $title = "chore(upstream): sync microsoft/terminal up to $shortTo" -# Same-repo PR: branch was pushed to origin (= microsoft/intelligent-terminal), -# so --head takes the bare branch name. `--head OWNER:BRANCH` would tell gh to -# look on a fork owned by OWNER, which is wrong for this scheduler. -# -# Retry up to 3 times with a short delay: `gh pr create` on Windows occasionally fails -# with "Head sha can't be blank" right after a push (see SKILL.md gotcha). -$prUrl = $null +# Same-repo PR: `--head` takes the bare branch name. Retry up to 3 times +# with a short delay - `gh pr create` on Windows occasionally fails with +# "Head sha can't be blank" right after a push. +$prUrl = $null $errFile = [System.IO.Path]::GetTempFileName() try { for ($attempt = 1; $attempt -le 3; $attempt++) { - # Capture stderr separately: merging via `2>&1` can let a `gh` warning - # (version notice, deprecation, etc.) become the last line, after - # which `Select-Object -Last 1` returns the warning text and the URL - # match fails even though the PR was successfully created. The temp - # file is reused across retries (overwritten each call); cleanup runs - # in the outer finally. + # Capture stderr to a separate temp file: a `gh` version-update / + # deprecation notice on stderr can otherwise become the last line + # of merged output, breaking URL match. Set-Content -LiteralPath $errFile -Value '' -NoNewline $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>$errFile | Select-Object -Last 1 if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } @@ -110,44 +99,15 @@ try { } } finally { - # Always clean up the temp PR body file and stderr-capture file — even if - # `gh pr create` failed after all retries, neither temp file should leak. Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue } $Ctx.PrUrl = $prUrl.Trim() -# Backfill PR URL into state.last_run AND state.history[0] (best-effort -# follow-up commit) BEFORE arming auto-merge. If auto-merge is already -# satisfied (all checks green, approvals in place) it can merge and delete -# the remote branch immediately, after which `git push origin $branch` -# would recreate a deleted upstream-sync/ branch as an orphan with -# the pr_url commit on top. Keeping the same run summary object in -# last_run and history[0] in sync so 'sessions' reports and bug-reports -# can find the PR link from either field. If this push fails the PR is -# still open and the baseline is still advanced on the branch — the only -# loss is the pr_url field in state, which is recoverable from the PR -# itself. -$state.last_run.pr_url = $Ctx.PrUrl -if ($state.history -and $state.history.Count -gt 0) { - $state.history[0].pr_url = $Ctx.PrUrl -} -Write-State $state -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null -if ($LASTEXITCODE -ne 0) { - Write-Warning "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); skipping pr_url backfill commit. PR is still open at $($Ctx.PrUrl)." -} else { - git commit -m "chore(upstream-sync): record PR url" | Out-Host - if ($LASTEXITCODE -eq 0) { - git push origin $branch | Out-Host - if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push pr_url backfill; PR is still open at $($Ctx.PrUrl)." } - } -} - -# Optional: arm GitHub auto-merge with the strategy that preserves per-commit -# history. 'rebase' is the recommended default when auto-merge is enabled — -# it lands all N commits flatly on main once CI + approvals pass. +# Optional: arm GitHub auto-merge with a strategy that preserves per-commit +# history. 'rebase' is the recommended default - it lands all N commits +# flatly on main once CI + approvals pass. Never squash. if ($AutoMergeStrategy -ne 'none') { $strategyFlag = "--$AutoMergeStrategy" gh pr merge -R microsoft/intelligent-terminal $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host diff --git a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 b/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 deleted file mode 100644 index 3682d7fe5..000000000 --- a/.github/skills/upstream-sync/scripts/06b-finalize-direct.ps1 +++ /dev/null @@ -1,111 +0,0 @@ -<# -.SYNOPSIS - Finalize the sync by fast-forwarding main directly to the sync branch. - Use when you have admin/bypass perms on main and want zero PR latency. - -.DESCRIPTION - Preserves per-commit content, order, and original author + committer - dates, because the cherry-pick loop pins both from each upstream commit. - - Assumes the caller is currently on the sync branch with all picks - applied. Performs: - 1. Add state.json + report, commit on the sync branch. - 2. Switch to main, pull --ff-only. - 3. Fast-forward main to the sync branch tip (refuses if main diverged). - 4. Push main. - 5. Delete the local sync branch (remote was never pushed). - -.PARAMETER Ctx - Run context. - -.PARAMETER To - Upstream HEAD SHA at fetch time (becomes new last_synced_upstream_sha). - -.PARAMETER ReportPath - Path to the markdown report (will be committed onto the sync branch). - -.OUTPUTS - Writes the new main HEAD SHA to stdout and to Ctx.MainHeadSha. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] $Ctx, - [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [string] $ReportPath -) - -. "$PSScriptRoot/Common.ps1" - -$branch = $Ctx.Branch -$shortTo = $To.Substring(0,9) - -# 1. Update state.json + commit (still on the sync branch). -$state = Read-State -$state.last_synced_upstream_sha = $To -$runSummary = [ordered] @{ - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host - status = 'ok' - branch = $branch - merge_mode = 'direct-push' - pr_url = $null - main_head_sha = $null # filled in after push - picked_count = $Ctx.Picked.Count - dropped_pair_count = $Ctx.DroppedPairs.Count - empty_count = $Ctx.SkippedEmpty.Count - tier0_resolutions = $Ctx.Tier0.Count -} -$state.last_run = $runSummary -$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 -Write-State $state - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $ReportPath) -if ($LASTEXITCODE -ne 0) { throw "git add of state.json + report failed." } -git commit -m "chore(upstream-sync): advance baseline to $shortTo" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit of state-update failed; aborting before touching main." } - -$syncTip = (git rev-parse HEAD).Trim() - -# 2. Switch to main and ensure we're current. -git switch main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git switch main failed." } -git pull --ff-only origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "main is not fast-forwardable from origin. Resolve manually." } - -# 3. Fast-forward main to the sync tip. If main has moved (someone landed -# something concurrently), this fails — and that's the correct behaviour: -# direct-push assumes you own the merge moment. Re-run the sync to rebase. -git merge --ff-only $syncTip | Out-Host -if ($LASTEXITCODE -ne 0) { - throw "Cannot fast-forward main onto $syncTip — main has commits the sync branch does not. Re-run the sync to start from the new main, or finalize via the PR path." -} - -# 4. Push main. -git push origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push origin main failed. The picks are local-only — push manually before the next scheduler tick or it will re-pick everything." } - -# 5. Local cleanup. Remote branch was never pushed in direct-push mode. -git branch -D $branch | Out-Host - -$mainHead = (git rev-parse HEAD).Trim() -$Ctx.MainHeadSha = $mainHead - -# Backfill main_head_sha into state.last_run (best-effort). -$state = Read-State -if ($state.last_run -and $state.history -and $state.history.Count -gt 0) { - $state.last_run.main_head_sha = $mainHead - $state.history[0].main_head_sha = $mainHead - Write-State $state - git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null - if ($LASTEXITCODE -ne 0) { - Write-Warning "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); skipping main_head_sha backfill (cosmetic only; sync content already on main)." - } else { - git commit -m "chore(upstream-sync): record main head $($mainHead.Substring(0,9))" | Out-Host - if ($LASTEXITCODE -eq 0) { - git push origin main | Out-Host - if ($LASTEXITCODE -ne 0) { Write-Warning "Backfill push failed (cosmetic only; sync content already on main)." } - } - } -} - -return $mainHead diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 index f1b451525..c99e1f741 100644 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 @@ -1,7 +1,15 @@ <# .SYNOPSIS - Set the stuck-lock and open a GitHub issue. Pushes the stuck branch - to origin so the human can pick it up. + Tier-3 (cherry-pick stopped at a real merge conflict) stuck-issue opener. + +.DESCRIPTION + No state.json. The "lock" is the OPEN labeled issue itself - the next + scheduler run calls Get-StuckIssues and bails when this issue is found. + The issue body carries a fenced ```yaml # wta-state``` block with the + same metadata the old state.json held (tier, stuck_on_sha, branch, + findings_hash) so re-runs can reason about it. + + Cleared by: a human closes the issue. That's it - no separate script. .PARAMETER Ctx Run context (must have StuckSha, StuckPaths, Branch set). @@ -20,25 +28,44 @@ param( . "$PSScriptRoot/Common.ps1" -if (-not $Ctx.StuckSha) { throw "Ctx.StuckSha is empty — nothing to escalate." } +if (-not $Ctx.StuckSha) { throw "Ctx.StuckSha is empty - nothing to escalate." } # Push the stuck branch so the human can resume on it. git push -u origin $Ctx.Branch 2>&1 | Out-Host -if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } +if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch - issue still being filed for visibility." } $shortSha = $Ctx.StuckSha.Substring(0,9) $subj = (git log -1 --format='%s' $Ctx.StuckSha).Trim() $title = "Upstream sync stuck at ${shortSha}: $subj" -# Header prepended to report content. +# Build the lock-state YAML block. Schedulers read this on the next run +# via Get-StuckMetaFromIssue - it carries enough metadata to recognize a +# re-issue of the same failure without re-spamming a new ticket. +$stuckErrorVal = if ($Ctx.PSObject.Properties.Name -contains 'StuckError' -and $Ctx.StuckError) { $Ctx.StuckError } else { '' } +$yamlBlock = Format-StuckYamlBlock @{ + tier = '3' + kind = 'cherry-pick-conflict' + stuck_on_sha = $Ctx.StuckSha + branch = $Ctx.Branch + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host + conflict_count = ($Ctx.StuckPaths | Measure-Object).Count + error = $stuckErrorVal +} + $header = @" -🛑 **Upstream sync stopped at a conflict that needs human judgment.** +> [!CAUTION] +> **Upstream sync stopped at a conflict that needs human judgment.** +> +> The scheduler will keep skipping its runs until this issue is **closed**. +> Closing the issue IS the lock-clear signal - no separate script needed. -The scheduler will keep skipping its runs until this issue is resolved -and the stuck-lock is cleared. No alarm — the lock is intentional. +**How to unblock:** follow "How to resume" in the report excerpt below, +merge your manual-resolution PR (keeping the ``(cherry picked from commit +)`` trailer - that's what the next sync run reads as its watermark), +then close this issue. -To unblock, follow "How to resume" in the report below, then close this -issue after ``clear-stuck.ps1`` runs cleanly. +$yamlBlock --- @@ -47,72 +74,32 @@ $body = $header + (Get-Content -Raw -LiteralPath $ReportPath) $tmp = New-TemporaryFile [System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) -# Ensure label exists (best-effort; ignore if already present). -# -R is pinned for the same reason as the issue-create call below: an `upstream` -# remote makes `gh` default to microsoft/terminal, where this account does not -# have label-create permission. +# Ensure label exists (best-effort). -R pinned because an `upstream` remote +# can make `gh` default to microsoft/terminal where this account has no +# label-create permission. gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null -# -R is explicit because the `upstream` remote can make gh default to microsoft/terminal. -# Capture stderr to a separate temp file so a `gh` warning on stderr (version notice etc.) -# can't displace the URL as the "last line" of merged output. -$errFile = [System.IO.Path]::GetTempFileName() -$errText = '' +# Capture stderr to a separate temp file so a `gh` warning on stderr can't +# displace the URL as the "last line" of merged output. +$errFile = [System.IO.Path]::GetTempFileName() +$errText = '' +$issueUrl = $null +$ghExit = 0 try { $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 $ghExit = $LASTEXITCODE if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } } finally { - Remove-Item $tmp -Force -ErrorAction SilentlyContinue - Remove-Item $errFile -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue } if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" } $Ctx.IssueUrl = $issueUrl.Trim() -# Set the stuck-lock on main (direct push — the lock is the gate; PR path is blocked). -git switch main | Out-Null -if ($LASTEXITCODE -ne 0) { throw "Could not switch to main before writing stuck-lock. Resolve manually and re-run; the stuck branch + issue are already in place." } -git pull --ff-only origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run; the stuck branch + issue are already in place." } - -$state = Read-State -$state.stuck_on_sha = $Ctx.StuckSha -$state.stuck_branch = $Ctx.Branch -$state.stuck_at = Format-Iso8601 $Ctx.StartedAt -$state.stuck_issue_url = $Ctx.IssueUrl -# Single active lock: clear the Tier-4 fields when promoting a Tier-3 lock so -# the scheduler and humans see one stuck reason, not two. -if ($state.PSObject.Properties.Name -contains 'stuck_validation') { - $state.stuck_validation = $null -} -$runSummary = [ordered] @{ - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host - status = 'stuck' - branch = $Ctx.Branch - issue_url = $Ctx.IssueUrl - stuck_on_sha = $Ctx.StuckSha - picked_count = $Ctx.Picked.Count -} -$state.last_run = $runSummary -$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 -Write-State $state - -# Copy the report into main too (so the report is visible without checking out the stuck branch). -$reportName = Split-Path -Leaf $ReportPath -$reportOnMain = Join-Path (Get-ReportsDir) $reportName -if ($ReportPath -ne $reportOnMain) { - Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force -} - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $reportOnMain) -if ($LASTEXITCODE -ne 0) { throw "git add of stuck state failed." } -git commit -m "chore(upstream-sync): stuck at $shortSha (#$($Ctx.IssueUrl -replace '.*/',''))" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit of stuck-lock failed — lock NOT set on origin/main. The next scheduled run will not see the lock; resolve manually." } -git push origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — stuck-lock is local only. Push manually before the next scheduler tick or it will re-run over the conflict." } - +# That's it. The open labeled issue IS the lock - no state file to write, +# no main-branch commit, nothing to push. The next scheduler run will see +# the open issue via Get-StuckIssues and skip. return $Ctx.IssueUrl diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index d3f9ec0e2..e283276fe 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -2,10 +2,22 @@ .SYNOPSIS Tier-4 (post-pick validation failure) stuck-issue opener. - Counterpart to 07-open-stuck-issue.ps1 (which handles Tier-3 = a - cherry-pick stopped mid-pick on a real merge conflict). Tier-4 means - all picks completed cleanly but the static scan, toolchain preflight, - or try-build step said NO. +.DESCRIPTION + Counterpart to 07-open-stuck-issue.ps1 (which handles Tier-3 = cherry-pick + stopped mid-pick on a real merge conflict). Tier-4 means all picks + completed cleanly but the static scan, toolchain preflight, or try-build + step said NO. + + No state.json - the lock is the open labeled issue itself (cleared by + the human closing the issue). The fenced ```yaml # wta-state``` block + in the body carries findings_hash so re-runs of the same broken batch + can be matched against the same issue. + + `toolchain-missing` is special: it's an INFRA problem (this host lacks + the required VS toolset), not a CODE problem. We do not open a GitHub + issue for it - issues are noise for things humans can't fix via the PR + surface. The scheduler simply retries next tick from another host (or + after provisioning). .PARAMETER Ctx Run context. Must have Branch set; uses Picked, Preflight, Scan, Build. @@ -19,13 +31,7 @@ .OUTPUTS Issue URL on stdout (and writes Ctx.IssueUrl + Ctx.StuckValidation). - - toolchain-missing is special: it's an INFRA problem (this host lacks - the required VS toolset), not a CODE problem. We do not open a - GitHub issue for it — issues are noise for things humans can't fix - from the PR side. Instead we set the stuck-lock so the scheduler - won't keep retrying, and the host owner notices via the next dev - run / monitoring. + For toolchain-missing, returns $null. #> [CmdletBinding()] param( @@ -36,9 +42,9 @@ param( . "$PSScriptRoot/Common.ps1" -# Compute a findings hash so re-runs of the same broken batch are detectable. -# Use [ordered] hashtables so JSON serialization (and therefore the hash) is -# stable across runs — plain @{} property order is not guaranteed. +# Compute a findings hash so re-runs of the same broken batch are detectable +# from a future run's gh issue body. Use [ordered] hashtables so JSON +# serialization (and therefore the hash) is stable across runs. $findingsForHash = switch ($Kind) { 'static-scan' { $Ctx.Scan.findings } 'build-failed' { @([ordered] @{ exit_code = $Ctx.Build.exit_code; tail_excerpt = ($Ctx.Build.log_tail -split "`n" | Select-Object -Last 20) -join "`n" }) } @@ -47,139 +53,95 @@ $findingsForHash = switch ($Kind) { } $findingsHash = Get-FindingsHash $findingsForHash -# Establish base/head for this batch (best-effort; tolerate detached states). -# `base` must be the *pre-pick* origin/main, not its current tip, because -# origin/main may have advanced while the run was in progress. The merge-base -# of the sync tip and origin/main recovers the branch's original starting -# point reliably. -$headRaw = git rev-parse HEAD 2>$null -$head = if ($LASTEXITCODE -eq 0 -and $headRaw) { $headRaw.Trim() } else { $null } -$base = $null -if ($head) { - $baseRaw = git merge-base HEAD origin/main 2>$null - if ($LASTEXITCODE -eq 0 -and $baseRaw) { $base = $baseRaw.Trim() } -} -if (-not $base) { - # Fallback: use origin/main tip if merge-base failed (e.g. no upstream ref). - $baseRaw = git rev-parse origin/main 2>$null - if ($LASTEXITCODE -eq 0 -and $baseRaw) { $base = $baseRaw.Trim() } -} - -# Push the sync branch so the human can resume on it (even toolchain-missing — +# Push the sync branch so the human can resume on it (even toolchain-missing - # the picks are still useful artifacts for whoever owns the host). git push -u origin $Ctx.Branch 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { - Write-Warning "Could not push sync branch — stuck-lock will still be set." + Write-Warning "Could not push sync branch - issue will still be filed for visibility (when applicable)." } -# Build validation payload (stored in state.json regardless of whether an issue is filed). +# Tier-4 metadata stashed on the context for the orchestrator's logs. $validation = [ordered] @{ - kind = $Kind - base = $base - head = $head - branch = $Ctx.Branch - range = @($Ctx.Picked) - findings_hash = $findingsHash - at = Format-Iso8601 $Ctx.StartedAt - issue_url = $null + kind = $Kind + branch = $Ctx.Branch + range = @($Ctx.Picked) + findings_hash = $findingsHash + at = Format-Iso8601 $Ctx.StartedAt + issue_url = $null } -# For toolchain-missing we do NOT open an issue (infra problem, not code). -if ($Kind -ne 'toolchain-missing') { - $titleKindLabel = switch ($Kind) { - 'static-scan' { 'static scan' } - 'build-failed' { 'build failure' } - 'build-inconclusive' { 'build inconclusive (timeout)' } - } - $title = "Upstream sync stuck after $($Ctx.Picked.Count) clean picks: $titleKindLabel ($findingsHash)" +# For toolchain-missing we do NOT open an issue (infra problem, not code - +# and no lock either: the next tick simply retries from any properly +# provisioned host). +if ($Kind -eq 'toolchain-missing') { + $Ctx.StuckValidation = $validation + return $null +} - $header = @" -🛑 **Upstream sync stopped after validation failed.** +$titleKindLabel = switch ($Kind) { + 'static-scan' { 'static scan' } + 'build-failed' { 'build failure' } + 'build-inconclusive' { 'build inconclusive (timeout)' } +} +$title = "Upstream sync stuck after $($Ctx.Picked.Count) clean picks: $titleKindLabel ($findingsHash)" -All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch -validation step said NO before any PR was opened. Stop reason: **$Kind**. +$yamlBlock = Format-StuckYamlBlock @{ + tier = '4' + kind = $Kind + branch = $Ctx.Branch + findings_hash = $findingsHash + picked_count = $Ctx.Picked.Count + at = Format-Iso8601 $Ctx.StartedAt + host = $Ctx.Host +} -The scheduler will keep skipping its runs until this is resolved and -``clear-stuck.ps1`` runs cleanly. No alarm — the lock is intentional. +$header = @" +> [!CAUTION] +> **Upstream sync stopped after validation failed.** +> +> All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch +> validation step said NO before any PR was opened. Stop reason: **$Kind**. +> +> The scheduler will keep skipping its runs until this issue is **closed**. +> Closing the issue IS the lock-clear signal - no separate script needed. Sync branch: ``$($Ctx.Branch)`` (pushed to origin). Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). +$yamlBlock + --- "@ - $body = $header + (Get-Content -Raw -LiteralPath $ReportPath) - $tmp = New-TemporaryFile - [System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - - # Ensure label exists (best-effort). -R pinned for the same reason as the - # issue-create call below (avoid the `upstream` remote tricking gh into - # microsoft/terminal). - gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null - - # Capture stderr to a separate temp file so a `gh` warning on stderr (version - # notice etc.) can't displace the URL as the "last line" of merged output — - # `state.stuck_validation.issue_url` must always be either a real URL or unset. - $errFile = [System.IO.Path]::GetTempFileName() - $errText = '' - try { - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } - } - finally { - Remove-Item $tmp -Force -ErrorAction SilentlyContinue - Remove-Item $errFile -Force -ErrorAction SilentlyContinue - } - if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" - } - $validation.issue_url = $issueUrl.Trim() - $Ctx.IssueUrl = $validation.issue_url +$body = $header + (Get-Content -Raw -LiteralPath $ReportPath) +$tmp = New-TemporaryFile +[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) + +# Ensure label exists (best-effort). -R pinned for the same reason as the +# issue-create call below (avoid the `upstream` remote tricking gh into +# microsoft/terminal). +gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null + +# Capture stderr to a separate temp file so a `gh` warning on stderr can't +# displace the URL as the "last line" of merged output. +$errFile = [System.IO.Path]::GetTempFileName() +$errText = '' +$issueUrl = $null +$ghExit = 0 +try { + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } } - -$Ctx.StuckValidation = $validation - -# Write the lock onto origin/main. -git switch main | Out-Null -if ($LASTEXITCODE -ne 0) { throw "Could not switch to main before writing stuck-lock. Resolve manually and re-run." } -git pull --ff-only origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "Could not fast-forward main before writing stuck-lock. Resolve manually and re-run." } - -$state = Read-State -$state.stuck_validation = $validation -# Single active lock: clear the Tier-3 fields when setting a Tier-4 lock so -# state.json reflects one stuck reason, not two stale ones. -foreach ($f in 'stuck_on_sha','stuck_branch','stuck_at','stuck_issue_url') { - if ($state.PSObject.Properties.Name -contains $f) { - $state.$f = $null - } +finally { + Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue } -$runSummary = [ordered] @{ - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host - status = "stuck-$Kind" - branch = $Ctx.Branch - issue_url = $validation.issue_url - findings_hash = $findingsHash - picked_count = $Ctx.Picked.Count +if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" } -$state.last_run = $runSummary -$state.history = @($runSummary) + @($state.history) | Select-Object -First 20 -Write-State $state - -# Copy report into main reports/ so it's discoverable without checking out the branch. -$reportName = Split-Path -Leaf $ReportPath -$reportOnMain = Join-Path (Get-ReportsDir) $reportName -if ($ReportPath -ne $reportOnMain) { Copy-Item -LiteralPath $ReportPath -Destination $reportOnMain -Force } - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) (ConvertTo-RepoRelativePath $reportOnMain) | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git add failed for Tier-4 stuck state." } - -$msgIssueRef = if ($validation.issue_url) { " (#$($validation.issue_url -replace '.*/',''))" } else { ' (no issue — infra)' } -git commit -m "chore(upstream-sync): stuck post-pick: $Kind$msgIssueRef" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit of Tier-4 stuck-lock failed — lock NOT set on origin/main." } -git push origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — Tier-4 stuck-lock local only." } +$validation.issue_url = $issueUrl.Trim() +$Ctx.IssueUrl = $validation.issue_url +$Ctx.StuckValidation = $validation return $validation.issue_url diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/10-try-build.ps1 index 3c1dcb71c..66d1febe9 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/10-try-build.ps1 @@ -14,7 +14,7 @@ .PARAMETER LogDir Where to write the full build log. Default: - /.github/upstream-sync/build-logs/. + `Generated Files/upstream-sync//build-logs/` (gitignored). .OUTPUTS JSON on stdout: @@ -48,10 +48,12 @@ param( . "$PSScriptRoot/Common.ps1" try { - $root = Get-RepoRoot if (-not $LogDir) { - $LogDir = Join-Path $root '.github/upstream-sync/build-logs' + # Default: per-day, per-skill artifact dir under the gitignored + # `Generated Files/` root. Get-GeneratedDir creates it on demand. + $LogDir = Get-GeneratedDir -Sub 'build-logs' } + $root = Get-RepoRoot if (-not (Test-Path -LiteralPath $LogDir)) { New-Item -ItemType Directory -Path $LogDir -Force | Out-Null } $stamp = (Get-Date).ToString('yyyy-MM-ddTHHmmss') diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 41ce46ea8..3dd0eea66 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -1,46 +1,45 @@ # Common.ps1 — shared helpers for upstream-sync scripts. # Dot-source from each script: . "$PSScriptRoot/Common.ps1" +# +# State model +# ----------- +# This skill does NOT keep a state.json file. Every persistent fact lives in +# the authoritative source that owns it: +# +# * "What's already been picked?" -> the `cherry picked from commit ` +# trailers we write on every `git cherry-pick -x` (parsed from origin/main). +# * "What's pending?" -> `git log --cherry-pick --right-only`. +# * "Is the scheduler locked?" -> any OPEN gh issue carrying the +# `upstream-sync-stuck` label. Lock metadata (kind, tier, stuck_on_sha, +# findings_hash) is encoded in a fenced ```yaml # wta-state ... ``` block +# in the issue body so re-runs can recognize the same failure. +# * "Transient artifacts" (build logs, generated reports) -> written under +# `Generated Files/upstream-sync//` which is gitignored at the +# repo root (`**/Generated Files/`). Never committed. $ErrorActionPreference = 'Stop' Set-StrictMode -Version Latest +# --------------------------------------------------------------------------- +# Repo + path helpers +# --------------------------------------------------------------------------- + function Get-RepoRoot { $r = git rev-parse --show-toplevel 2>$null if ($LASTEXITCODE -ne 0) { throw "Not inside a git repo." } return $r.Trim() } -function Get-StateDir { - Join-Path (Get-RepoRoot) '.github/upstream-sync' -} - -function Get-StatePath { - Join-Path (Get-StateDir) 'state.json' -} - -function Get-ReportsDir { - $d = Join-Path (Get-StateDir) 'reports' - if (-not (Test-Path $d)) { New-Item -ItemType Directory -Path $d | Out-Null } - return $d -} - function ConvertTo-RepoRelativePath { - # `git add` path arguments behave differently when absolute vs. - # repo-relative, depending on the worktree-root vs. cwd interaction. - # Normalize every path our scripts hand to `git add` to a - # forward-slash, repo-relative form so the call is portable and - # will not silently no-op (or worse, leak a path-outside-tree - # error) on platforms where git treats absolute paths strictly. + # Normalize a path to forward-slash, repo-relative form so callers can + # safely embed it in committed text without leaking machine-specific + # drive letters / user dirs. param([Parameter(Mandatory)] [string] $Path) $root = ((Get-RepoRoot) -replace '\\','/').TrimEnd('/') $abs = $Path -replace '\\','/' if ($abs.Equals($root, [System.StringComparison]::OrdinalIgnoreCase)) { throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" } - # Require a path-segment boundary after the prefix so 'C:/repo' does not - # match 'C:/repo-old/file' (prior implementation took the substring after - # the root length and only trimmed leading '/', which silently mangled - # sibling paths into garbage relative ones). $prefix = "$root/" if ($abs.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { return $abs.Substring($prefix.Length) @@ -48,21 +47,25 @@ function ConvertTo-RepoRelativePath { throw "ConvertTo-RepoRelativePath: '$Path' is not under repo root '$root'." } -function Read-State { - $p = Get-StatePath - if (-not (Test-Path $p)) { - throw "state.json not found at $p. Run .github/skills/upstream-sync/scripts/00-bootstrap.ps1 (from the repo root) first — see references/bootstrap.md." +function Get-GeneratedDir { + # Per-skill, per-day artifact directory under the repo's gitignored + # `Generated Files/` root (matches the workspace convention used by other + # skills; the repo's top-level .gitignore has `**/Generated Files/`). + # Optional -Sub appends a subdirectory (e.g. 'build-logs'). + param([string] $Sub) + $root = Get-RepoRoot + $date = (Get-Date).ToString('yyyy-MM-dd') + $path = Join-Path $root "Generated Files/upstream-sync/$date" + if ($Sub) { $path = Join-Path $path $Sub } + if (-not (Test-Path -LiteralPath $path)) { + New-Item -ItemType Directory -Path $path -Force | Out-Null } - return Get-Content -Raw -LiteralPath $p | ConvertFrom-Json + return $path } -function Write-State { - param([Parameter(Mandatory)] $State) - $p = Get-StatePath - $json = ($State | ConvertTo-Json -Depth 12) + [Environment]::NewLine - # Use UTF-8 *without* BOM to match git's default text handling on this repo. - [System.IO.File]::WriteAllText($p, $json, (New-Object System.Text.UTF8Encoding($false))) -} +# --------------------------------------------------------------------------- +# Git + remote setup +# --------------------------------------------------------------------------- function Ensure-UpstreamRemote { param( @@ -88,11 +91,174 @@ function Assert-CleanWorktree { function Resolve-FullCommitSha { param([Parameter(Mandatory)] [string] $Sha) - $full = (git rev-parse "$Sha^{commit}" 2>$null).Trim() + $full = (git rev-parse "$Sha^{commit}" 2>$null) if ($LASTEXITCODE -ne 0 -or -not $full) { throw "Could not resolve commit SHA '$Sha'." } - return $full + return $full.Trim() +} + +# --------------------------------------------------------------------------- +# Derived state — replaces the old Read-State/Write-State on state.json +# --------------------------------------------------------------------------- + +function Get-LastSyncedUpstreamSha { + # "How far have we already synced from upstream/main?" Derived from the + # `(cherry picked from commit )` trailers that `git cherry-pick -x` + # writes on every pick. We walk origin/main newest-first and return the + # FIRST trailer that points at a commit reachable from upstream/main. + # + # Why newest-first instead of "highest topological position": cherry-picks + # land in chronological order on origin/main, so the most recent trailer + # is the watermark. A picked-then-reverted upstream commit will appear in + # an OLDER trailer (with a corresponding `Revert "..."` commit later) - + # `git cherry` (used by Get-PendingUpstreamShas) will see the revert and + # correctly re-list it as pending if needed, so this watermark only needs + # to be the high-water-mark of progress, not the strict frontier. + # + # Known limitation: a HUMAN who manually cherry-picks an upstream hotfix + # onto origin/main jumps the watermark forward even though earlier + # upstream commits remain unsynced. Mitigation: Get-PendingUpstreamShas + # uses patch-id comparison against ALL of origin/main (not just the + # commits after the watermark), so the unsynced earlier commits will + # still be picked up on the next scheduler run. The watermark only + # narrows the `git log` walk for speed - it isn't load-bearing for + # correctness of the pending list. + # + # Performance: capped at the most recent 5000 origin/main commits via + # --max-count. Even years of fork history fits well under that cap; if + # a real deployment ever exceeds it we re-throw with the standard + # not-found message so the operator can re-seed with the fast path. + # Requires `upstream` remote fetched: caller must Ensure-UpstreamRemote + + # `git fetch upstream main --no-tags` first. + $commits = @(git log origin/main --max-count=5000 --grep='cherry picked from commit' --format='%H' 2>$null) + if ($LASTEXITCODE -ne 0) { throw "git log on origin/main failed while deriving last-synced SHA." } + foreach ($c in $commits) { + $body = git log -1 --format='%B' $c 2>$null + if ($body -match '\(cherry picked from commit ([0-9a-f]{7,40})\)') { + $upstreamSha = $matches[1] + $null = git merge-base --is-ancestor $upstreamSha upstream/main 2>$null + if ($LASTEXITCODE -eq 0) { + return (Resolve-FullCommitSha $upstreamSha) + } + } + } + throw "No 'cherry picked from commit' trailer pointing at upstream/main was found on origin/main (scanned the most recent 5000 commits). Either the fork hasn't synced via this skill before, or the trailer convention drifted. The very first sync needs an operator to seed the watermark commit (see SKILL.md - 'First-time sync')." } +function Get-PendingUpstreamShas { + # Returns upstream/main commits that don't have an equivalent on + # origin/main, in chronological (oldest-first) order. Uses + # `git log --cherry-pick --right-only` which compares patch IDs (not just + # trailers), so picked-and-reverted commits correctly re-appear. + # + # The range we walk is ALWAYS `origin/main...upstream/main` regardless of + # -Since. -Since (the watermark) is treated as a known floor: we drop any + # commit that is an ancestor of -Since. This way: + # * The patch-id filter still considers ALL of origin/main (so commits + # that landed on origin/main outside the scheduler's trailer trail - + # e.g. a manual cherry-pick - still get filtered out). + # * The watermark only trims the obviously-old tail at the bottom of + # the resulting list, which is what makes the walk fast. + param( + [string] $Since, + [int] $Limit = 0 + ) + $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) + if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } + $shas = @($out | Where-Object { $_ -match '^[0-9a-f]{40}$' }) + if ($Since) { + # Skip commits that are ancestors of the watermark. We can't use + # `..upstream/main` for the range because that would re-include + # commits filtered by --cherry-pick; explicit per-sha ancestry check + # is O(n) git calls but n is small (only the suffix matters). + $filtered = New-Object 'System.Collections.Generic.List[string]' + foreach ($sha in $shas) { + $null = git merge-base --is-ancestor $sha $Since 2>$null + if ($LASTEXITCODE -ne 0) { [void] $filtered.Add($sha) } + } + $shas = @($filtered) + } + if ($Limit -gt 0) { $shas = @($shas | Select-Object -First $Limit) } + return ,$shas +} + +# --------------------------------------------------------------------------- +# Stuck-lock — derived from open `upstream-sync-stuck` labeled issues +# --------------------------------------------------------------------------- + +$script:StuckLabel = 'upstream-sync-stuck' +$script:WtaStateFence = '# wta-state' # marker inside ```yaml ... ``` blocks + +function Get-StuckIssues { + # Returns all OPEN issues carrying the upstream-sync-stuck label. -R is + # pinned because an `upstream` remote can trick gh into defaulting to + # microsoft/terminal, where this account has no permission. Stderr goes + # to a temp file so a gh deprecation/version notice can't break the JSON. + $errFile = [System.IO.Path]::GetTempFileName() + $errText = '' + $ghExit = 0 + try { + $json = gh issue list --repo microsoft/intelligent-terminal --label $script:StuckLabel --state open --json number,title,body,url,labels,createdAt 2>$errFile + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } + } + finally { + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue + } + if ($ghExit -ne 0) { + throw "gh issue list failed (exit $ghExit): $errText" + } + if (-not $json) { return @() } + return @($json | ConvertFrom-Json) +} + +function Get-StuckMetaFromIssue { + # Parse a fenced ```yaml ... # wta-state ... ``` block out of an issue body. + # Accepts the single-quoted form Format-StuckYamlBlock emits; values are + # un-escaped (`''` -> `'`). Returns $null if no block is found (degraded + # but safe: callers should still treat the open issue as a lock - the + # metadata is for findings_hash compare and resume hints, not for the + # lock decision itself). + param([Parameter(Mandatory)] $Issue) + if (-not $Issue.body) { return $null } + $pattern = '(?ms)```yaml\s*\r?\n#\s*wta-state\s*\r?\n(.+?)\r?\n```' + if ($Issue.body -notmatch $pattern) { return $null } + $yaml = $matches[1] + $h = [ordered] @{} + foreach ($l in $yaml -split "`r?`n") { + if ($l -match "^\s*([a-z_][a-z0-9_]*)\s*:\s*'((?:[^']|'')*)'\s*$") { + $h[$matches[1]] = $matches[2] -replace "''", "'" + } elseif ($l -match '^\s*([a-z_][a-z0-9_]*)\s*:\s*(.+?)\s*$') { + # Tolerate bare scalars for backward compatibility with hand-edited + # issues; the writer always quotes, but a human edit might not. + $h[$matches[1]] = $matches[2] + } + } + return [pscustomobject] $h +} + +function Format-StuckYamlBlock { + # Build the fenced YAML block that 07/07b embed in stuck-issue bodies. + # Values are always single-quoted with `'` -> `''` escaping so embedded + # colons, newlines, leading dashes, etc. round-trip without breaking the + # parser in Get-StuckMetaFromIssue or any other YAML reader. Multiline + # values are folded to spaces (we don't need full block-scalar support; + # the lock decision is just "is the list non-empty"). + param([Parameter(Mandatory)] [hashtable] $Fields) + $lines = @('```yaml', $script:WtaStateFence) + foreach ($k in $Fields.Keys) { + $raw = "$($Fields[$k])" + $folded = $raw -replace "`r?`n", ' ' + $escaped = $folded -replace "'", "''" + $lines += ("{0}: '{1}'" -f $k, $escaped) + } + $lines += '```' + return ($lines -join "`n") +} + +# --------------------------------------------------------------------------- +# Misc helpers +# --------------------------------------------------------------------------- + function Get-GhUserLogin { $login = gh api user --jq '.login' 2>$null if ($LASTEXITCODE -ne 0 -or -not $login) { throw "gh CLI is not authenticated. Run 'gh auth login'." } @@ -104,13 +270,6 @@ function Format-Iso8601 { return $When.ToString('yyyy-MM-ddTHH:mm:sszzz') } -function Format-ReportFilename { - param([DateTime] $When = (Get-Date), [string] $Suffix = '') - $stamp = $When.ToString('yyyy-MM-ddTHHmm') - if ($Suffix) { return "$stamp-$Suffix.md" } - return "$stamp.md" -} - function New-RunContext { [pscustomobject] @{ StartedAt = Get-Date @@ -124,12 +283,11 @@ function New-RunContext { Tier2 = @() StuckSha = $null StuckPaths = @() - # Tier-4 (post-pick validation failure): - StuckValidation = $null # hashtable { kind, base, head, range, findings_hash, branch, at, issue_url, ... } - # Validation step results (whether or not they blocked): - Preflight = $null # JSON from 09-toolchain-preflight.ps1 - Scan = $null # JSON from 08-static-scan.ps1 - Build = $null # JSON from 10-try-build.ps1 + StuckError = $null + StuckValidation = $null + Preflight = $null + Scan = $null + Build = $null Status = 'unknown' ReportPath = $null PrUrl = $null @@ -139,8 +297,9 @@ function New-RunContext { function Get-FindingsHash { param([Parameter(Mandatory)] $Findings) - # Stable hash of a findings list — used as a stuck_validation.findings_hash - # so repeat-runs of the same broken batch can be detected (and not re-issued). + # Stable hash of a findings list — used as a stuck-issue findings_hash + # so repeat-runs of the same broken batch can detect "same failure as + # last time" and avoid re-opening duplicate issues. $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) $sha = [System.Security.Cryptography.SHA256]::Create() try { diff --git a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 b/.github/skills/upstream-sync/scripts/clear-stuck.ps1 deleted file mode 100644 index 436a8c968..000000000 --- a/.github/skills/upstream-sync/scripts/clear-stuck.ps1 +++ /dev/null @@ -1,113 +0,0 @@ -<# -.SYNOPSIS - Clear the stuck-lock (Tier-3 or Tier-4) after a human has resolved - the underlying issue. - -.DESCRIPTION - Detects which kind of stuck is currently set in state.json and clears it: - - - Tier-3 (stuck_on_sha set): the scheduler stopped mid-cherry-pick on - a real merge conflict. -ResolvedThroughSha is REQUIRED and is - validated to be on upstream/main and >= stuck_on_sha; it becomes - the new last_synced_upstream_sha. - - - Tier-4 (stuck_validation set): the picks were clean but post-pick - validation failed (static scan / build / toolchain). The human - fixed it (e.g. resw dedup, restored fork invariant, fixed build, - or provisioned the missing toolset). -ResolvedThroughSha is - OPTIONAL — if omitted, last_synced_upstream_sha is left as-is so - the next scheduler run re-attempts the same range; if provided, - it advances the watermark just like Tier-3. - -.PARAMETER ResolvedThroughSha - See above. - -.PARAMETER Reason - Optional human-readable note recorded in the history entry. -#> -[CmdletBinding()] -param( - [string] $ResolvedThroughSha, - [string] $Reason = '' -) - -. "$PSScriptRoot/Common.ps1" - -# Fast-forward local main from origin BEFORE reading state.json so the -# stuck-lock decision (and any subsequent mutations) operates on the -# authoritative state from origin/main rather than a stale local copy. -# A stale local main would otherwise let the script falsely conclude -# "no stuck-lock is set" (early-return) when origin/main actually has -# one, or — worse — overwrite a newer state.json from origin/main when -# the clear-stuck commit lands. -Assert-CleanWorktree -Ensure-UpstreamRemote -git fetch upstream main --no-tags | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed; refusing to clear stuck-lock against stale refs." } - -git switch main | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git switch main failed; refusing to clear stuck-lock from the wrong branch." } -git pull --ff-only origin main | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git pull --ff-only origin main failed; refusing to clear stuck-lock until main is current." } - -$state = Read-State -$tier3 = [bool] $state.stuck_on_sha -$tier4 = [bool] $state.stuck_validation -if (-not ($tier3 -or $tier4)) { - Write-Warning "No stuck-lock is set on origin/main. Nothing to clear." - return -} - -$resolvedFullSha = if ($ResolvedThroughSha) { Resolve-FullCommitSha $ResolvedThroughSha } else { $null } - -if ($tier3) { - if (-not $resolvedFullSha) { - throw "Tier-3 stuck_on_sha is set ($($state.stuck_on_sha)) — -ResolvedThroughSha is required to clear it." - } - $null = git merge-base --is-ancestor $resolvedFullSha upstream/main - if ($LASTEXITCODE -ne 0) { - throw "ResolvedThroughSha $resolvedFullSha is not on upstream/main. Refusing to clear lock." - } - $null = git merge-base --is-ancestor $state.stuck_on_sha $resolvedFullSha - if ($LASTEXITCODE -ne 0) { - throw "stuck_on_sha $($state.stuck_on_sha) is not an ancestor of $resolvedFullSha. Refusing — pass the same SHA or a later one." - } - $state.last_synced_upstream_sha = $resolvedFullSha - $state.stuck_on_sha = $null - $state.stuck_branch = $null - $state.stuck_at = $null - $state.stuck_issue_url = $null -} - -if ($tier4) { - if ($resolvedFullSha) { - $null = git merge-base --is-ancestor $resolvedFullSha upstream/main - if ($LASTEXITCODE -ne 0) { - throw "ResolvedThroughSha $resolvedFullSha is not on upstream/main. Refusing to clear lock." - } - $state.last_synced_upstream_sha = $resolvedFullSha - } - $state.stuck_validation = $null -} - -# Append a history note so we can see when locks were cleared. -$entry = [ordered] @{ - at = Format-Iso8601 - host = $env:COMPUTERNAME - status = if ($tier3 -and $tier4) { 'cleared-stuck (tier3+tier4)' } - elseif ($tier3) { 'cleared-stuck (tier3)' } - else { 'cleared-stuck (tier4)' } - advanced_to = $resolvedFullSha - reason = $Reason -} -$state.history = @($entry) + @($state.history) | Select-Object -First 20 -Write-State $state - -git add -- (ConvertTo-RepoRelativePath (Get-StatePath)) | Out-Null -if ($LASTEXITCODE -ne 0) { throw "git add of state.json failed (LASTEXITCODE=$LASTEXITCODE); lock is NOT cleared on origin/main." } -$shortLabel = if ($resolvedFullSha) { $resolvedFullSha.Substring(0,9) } else { 'no-advance' } -git commit -m "chore(upstream-sync): clear stuck-lock ($shortLabel)" | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git commit failed (state unchanged?); lock is NOT cleared on origin/main." } -git push origin main | Out-Host -if ($LASTEXITCODE -ne 0) { throw "git push origin main failed — lock cleared locally only. Push manually." } -Write-Host "Stuck-lock cleared." -ForegroundColor Green diff --git a/.github/upstream-sync/.gitignore b/.github/upstream-sync/.gitignore deleted file mode 100644 index 727f42cca..000000000 --- a/.github/upstream-sync/.gitignore +++ /dev/null @@ -1,11 +0,0 @@ -build-logs/ - -# Local-only reports — see .github/skills/upstream-sync/references/reporting.md -# for the retention model. -# - 'ok' and 'stuck*' reports get committed (orchestrator stages them). -# - no-op / dry-run / skipped-* reports are diagnostic-only and stay -# local. Without ignoring them, the next scheduled run would fail at -# Assert-CleanWorktree because the worktree would be dirty. -reports/*-noop.md -reports/*-dry-run.md -reports/*-skipped.md diff --git a/.github/upstream-sync/reports/.gitkeep b/.github/upstream-sync/reports/.gitkeep deleted file mode 100644 index e69de29bb..000000000 diff --git a/.github/upstream-sync/state.json b/.github/upstream-sync/state.json deleted file mode 100644 index 49c68d382..000000000 --- a/.github/upstream-sync/state.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "version": 1, - "upstream_remote_url": "https://github.com/microsoft/terminal.git", - "upstream_branch": "main", - "last_synced_upstream_sha": "a325a2fa5a1cee7e46c23934fa6f82bc1922af3c", - "stuck_on_sha": null, - "stuck_branch": null, - "stuck_at": null, - "stuck_issue_url": null, - "stuck_validation": null, - "last_run": null, - "history": [] -} From f99c2c1fd0d5e9b83b63146d732c4c64b49cc899 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 15:21:59 +0800 Subject: [PATCH 51/82] fix(upstream-sync): address Copilot review round (15c6cbf14) - Common.ps1 Get-LastSyncedUpstreamSha: resolve full 40-char SHA via Resolve-FullCommitSha BEFORE running `git merge-base --is-ancestor`. An ambiguous or abbreviated trailer SHA could otherwise fail the ancestry probe and silently skip an otherwise-valid watermark. - Common.ps1 Get-StuckMetaFromIssue / Format-StuckYamlBlock: switch the regex split/replace patterns from "`r?`n" (double-quoted, valid but easily misread as a string literal containing `?`) to '\r?\n' (single-quoted regex). Functionally equivalent in PowerShell regex, but no longer looks like a bug to reviewers. - Common.ps1 New-RunContext: branch name now incorporates UTC HHmmss + 4 random hex chars in addition to the date. Same-day re-runs and the post-rebase-merge "branch not auto-deleted" edge case no longer collide on `upstream-sync/YYYY-MM-DD` and can no longer re-push already-merged commits onto a stale branch. - 05-write-report.ps1 / 07b-open-validation-stuck-issue.ps1: drop the "(pushed to origin)" claim from report/issue bodies - those lines are rendered BEFORE 07/07b actually run `git push`, and the push can fail. The new wording tells the operator to verify with `git ls-remote --heads origin `. - .github/actions/spelling/excludes.txt: exclude .github/skills/ from check-spelling entirely, per reviewer feedback that skill docs naturally contain technical identifiers the project-wide allow / expect lists shouldn't have to track. Drop the upstream-sync patterns that were only needed because of the now-excluded files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/excludes.txt | 1 + .../actions/spelling/patterns/patterns.txt | 5 ----- .../upstream-sync/scripts/05-write-report.ps1 | 4 ++-- .../07b-open-validation-stuck-issue.ps1 | 2 +- .../skills/upstream-sync/scripts/Common.ps1 | 22 ++++++++++++++----- 5 files changed, 20 insertions(+), 14 deletions(-) diff --git a/.github/actions/spelling/excludes.txt b/.github/actions/spelling/excludes.txt index 71ba80c66..ec37cdac1 100644 --- a/.github/actions/spelling/excludes.txt +++ b/.github/actions/spelling/excludes.txt @@ -126,6 +126,7 @@ Resources/(?!en) ^XamlStyler\.json$ ^\.clang-format$ ^\.github/actions/spelling/ +^\.github/skills/ ^\.github/workflows/spelling\d*\.yml$ ^\.vsconfig$ ^\Qbuild/config/release.gdnbaselines\E$ diff --git a/.github/actions/spelling/patterns/patterns.txt b/.github/actions/spelling/patterns/patterns.txt index e2dfb2b8b..e16ba9e70 100644 --- a/.github/actions/spelling/patterns/patterns.txt +++ b/.github/actions/spelling/patterns/patterns.txt @@ -320,8 +320,3 @@ env_remove\("[A-Z_]+"\) # GitHub GraphQL node ID prefix for PullRequestReviewThread \bPRRT_[A-Za-z0-9_-]+\b - -# upstream-sync skill: identifiers and timestamp placeholders -\b(?:ACMR|MFC)\b -\b(?:u(?:sha|pid)|quotepath|(?:required|available|missing)_[Tt]oolsets)\b -\b(?:(?:DD|dd)THH?(?:mm(?:ss)?|:mm:sszzz)|Hmmss|sszzz)\b diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 index 99b01e855..485ead8fd 100644 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ b/.github/skills/upstream-sync/scripts/05-write-report.ps1 @@ -160,7 +160,7 @@ if ($Status -eq 'stuck' -and $Ctx.StuckSha) { } $lines.Add("") } - $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") + $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (push attempted - run ````git ls-remote --heads origin $($Ctx.Branch)```` to verify it landed)") $lines.Add("") $lines.Add("**How to resume:**") $lines.Add("") @@ -235,7 +235,7 @@ if ($Status -like 'stuck-*') { $lines.Add("") } } - $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (pushed to origin)") + $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (push attempted - run ````git ls-remote --heads origin $($Ctx.Branch)```` to verify it landed)") $lines.Add("") if ($kind -eq 'toolchain-missing') { $lines.Add("**How to resume (infra-only - no PR needed):**") diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 index e283276fe..0b09818ca 100644 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 @@ -105,7 +105,7 @@ $header = @" > The scheduler will keep skipping its runs until this issue is **closed**. > Closing the issue IS the lock-clear signal - no separate script needed. -Sync branch: ``$($Ctx.Branch)`` (pushed to origin). +Sync branch: ``$($Ctx.Branch)`` (push attempted - run ``git ls-remote --heads origin $($Ctx.Branch)`` to verify it landed). Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). $yamlBlock diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 3dd0eea66..782c45d99 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -134,10 +134,16 @@ function Get-LastSyncedUpstreamSha { foreach ($c in $commits) { $body = git log -1 --format='%B' $c 2>$null if ($body -match '\(cherry picked from commit ([0-9a-f]{7,40})\)') { - $upstreamSha = $matches[1] - $null = git merge-base --is-ancestor $upstreamSha upstream/main 2>$null + $rawSha = $matches[1] + # Resolve to full 40-char SHA first so the ancestry check below + # works against a canonical object name (an abbreviated or ambiguous + # prefix could cause `git merge-base --is-ancestor` to fail and + # silently skip an otherwise-valid watermark candidate). + $fullSha = $null + try { $fullSha = Resolve-FullCommitSha $rawSha } catch { continue } + $null = git merge-base --is-ancestor $fullSha upstream/main 2>$null if ($LASTEXITCODE -eq 0) { - return (Resolve-FullCommitSha $upstreamSha) + return $fullSha } } } @@ -224,7 +230,7 @@ function Get-StuckMetaFromIssue { if ($Issue.body -notmatch $pattern) { return $null } $yaml = $matches[1] $h = [ordered] @{} - foreach ($l in $yaml -split "`r?`n") { + foreach ($l in $yaml -split '\r?\n') { if ($l -match "^\s*([a-z_][a-z0-9_]*)\s*:\s*'((?:[^']|'')*)'\s*$") { $h[$matches[1]] = $matches[2] -replace "''", "'" } elseif ($l -match '^\s*([a-z_][a-z0-9_]*)\s*:\s*(.+?)\s*$') { @@ -247,7 +253,7 @@ function Format-StuckYamlBlock { $lines = @('```yaml', $script:WtaStateFence) foreach ($k in $Fields.Keys) { $raw = "$($Fields[$k])" - $folded = $raw -replace "`r?`n", ' ' + $folded = $raw -replace '\r?\n', ' ' $escaped = $folded -replace "'", "''" $lines += ("{0}: '{1}'" -f $k, $escaped) } @@ -274,7 +280,11 @@ function New-RunContext { [pscustomobject] @{ StartedAt = Get-Date Host = $env:COMPUTERNAME - Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))" + # Branch name carries date + UTC timestamp + 4 random hex chars so + # repeated runs on the same day - or two consecutive runs after a + # rebase-merge that didn't auto-delete the previous branch - never + # check out a stale branch and replay already-merged commits. + Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))-$((Get-Date).ToUniversalTime().ToString('HHmmss'))-$(([guid]::NewGuid().ToString('N').Substring(0,4)))" Picked = @() Pending = @() DroppedPairs = @() From 2fc5f8e2cfd3a878ef82d48258c0d00d20db6ed6 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 15:36:08 +0800 Subject: [PATCH 52/82] docs(upstream-sync): sync branch-name docs with f99c2c1fd implementation Round 2 Copilot review caught documentation drift: round 1 changed the sync branch name to date+UTC-HHmmss+4-random-hex, but six doc sites still showed the old date-only form (`upstream-sync/`), which would lead operators to a 404 when they tried to copy/paste the branch name. - SKILL.md prerequisite: documents `upstream-sync/*` push permission and the per-run name format with an example. - SKILL.md follow-up-PR example: uses a realistic per-run branch name and tells operators to copy it from the sync PR, not derive from the date. - references/workflow.md step 5: shows the actual New-RunContext branch-name expression and removes the "reuse if same-day" hint. - references/workflow.md stuck-resume step: points operators at the exact branch name in the stuck-issue body. - references/follow-up-pr.md example: updated $syncBranch with the per-run format + explanatory comment. - 04-run-batch.ps1 file header: matches actual branch naming. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 4 ++-- .../skills/upstream-sync/references/follow-up-pr.md | 2 +- .github/skills/upstream-sync/references/workflow.md | 13 +++++++++---- .../skills/upstream-sync/scripts/04-run-batch.ps1 | 2 +- 4 files changed, 13 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index a36c4b4b1..0dc0cd150 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -27,7 +27,7 @@ conflict appears. ## Prerequisites -- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches** (`upstream-sync/`) and **issue + label create** in the same repo (the stuck-lock is an open labeled issue, not a commit on `main`). +- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches matching `upstream-sync/*`** (the orchestrator uses `upstream-sync/--` per run, e.g. `upstream-sync/2026-06-04-091512-a3f1`, so a fresh branch lands every time) and **issue + label create** in the same repo (the stuck-lock is an open labeled issue, not a commit on `main`). - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). - Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` @@ -195,7 +195,7 @@ attribution the cherry-pick approach was chosen to preserve. - New worktree + branch `dev//sync--review-fixes` off the sync PR's HEAD (see [branch-worktree-workflow](./references/follow-up-pr.md#worktree-setup)). -- Base = the sync branch (e.g. `upstream-sync/2026-06-04`), **not** +- Base = the sync branch (e.g. `upstream-sync/2026-06-04-091512-a3f1` — copy the exact name from the sync PR, since each run uses a fresh date+timestamp+random suffix), **not** `main`. The follow-up rides along with the sync PR. - One focused commit per concern (code-bugs / translations / spelling-cleanup / etc.) — same "audit trail per finding" rule as the diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index 06c693e42..521cb71be 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -48,7 +48,7 @@ follow-up so the main worktree stays clean: ```pwsh $syncPr = 220 -$syncBranch = 'upstream-sync/2026-06-04' # the sync PR's head +$syncBranch = 'upstream-sync/2026-06-04-091512-a3f1' # the sync PR's head — copy the exact name from the PR, branches are date+UTC-HHmmss+random-hex per run $fixBranch = "dev/$env:USERNAME/sync-$syncPr-review-fixes" $fixWorktree = "..\it-$syncPr`fix" diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md index 330ed68c3..ac7fc82ac 100644 --- a/.github/skills/upstream-sync/references/workflow.md +++ b/.github/skills/upstream-sync/references/workflow.md @@ -74,11 +74,15 @@ pick and reset.) ### 5. Create the sync branch ```pwsh -$branch = "upstream-sync/$(Get-Date -Format 'yyyy-MM-dd')" -git switch -c $branch # or "git switch $branch" if resuming +# Branch name is per-run, never reused: date + UTC HHmmss + 4 random hex chars. +# This guarantees a fresh branch for every run (no risk of replaying onto a +# stale branch that GitHub didn't auto-delete after a rebase-merge). +$branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))-$((Get-Date).ToUniversalTime().ToString('HHmmss'))-$(([guid]::NewGuid().ToString('N').Substring(0,4)))" +git switch -c $branch ``` -If the branch already exists (resume from same-day run), reuse it. +Resume = pick up the branch name from the run report or the open +`upstream-sync-stuck` issue body, not by deriving from the date. ### 6. Cherry-pick loop @@ -279,7 +283,8 @@ issue, the orchestrator: To clear the lock after the human has resolved the underlying issue: ``` -1. Resolve the conflict on the stuck branch (`upstream-sync/`), +1. Resolve the conflict on the stuck branch (the exact name is in the + stuck-issue body, e.g. `upstream-sync/2026-06-04-091512-a3f1`), keeping every `(cherry picked from commit )` trailer intact. 2. Open a PR for the fix, merge it (rebase or merge — NOT squash). 3. CLOSE the stuck issue. That's the lock-clear signal — no script. diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 index b45f14d1a..3ed563b6d 100644 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 @@ -13,7 +13,7 @@ with a `skipped-locked` report and exits 0. Otherwise: 1. Fetches upstream/main. 2. Computes pending commits, dropping revert pairs and empties. - 3. Creates branch upstream-sync/YYYY-MM-DD. + 3. Creates a fresh sync branch upstream-sync/-- (per-run; no reuse). 4. Cherry-picks one-by-one with Tier-0/Tier-1 auto-resolution. On cherry-pick conflict -> Tier-3 stuck path (07). 5. Post-batch HARD GATES (in order, before any push/PR): From 38b831768287bf522cc9bf3c5ee9973b4cd878d0 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 15:59:04 +0800 Subject: [PATCH 53/82] restructure: orchestration moves to SKILL.md, scripts become atomic ops MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses 7 inline review comments on PR #218 asking for a deeper restructure of the upstream-sync skill: * Delete 04-run-batch.ps1: the orchestration loop is now numbered steps in SKILL.md that the agent executes directly. Each script is one atomic operation; the agent (LLM) reads JSON output and decides what to do next. * Delete 05-write-report.ps1, 08-static-scan.ps1, 09-toolchain-preflight.ps1 and references/static-scan.md + references/fork-invariants.json. No reports; PR review catches the issues those scans were guessing at; toolchain provisioning is the operator's problem (surfaces as a Tier-4 build failure naturally). * Reorder build BEFORE finalize. The new flow is fetch -> compute -> pick-loop -> build -> finalize. If the build fails, a single focused fix commit lands on the same branch and ships in the same PR. * Simplify Common.ps1 to only the two helpers used by 2+ callers (Format-StuckYamlBlock, Format-Iso8601). All single-use helpers inlined into their one caller with a labeled comment block. * Drop \ context object from 05/06/06b — each takes explicit scalar params so the agent can call them straight from SKILL.md without constructing a context. * Tier-4 collapses to build-failed + build-inconclusive (no more static-scan / toolchain-missing sub-tiers). * Rename references/*.md with step-number prefixes so the reference ordering matches the script ordering: 03-conflict-triage.md, 03-known-conflicts.md, 04-build-verification.md. * Delete redundant references/workflow.md (now duplicates SKILL.md). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 429 +++++++++++------- ...nflict-triage.md => 03-conflict-triage.md} | 52 +-- ...own-conflicts.md => 03-known-conflicts.md} | 0 .../references/04-build-verification.md | 85 ++++ .../references/build-verification.md | 104 ----- .../upstream-sync/references/follow-up-pr.md | 2 +- .../references/fork-invariants.json | 15 - .../upstream-sync/references/static-scan.md | 135 ------ .../upstream-sync/references/workflow.md | 327 ------------- .../scripts/01-fetch-upstream.ps1 | 15 +- .../scripts/02-compute-pending.ps1 | 105 +++-- .../scripts/03-cherry-pick-one.ps1 | 2 +- .../upstream-sync/scripts/04-run-batch.ps1 | 359 --------------- .../{10-try-build.ps1 => 04-try-build.ps1} | 97 ++-- .../upstream-sync/scripts/05-finalize-pr.ps1 | 111 +++++ .../upstream-sync/scripts/05-write-report.ps1 | 277 ----------- .../upstream-sync/scripts/06-finalize-pr.ps1 | 121 ----- .../scripts/06-open-stuck-issue.ps1 | 116 +++++ .../scripts/06b-open-build-stuck-issue.ps1 | 141 ++++++ .../scripts/07-open-stuck-issue.ps1 | 105 ----- .../07b-open-validation-stuck-issue.ps1 | 147 ------ .../upstream-sync/scripts/08-static-scan.ps1 | 195 -------- .../scripts/09-toolchain-preflight.ps1 | 141 ------ .../skills/upstream-sync/scripts/Common.ps1 | 308 +------------ 24 files changed, 900 insertions(+), 2489 deletions(-) rename .github/skills/upstream-sync/references/{conflict-triage.md => 03-conflict-triage.md} (72%) rename .github/skills/upstream-sync/references/{known-conflicts.md => 03-known-conflicts.md} (100%) create mode 100644 .github/skills/upstream-sync/references/04-build-verification.md delete mode 100644 .github/skills/upstream-sync/references/build-verification.md delete mode 100644 .github/skills/upstream-sync/references/fork-invariants.json delete mode 100644 .github/skills/upstream-sync/references/static-scan.md delete mode 100644 .github/skills/upstream-sync/references/workflow.md delete mode 100644 .github/skills/upstream-sync/scripts/04-run-batch.ps1 rename .github/skills/upstream-sync/scripts/{10-try-build.ps1 => 04-try-build.ps1} (57%) create mode 100644 .github/skills/upstream-sync/scripts/05-finalize-pr.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/05-write-report.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/06-finalize-pr.ps1 create mode 100644 .github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 create mode 100644 .github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/08-static-scan.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 0dc0cd150..a84b23d5b 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -1,6 +1,6 @@ --- name: upstream-sync -description: 'Periodically sync new commits from microsoft/terminal into this manually-forked intelligent-terminal repo by cherry-picking commit-by-commit onto a dated sync branch, auto-skipping revert pairs and empty commits, auto-resolving known take-upstream files, and stopping cleanly on genuine conflicts with a written report and a GitHub issue. Use when the user asks to "sync upstream", "pull from microsoft/terminal", "run upstream sync", "catch up to upstream", or wires this into a scheduler (weekly/daily). Designed to be safe under repeated unattended runs.' +description: 'Periodically sync new commits from microsoft/terminal into this manually-forked intelligent-terminal repo by cherry-picking commit-by-commit onto a dated sync branch, auto-skipping revert pairs and empty commits, auto-resolving known take-upstream files, and stopping cleanly on genuine conflicts. The agent (you, reading this file) is the orchestrator — the PowerShell scripts are atomic operations you invoke one at a time. Use when the user asks to "sync upstream", "pull from microsoft/terminal", "run upstream sync", "catch up to upstream", or wires this into a scheduler (weekly/daily). Designed to be safe under repeated unattended runs.' license: MIT --- @@ -11,11 +11,18 @@ into this fork, preserving per-commit attribution, skipping commits that cancel each other out, and stopping cleanly the moment a human-judgment conflict appears. +**You — the agent reading this file — are the orchestrator.** Each step +below is a single atomic call into one of the `scripts/*.ps1` files. Run +them in order, parse their JSON output, and decide where to branch based +on the result. There is intentionally no PowerShell driver that calls +them for you, because every interesting decision (build-fix vs. open +stuck issue, retry vs. bail, finalize vs. dry-run) wants LLM judgment +the operator can audit in your transcript. + ## When to Use This Skill - User asks to "sync upstream", "pull from microsoft/terminal", "catch up to upstream", or "run upstream sync". -- A scheduler (Task Scheduler, cron, GitHub Actions) invokes - [`scripts/04-run-batch.ps1`](./scripts/04-run-batch.ps1) on a weekly/daily cadence. +- A scheduler invokes the agent on a weekly/daily cadence to walk this file's [Run a sync](#run-a-sync) procedure. - The previous run left an `upstream-sync-stuck` labeled issue open and the human has finished resolving the conflict — **close the issue** (that IS the lock-clear signal) and re-run. @@ -27,149 +34,271 @@ conflict appears. ## Prerequisites -- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`, used by `scripts/03-cherry-pick-one.ps1`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches matching `upstream-sync/*`** (the orchestrator uses `upstream-sync/--` per run, e.g. `upstream-sync/2026-06-04-091512-a3f1`, so a fresh branch lands every time) and **issue + label create** in the same repo (the stuck-lock is an open labeled issue, not a commit on `main`). +- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches matching `upstream-sync/*`** and **issue + label create** on the same repo. - PowerShell 7+ (`pwsh`) on PATH. -- Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment for the default validation gates (or use `-SkipBuild` only for explicit dev/debug runs). -- Remote named `upstream` pointing at `https://github.com/microsoft/terminal.git` - (the scripts create it if missing). +- Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment (build is a hard gate before finalize — see [step 7](#7-build)). +- Remote named `upstream` — the scripts create it if missing. - **No `state.json` to bootstrap.** Watermark comes from the `(cherry picked from commit )` trailers on `origin/main`. If the fork has never used `cherry-pick -x` (or trailers were stripped), - see "First-time sync" below for the one-time operator step. + see [First-time sync](#first-time-sync) below for the one-time operator step. -## State model (no `state.json`) +## State Model (no state file) Every persistent fact lives in the source that owns it: | Question | Source of truth | |---|---| -| What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Computed by `Get-LastSyncedUpstreamSha` in `scripts/Common.ps1`. | +| What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived inline by [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | | What's pending? | `git log --cherry-pick --right-only --no-merges ...upstream/main`. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | | Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | -| What does the lock mean? | A fenced ```yaml # wta-state``` block in the issue body carries `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. (parsed by `Get-StuckMetaFromIssue`). | -| Where do reports + build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | +| What does the lock mean? | A fenced ```yaml # wta-state``` block in the issue body carries `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. | +| Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | -### First-time sync +### Why cherry-pick (and not rebase or merge) -If the fork has no `(cherry picked from commit )` trailer on -`origin/main` yet, `Get-LastSyncedUpstreamSha` will throw. To seed, -the operator commits an **empty seed commit** carrying the trailer -in its message — the same format `cherry-pick -x` would emit, written -by hand exactly once: +| Approach | Why rejected / chosen | +|---|---| +| **Rebase** `upstream/main` | ❌ Fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts. Verified failure on sister repo `agentic-terminal`. | +| **Merge** `upstream/main` | ⚠️ Works, but collapses the whole sync into one blob commit — kills per-commit review, kills `git bisect`. | +| **Cherry-pick commit-by-commit** | ✅ Preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR with N small commits. | + +## Run a sync + +This is the orchestration you (the agent) execute. **Do not skip steps.** +Each step's "On failure" path is mandatory. + +### 1. Preconditions (bail fast) ```pwsh -git remote add upstream https://github.com/microsoft/terminal.git -git fetch upstream main --no-tags +# (a) working tree clean +git status --porcelain +# → empty? continue. nonempty? bail and tell the operator. + +# (b) on main, fast-forward git switch main -# Pick an upstream SHA you consider "already in this fork" (typically the -# commit the fork was originally branched from). Write the trailer EXACTLY -# as cherry-pick -x would so Get-LastSyncedUpstreamSha picks it up. -$seedSha = '<40-char-upstream-sha-already-in-fork>' -git commit --allow-empty -m "chore(upstream): seed upstream-sync watermark" -m "(cherry picked from commit $seedSha)" -git push origin main +git pull --ff-only + +# (c) no open stuck-lock +gh issue list -R microsoft/intelligent-terminal --label upstream-sync-stuck --state open --json number,title,url +# → []? continue. +# → nonempty? STOP. Surface the issue URL(s) to the operator and exit. +# The lock-clear signal is the human closing the issue, not a script. ``` -That one merged trailer becomes the watermark. From then on, every -run extends it. No script needed — the bootstrap is just one -`commit --allow-empty` ever. +### 2. Build a branch name -### Squash-merge recovery (don't do this, but if you did) +```pwsh +$date = (Get-Date).ToString('yyyy-MM-dd') +$tstamp = (Get-Date).ToUniversalTime().ToString('HHmmss') +$rand = [guid]::NewGuid().ToString('N').Substring(0,4) +$branch = "upstream-sync/$date-$tstamp-$rand" +git switch -c $branch +``` -The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` -locks in the safe path. If a reviewer squash-merges anyway: +The fresh suffix means re-runs never collide with stale local branches. -- The squash commit's body usually concatenates every cherry-picked - message, so it contains MANY `(cherry picked from commit )` - trailers. `Get-LastSyncedUpstreamSha` matches the FIRST one it sees - via regex, which is the oldest cherry-pick in the squashed batch. - That means the watermark moves backward, and the next sync run - re-picks everything between the oldest and newest commits of the - squashed batch. -- The recovery is the same as a first-time seed: commit an empty - watermark commit on `main` carrying the trailer for the upstream - HEAD that was actually merged, and push. +### 3. Fetch upstream +```pwsh +$upstreamSha = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +``` +Returns the `upstream/main` SHA. Creates the `upstream` remote if missing. -| Approach | Why rejected / chosen | -|---|---| -| **Rebase** `upstream/main` | ❌ Fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts. Verified failure on sister repo `agentic-terminal`. | -| **Merge** `upstream/main` | ⚠️ Works, but collapses the whole sync into one blob commit — kills per-commit review, kills `git bisect`. | -| **Cherry-pick commit-by-commit** | ✅ Preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR with N small commits. | +### 4. Compute pending + +```pwsh +$pendingJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/02-compute-pending.ps1 +$pending = $pendingJson | ConvertFrom-Json +# $pending.from = last-synced watermark SHA +# $pending.to = upstream/main SHA +# $pending.pending = SHAs to pick, oldest-first +# $pending.dropped_pairs = [[picked, reverted], ...] — auto-cancelled +# $pending.skipped_empty = SHAs the script can statically prove are no-ops +``` + +If `$pending.pending.Count -eq 0`: nothing to do. Delete the branch, tell +the operator "fork is up to date with ", exit. -## Step-by-Step Workflow +### 5. Cherry-pick loop -The scheduler entrypoint is a single PowerShell script. Full procedure -with commands, exit codes, and the per-step delegation map lives in -[references/workflow.md](./references/workflow.md). +For each SHA in `$pending.pending`: +```pwsh +$pickJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 -Sha $sha +$pick = $pickJson | ConvertFrom-Json ``` -fetch upstream → compute pending → drop revert pairs → drop empties - → create sync branch → cherry-pick loop (auto-resolve T0/T1, abort on T3) - → toolchain preflight → static breakage scan → try-build (Tier-4 gates) - → write report (always) → push + open PR OR open stuck issue + lock + +Branch on `$pick.status`: + +- `"applied"` or `"auto-resolved"` — record `$sha` in `$picked`, continue. +- `"empty"` — record `$sha` in `$skippedEmpty`, continue. +- `"stuck"` — **stop the loop** and go to step 5a. + +#### 5a. On stuck — open the Tier-3 issue and exit + +```pwsh +$issueUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 ` + -Branch $branch ` + -StuckSha $sha ` + -ConflictPaths $pick.conflict_paths ` + -StuckError $pick.error ``` -**Safety guarantees (post-pick validation pipeline).** Even when every -cherry-pick applies cleanly, content-level breakage can slip in (duplicate -`.resw` keys from a fork-local commit + an upstream rename touching the -same names; a take-upstream resolution silently dropping a fork-specific -warning suppression; etc. — see PR #220 audit). Before any push or PR, -the orchestrator now runs three hard gates: +Surface `$issueUrl` and `$branch` to the operator. The human is expected to +check out the branch, resolve the conflict, push it, open a PR, merge it +(keeping the `(cherry picked from commit )` trailer!), then close +the issue. EXIT — do not attempt the build. + +See [references/03-conflict-triage.md](./references/03-conflict-triage.md) +for what "Tier-3" means and the resolution rubric. -1. **Toolchain preflight** ([`scripts/09-toolchain-preflight.ps1`](./scripts/09-toolchain-preflight.ps1)) — verifies the host has the `PlatformToolset` versions the repo requires. Missing → **Tier-4d infra-stuck**: NO GitHub issue is opened and NO lock is set (PR review can't fix host provisioning). The next scheduler tick simply retries; if this host keeps tripping it, run from a properly provisioned host instead. -2. **Static breakage scan** ([`scripts/08-static-scan.ps1`](./scripts/08-static-scan.ps1)) — baseline-diffs `.resw` files for newly-duplicated `` keys, and regex-checks fork invariants from [`references/fork-invariants.json`](./references/fork-invariants.json). Blocking → **Tier-4a stuck**. -3. **Try-build** ([`scripts/10-try-build.ps1`](./scripts/10-try-build.ps1)) — runs `tools\razzle.cmd && bz no_clean` with a 45-minute wall-clock cap. Build failed → **Tier-4b stuck**; timeout → **Tier-4c stuck** (unless `-AllowInconclusiveBuild`). +### 6. (No commits picked? exit clean.) -See [references/static-scan.md](./references/static-scan.md), [references/build-verification.md](./references/build-verification.md), and [references/conflict-triage.md](./references/conflict-triage.md) Tier-4 for details. +If the loop emitted only `empty` results (rare — every pending commit was +a no-op) there is nothing to build or finalize. Delete the branch, report +"sync completed with 0 commits picked (all empty)", exit. -**Run it:** +### 7. Build ```pwsh -# Default: open a PR, let a human pick the merge strategy (must be rebase or merge — NOT squash) -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 +$buildJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/04-try-build.ps1 +$build = $buildJson | ConvertFrom-Json +``` -# Open a PR AND arm GitHub auto-merge with rebase strategy (hands-off once CI/approvals pass) -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -AutoMergeStrategy rebase +Build runs BEFORE finalize on purpose — if it fails, the fix lands as ONE +extra commit on the same branch so it ends up in the same PR. + +Branch on `$build.kind`: + +- `"build-ok"` — continue to step 8. +- `"build-failed"` — try ONE focused build-fix: + 1. Read `$build.log_tail`. Decide if the failure is small, mechanical, and + clearly caused by the cherry-pick batch (e.g. duplicate `.resw` key + from a fork-local commit colliding with an upstream rename, missing + `#include` resolved by the upstream batch, etc.). + 2. If yes — fix it, `git add` only the affected files, commit with + subject `chore(upstream): fix build after upstream sync` (carry the + `Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>` + trailer if you authored the fix), re-run `04-try-build.ps1`, go back + to start of step 7. + 3. If no (large change, scope creep, requires design decisions) — go to step 7a. + 4. If the fix attempt itself fails to compile — go to step 7a. Do NOT + pile up multiple fix commits; the one-fix-per-PR rule is policy. +- `"build-inconclusive"` — go to step 7a (timeout, treated as a real failure + unless the operator explicitly opts out). + +#### 7a. On build failure — open the Tier-4 issue and exit -# Skip the PR entirely — fast-forward main to the sync tip. Requires admin/bypass on main. -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -PushDirectToMain +```pwsh +$issueUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 ` + -Branch $branch ` + -Kind $build.kind ` + -PickedCount $picked.Count ` + -BuildExitCode $build.exit_code ` + -BuildLogTail $build.log_tail ` + -BuildLogPath $build.log_path +``` + +The branch is pushed by the script. Surface `$issueUrl` to the operator +and EXIT. + +### 8. Finalize the PR + +Build a concise PR body (just YOU, the agent, composing markdown — there +is no report file to inline): + +```pwsh +$body = @" +Syncs **$($picked.Count)** commit(s) from microsoft/terminal up to ``$($upstreamSha.Substring(0,9))``. + +## Picked +$( ($picked | ForEach-Object { "- ``$($_.Substring(0,9))`` $(git log -1 --format='%s' $_)" }) -join "`n" ) + +$(if ($pending.dropped_pairs.Count) { @" +## Auto-dropped revert pairs +$( ($pending.dropped_pairs | ForEach-Object { "- ``$($_[0].Substring(0,9))`` ↔ ``$($_[1].Substring(0,9))`` (cancel out within range)" }) -join "`n" ) +"@ }) + +$(if ($skippedEmpty.Count) { @" +## Skipped (empty / already applied) +$( ($skippedEmpty | ForEach-Object { "- ``$($_.Substring(0,9))``" }) -join "`n" ) +"@ }) + +## Build +``04-try-build.ps1`` → ``$($build.kind)`` (exit $($build.exit_code), $($build.duration_ms) ms). Log: ``$($build.log_path)`` (gitignored). +"@ + +$prUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/05-finalize-pr.ps1 ` + -Branch $branch ` + -UpstreamHeadSha $upstreamSha ` + -PickedCount $picked.Count ` + -PrBody $body ` + -AutoMergeStrategy 'none' # or 'rebase' / 'merge' for hands-off; NEVER squash +``` -# Compute & report without picking -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -DryRun +Pass `-AutoMergeStrategy rebase` if the operator wants GitHub to merge the +PR automatically once CI + approvals pass. **Never** pass `'squash'` — +the script doesn't accept it and the PR body's banner shouts about it. -# Skip the static scan (debugging only — schedulers must run it) -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -SkipStaticScan +Surface `$prUrl` to the operator. Done. -# Skip the try-build (debugging only — schedulers must run it). -# Also skips toolchain preflight since they share the same infra prerequisite. -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -SkipBuild +### Direct-to-main (admin-only escape hatch) -# Don't treat a build timeout as Tier-4 stuck (dev opt-in; never in a scheduler) -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -AllowInconclusiveBuild +If the operator explicitly says "skip the PR, push straight to main": after +step 7 succeeds, skip step 8 and instead: -# Override build timeout (default 45 minutes) -pwsh .github/skills/upstream-sync/scripts/04-run-batch.ps1 -BuildTimeoutMinutes 60 +```pwsh +git switch main +git merge --ff-only $branch +git push origin main +git branch -D $branch ``` -### Finalize modes — what each preserves +This requires bypass-branch-protection rights. No PR, no review checkpoint — +use only for explicit admin runs. -| Mode | Per-commit content | Order on main | Original author + committer dates | Reviewer checkpoint | Requires admin? | -|---|---|---|---|---|---| -| PR + **rebase-merge** | ✅ | ✅ | ✅ | ✅ | No | -| PR + **merge commit** | ✅ | ✅ | ✅ | ✅ | No | -| PR + **squash** | ❌ collapsed | ❌ | ⚠️ folded | ✅ | No | -| **`-PushDirectToMain`** | ✅ | ✅ | ✅ | ❌ | Yes (push to main) | +### First-time sync -The cherry-pick loop pins both author and committer identity/dates to the -upstream commit so audit timestamps match the original commit-by-commit history. +If the fork has no `(cherry picked from commit )` trailer on +`origin/main` yet, `02-compute-pending.ps1` will throw. To seed, +the operator commits an **empty seed commit** carrying the trailer +in its message — the same format `cherry-pick -x` would emit, written +by hand exactly once: -Resumability is built into the trailer model — re-running after a successful -run is a fast no-op (the new merged commits extend the watermark, so the -pending-list is empty), and re-running while a stuck-issue is open exits -early without touching the branch. +```pwsh +git remote add upstream https://github.com/microsoft/terminal.git +git fetch upstream main --no-tags +git switch main +# Pick an upstream SHA you consider "already in this fork" (typically the +# commit the fork was originally branched from). Write the trailer EXACTLY +# as cherry-pick -x would so the next sync picks it up. +$seedSha = '<40-char-upstream-sha-already-in-fork>' +git commit --allow-empty -m "chore(upstream): seed upstream-sync watermark" -m "(cherry picked from commit $seedSha)" +git push origin main +``` + +That one merged trailer becomes the watermark. From then on, every +run extends it. No script needed — the bootstrap is just one +`commit --allow-empty` ever. + +### Squash-merge recovery (don't do this, but if you did) -### After-PR review handling — fix-in-PR vs. follow-up PR +The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` +locks in the safe path. If a reviewer squash-merges anyway: + +- The squash commit's body usually concatenates every cherry-picked + message, so it contains MANY `(cherry picked from commit )` + trailers. The watermark resolver matches the FIRST one it sees, + which is the oldest cherry-pick in the squashed batch. That means + the watermark moves backward, and the next sync run re-picks + everything between the oldest and newest commits of the squashed batch. +- The recovery is the same as a first-time seed: commit an empty + watermark commit on `main` carrying the trailer for the upstream + HEAD that was actually merged, and push. + +## After-PR review handling — fix-in-PR vs. follow-up PR Once the sync PR is open, Copilot and human reviewers will leave comments. The cherry-pick PR's commits must stay reviewable as @@ -179,7 +308,7 @@ That constraint shapes the response policy: | Comment type | Where to fix | |---|---| -| **Build-blocking** (compile error, dedup of resw/manifest collisions exposed only at build time, CI gate failure on the sync PR itself) | **One** focused extra commit on the sync branch. The cherry-pick PR is what's broken; the cherry-pick PR is what gets the fix. | +| **Build-blocking** (compile error, dedup of resw/manifest collisions exposed only at build time, CI gate failure on the sync PR itself) | **One** focused extra commit on the sync branch. The cherry-pick PR is what's broken; the cherry-pick PR is what gets the fix. The build-then-finalize order in this file already lands this commit in the same PR as the picks. | | **Everything else** — code-quality findings, logic-bug suggestions, translation corrections, spelling-allowlist migrations, typo fixes, doc nits, design feedback | **Follow-up PR** on top of the cherry-pick PR (see [references/follow-up-pr.md](./references/follow-up-pr.md)) | **Why split.** A reviewer scanning the cherry-pick PR is auditing @@ -193,10 +322,10 @@ attribution the cherry-pick approach was chosen to preserve. [references/follow-up-pr.md](./references/follow-up-pr.md)): - New worktree + branch `dev//sync--review-fixes` - off the sync PR's HEAD (see - [branch-worktree-workflow](./references/follow-up-pr.md#worktree-setup)). -- Base = the sync branch (e.g. `upstream-sync/2026-06-04-091512-a3f1` — copy the exact name from the sync PR, since each run uses a fresh date+timestamp+random suffix), **not** - `main`. The follow-up rides along with the sync PR. + off the sync PR's HEAD. +- Base = the sync branch (e.g. `upstream-sync/2026-06-04-091512-a3f1` — + copy the exact name from the sync PR), **not** `main`. The follow-up + rides along with the sync PR. - One focused commit per concern (code-bugs / translations / spelling-cleanup / etc.) — same "audit trail per finding" rule as the Copilot PR review loop skill. @@ -205,92 +334,76 @@ attribution the cherry-pick approach was chosen to preserve. - If the sync PR merges first, rebase the follow-up onto `main` before it merges. -The orchestrator's PR banner ([scripts/06-finalize-pr.ps1](./scripts/06-finalize-pr.ps1)) +The orchestrator's PR banner ([scripts/05-finalize-pr.ps1](./scripts/05-finalize-pr.ps1)) spells this policy out to the first reviewer so they don't push back on deferred fixes. ## Gotchas -- **Never squash-merge the sync PR.** Squash collapses every cherry-picked - upstream commit into one, destroying per-commit attribution, original - author dates, and `git bisect` resolution. Use **"Rebase and merge"** +- **Never squash-merge the sync PR.** Use **"Rebase and merge"** (preferred) or **"Create a merge commit"**. The PR body opens with a banner reminding the reviewer; `-AutoMergeStrategy rebase` arms GitHub - auto-merge with the right strategy so a tired reviewer can't get it - wrong. -- **Don't amend review fixes into the sync PR.** Only build-blocking - fixes (compile errors, dedup of conflicts that surface at build time, - CI gate failures on the sync PR itself) get **one** extra commit on - the sync branch. Substantive Copilot/human review feedback — - code-quality, logic, translations, spelling-list migrations, doc nits - — goes into a separate follow-up PR off the sync branch's HEAD. See - [references/follow-up-pr.md](./references/follow-up-pr.md). Mixing - them poisons the "faithful to upstream" audit of the cherry-pick PR. -- **Never rebase `upstream/main` onto this fork.** Old "Merge upstream" - commits in the fork history replay and cascade conflicts. Use cherry-pick. + auto-merge with the right strategy so a tired reviewer can't get it wrong. +- **Don't amend substantive review fixes into the sync PR.** Only + build-blocking fixes get **one** extra commit on the sync branch. See + [references/follow-up-pr.md](./references/follow-up-pr.md). +- **Never rebase `upstream/main` onto this fork.** Use cherry-pick. Verified failure mode on the sister repo `agentic-terminal`. - **`.github/workflows/spelling2.yml` always conflicts** and the correct resolution is always "take upstream wholesale". The Tier-0 list in - [references/known-conflicts.md](./references/known-conflicts.md) handles - this automatically — extend the list when you discover the next file - with the same pattern. + [references/03-known-conflicts.md](./references/03-known-conflicts.md) + handles this automatically — extend the list when you discover the + next file with the same pattern. - **`gh pr create` on Windows can fail with "Head sha can't be blank"** if the - branch is freshly pushed and not yet visible. The same-repo finalize script - intentionally uses `--head ` plus a 5s retry — do not "fix" it to - `--head :`, which would point `gh` at a fork. -- **Do not run the scheduler twice while a stuck issue is open.** The - open labeled issue makes the second run a no-op, but a human running - the script manually with `-Force` will overwrite the stuck branch and - lose their in-progress resolution. The `-Force` flag is documented but - intentionally not the default. + branch is freshly pushed and not yet visible. `05-finalize-pr.ps1` + retries 3× — do not "fix" the script to use `--head :` + (which would point `gh` at a fork). +- **Do not run the orchestration twice while a stuck issue is open.** The + step-1 preflight catches it, but a human bypassing that gate manually + would overwrite the stuck branch and lose their in-progress resolution. - **Cherry-pick over `git revert` style commits is intentional, not skipped.** We only skip revert-pairs where **both** sides are inside the pending range. A revert of a commit we already merged last week must land as a normal pick — otherwise the fork diverges silently. - **Never strip the `(cherry picked from commit )` trailer** when hand-resolving a stuck pick. That trailer IS the watermark the next - sync run reads. A squash-merge or `--no-commit` workflow that drops - the trailer breaks the next sync as effectively as deleting a - `state.json` used to. -- **Reports and build logs live under `Generated Files/upstream-sync//`.** + sync run reads. +- **Build logs live under `Generated Files/upstream-sync//`.** This directory is gitignored at the repo root (`**/Generated Files/`). - Do not check these in — they're transient diagnostics, and the issue - body inlines the parts that matter for review. -- **Prefer the PR path.** The default workflow always opens a PR so CI and - human review checkpoint the upstream batch. Use `-PushDirectToMain` only - for an explicit admin/bypass run where skipping PR latency is intentional. -- **CRLF/LF on manifest files.** Cherry-picks normally preserve upstream - line endings, but any in-flight resolution touched by an LLM may downgrade - to LF. If a Tier-2 resolution touches a `.yml`/`.xml`/`.csproj`/winget - manifest, re-normalize before staging — see - [references/conflict-triage.md](./references/conflict-triage.md#line-endings). - + Do not check these in. - **Single-host scheduler.** The stuck-lock (open labeled issue) is a read-then-check gate, not an atomic lease — two hosts running on the same tick can both observe "no open issue" and proceed in parallel. Run the scheduler from ONE host. For multi-host fan-out, layer atomic locking on top (GitHub Actions `concurrency: upstream-sync` group is the easiest). +- **CRLF/LF on manifest files.** Cherry-picks normally preserve upstream + line endings, but any in-flight resolution touched by an LLM may + downgrade to LF. If a Tier-2 resolution touches a + `.yml`/`.xml`/`.csproj`/winget manifest, re-normalize before staging — + see [references/03-conflict-triage.md](./references/03-conflict-triage.md#line-endings). ## Troubleshooting | Issue | Solution | |---|---| -| `Get-LastSyncedUpstreamSha` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [State model — First-time sync](#first-time-sync). | +| `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [First-time sync](#first-time-sync). | | Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | -| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; the loop auto-resets and marks them skipped. No action needed. | -| Same file conflicts every run | Add it to the Tier-0 list in [references/known-conflicts.md](./references/known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | -| `gh pr create` returns "Head sha can't be blank" | Retry — the finalize script already does, but on slow networks may need a manual second run. | -| Report says "no-op" but I expected commits | Run `git fetch upstream main` manually and recompute — the scheduler may have run between upstream pushes. | +| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"empty"` and the agent's loop skips it. No action needed. | +| Same file conflicts every run | Add it to the Tier-0 list in [references/03-known-conflicts.md](./references/03-known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | +| `gh pr create` returns "Head sha can't be blank" | `05-finalize-pr.ps1` retries 3× automatically. On slow networks may need a manual second run. | ## References -- [references/workflow.md](./references/workflow.md) — full per-step procedure with exit codes and delegation map. -- [references/conflict-triage.md](./references/conflict-triage.md) — Tier 0/1/2/3/4 resolution rubric with examples. -- [references/known-conflicts.md](./references/known-conflicts.md) — files that always need a fixed resolution. -- [references/static-scan.md](./references/static-scan.md) — post-pick static breakage scan rules. -- [references/fork-invariants.json](./references/fork-invariants.json) — fork-specific patterns that must survive any upstream pick. -- [references/build-verification.md](./references/build-verification.md) — try-build pipeline + toolchain preflight policy. -- [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow for handling post-PR review. -- [scripts/04-run-batch.ps1](./scripts/04-run-batch.ps1) — the scheduler entrypoint. -- [scripts/Common.ps1](./scripts/Common.ps1) — derived-state helpers (`Get-LastSyncedUpstreamSha`, `Get-PendingUpstreamShas`, `Get-StuckIssues`, `Get-GeneratedDir`). +- [references/03-conflict-triage.md](./references/03-conflict-triage.md) — Tier 0/1/2/3 resolution rubric with examples. +- [references/03-known-conflicts.md](./references/03-known-conflicts.md) — files that always need a fixed resolution. +- [references/04-build-verification.md](./references/04-build-verification.md) — try-build pipeline expectations. +- [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. +- [scripts/01-fetch-upstream.ps1](./scripts/01-fetch-upstream.ps1) — fetch microsoft/terminal main; return SHA. +- [scripts/02-compute-pending.ps1](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). +- [scripts/03-cherry-pick-one.ps1](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. +- [scripts/04-try-build.ps1](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. +- [scripts/05-finalize-pr.ps1](./scripts/05-finalize-pr.ps1) — push branch + open PR with squash-warning banner. +- [scripts/06-open-stuck-issue.ps1](./scripts/06-open-stuck-issue.ps1) — Tier-3 stuck issue (mid-pick conflict). +- [scripts/06b-open-build-stuck-issue.ps1](./scripts/06b-open-build-stuck-issue.ps1) — Tier-4 stuck issue (build failed after clean batch). +- [scripts/Common.ps1](./scripts/Common.ps1) — the only two helpers shared by 2+ scripts. diff --git a/.github/skills/upstream-sync/references/conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md similarity index 72% rename from .github/skills/upstream-sync/references/conflict-triage.md rename to .github/skills/upstream-sync/references/03-conflict-triage.md index bcd8f694e..1e9ba00d9 100644 --- a/.github/skills/upstream-sync/references/conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -45,8 +45,10 @@ if ((git diff --cached --quiet; $LASTEXITCODE) -eq 0) { ## Tier 2 — LLM-assisted trivial textual (opt-in) -Disabled by default; enable with `04-run-batch.ps1 -TryTier2`. Even when -enabled, this tier only fires when **all** of the following hold: +Disabled by default. The orchestrator in [`SKILL.md`](../SKILL.md) does +not invoke Tier-2 — if you want it, an agent walking the cherry-pick loop +can opt in per-conflict using the rubric below. Even when invoked, this +tier only fires when **all** of the following hold: - No more than 3 conflicting files. - Each file has fewer than 5 conflict hunks. @@ -89,13 +91,12 @@ Anything not resolved by Tier 0–2: ```pwsh git cherry-pick --abort -# Open the labeled stuck issue (07-open-stuck-issue.ps1) — issue body -# carries the ```yaml # wta-state``` block with stuck_on_sha + branch -# Write the report with the conflict diagnostics -# Exit with code 10 +# Open the labeled stuck issue (06-open-stuck-issue.ps1) — issue body +# carries the ```yaml # wta-state``` block with stuck_on_sha + branch. +# Surface the issue URL + branch to the operator and exit. ``` -The report **must** include: +The issue body (built by [`scripts/06-open-stuck-issue.ps1`](../scripts/06-open-stuck-issue.ps1)) **must** include: - The conflicting commit SHA, subject, author, and upstream URL. - The list of conflicting paths with a one-line classification each @@ -105,32 +106,17 @@ The report **must** include: that keeps the `(cherry picked from commit )` trailer, then CLOSE the stuck issue (that's the lock-clear signal — no script). -## Tier 4 — Post-pick validation failed - -The cherry-picks all applied cleanly, but a hard gate after the loop -said NO before any push or PR. This catches the class of bug missed by -git-level conflict detection: clean-merge-but-broken-content (PR #220 -audit found duplicate `.resw` keys + a dropped fork-specific `C4459` -suppression — both committed without git ever printing a conflict). - -The orchestrator runs three gates after the cherry-pick loop, in order: - -| Sub-tier | Gate | Symptom | Action | -|---|---|---|---| -| **4a** | Static breakage scan ([`scripts/08-static-scan.ps1`](../scripts/08-static-scan.ps1)) | New duplicate `.resw` keys vs base, or a missing fork invariant from [`fork-invariants.json`](./fork-invariants.json) | Lock + issue + exit 10 | -| **4b** | Try-build ([`scripts/10-try-build.ps1`](../scripts/10-try-build.ps1)) | Build exited non-zero within timeout | Lock + issue + exit 10 | -| **4c** | Try-build timeout | Wall-clock cap (default 45m) hit | Lock + issue + exit 10 — unless `-AllowInconclusiveBuild` (dev opt-in) | -| **4d** | Toolchain preflight ([`scripts/09-toolchain-preflight.ps1`](../scripts/09-toolchain-preflight.ps1)) | Required `PlatformToolset` (e.g. v143) not present on host | Lock + **NO issue** — provisioning problem, not code | - -**Why these three gates and not more.** They were sized to catch the -historical real-world failures with zero false positives: -- 4a covers content-level pattern breakage where git is happy but the - resulting file violates a fork-specific invariant. -- 4b is the broadest possible "did this even compile" check. -- 4c distinguishes "build hung — needs investigation" from "build - failed for a discoverable reason". -- 4d distinguishes "this code is broken" from "this host can't even - try to build it" — the latter must never open a GitHub issue. +## Tier 4 — Post-pick build failed + +The cherry-picks all applied cleanly, but [`scripts/04-try-build.ps1`](../scripts/04-try-build.ps1) +said NO before the PR could be finalized. The build runs before finalize +on purpose — see [`SKILL.md` step 7](../SKILL.md#7-build) for the +build-then-finalize ordering and the one-focused-fix-commit rule. + +| Sub-kind | Trigger | Action | +|---|---|---| +| **build-failed** | `bz no_clean` exited non-zero within timeout | Try ONE focused build-fix commit per [SKILL.md step 7](../SKILL.md#7-build). If that fails or scope is too large → open [Tier-4 stuck issue](../scripts/06b-open-build-stuck-issue.ps1) and exit. | +| **build-inconclusive** | Wall-clock cap (default 45 min) hit | Open Tier-4 stuck issue immediately (don't guess at fixing a hang). | Tier-4 state lives in the body of an open `upstream-sync-stuck` labeled issue (separate per kind by `findings_hash`); any such open issue causes diff --git a/.github/skills/upstream-sync/references/known-conflicts.md b/.github/skills/upstream-sync/references/03-known-conflicts.md similarity index 100% rename from .github/skills/upstream-sync/references/known-conflicts.md rename to .github/skills/upstream-sync/references/03-known-conflicts.md diff --git a/.github/skills/upstream-sync/references/04-build-verification.md b/.github/skills/upstream-sync/references/04-build-verification.md new file mode 100644 index 000000000..fa30c8905 --- /dev/null +++ b/.github/skills/upstream-sync/references/04-build-verification.md @@ -0,0 +1,85 @@ +# Build verification + +Post-pick hard gate. Runs **after** the cherry-pick loop and **before** +the PR is opened. If the build fails, the agent either lands one focused +build-fix commit on the same branch (so it ships in the same PR) or — if +the fix is too large / scope creep — opens a Tier-4 stuck issue and exits. + +The full orchestration around try-build lives in +[`SKILL.md` → Run a sync → step 7](../SKILL.md#7-build); this file is +just the contract for the `04-try-build.ps1` script and its diagnostics. + +## Why this exists + +A scheduler that opens PRs without proof the codebase still builds is +opening broken PRs. PR #220 was the motivating real-world failure: +every cherry-pick applied cleanly, every file looked right under +`git diff`, and the build broke because two unrelated upstream renames +landed `.resw` keys that collided with a fork-local commit. The compiler +catches that with zero false positives — `git` cannot. + +Toolchain provisioning (e.g. `PlatformToolset` v143/v145) is treated as +the operator's problem, not the scheduler's: an under-provisioned host +just keeps tripping the build gate and the human notices via the open +stuck issue. We intentionally do **not** auto-bump toolset versions in +the repo on behalf of a single host. + +## Try-build (`scripts/04-try-build.ps1`) + +Default invocation: + +```cmd +cmd.exe /c "tools\razzle.cmd && bz no_clean" +``` + +(`bz no_clean` = incremental Debug build of the full solution.) + +Configurable via the script's `-BuildCommand` parameter. The default is +verified on the maintainer host; if the build fails, the diagnostics +include the log path and tail. + +Output (returned as JSON on stdout): + +| Field | Meaning | +|---|---| +| `kind` | `build-ok` / `build-failed` / `build-inconclusive` | +| `exit_code` | Process exit code (`-1` for `build-inconclusive`) | +| `duration_ms` | Wall-clock ms | +| `command` | The build command that was run | +| `log_path` | Repo-relative path to the full log (under `Generated Files/upstream-sync//build-logs/`, gitignored) | +| `log_tail` | Last ~200 lines for inline display in the stuck issue | + +Timeout: + +- Default 45 minutes (`-TimeoutMinutes`). +- On timeout the build is killed and classified as `build-inconclusive`. + +## When the build fails + +The agent's decision tree is in [`SKILL.md` step 7](../SKILL.md#7-build). +In short: try ONE focused fix commit when the cause is mechanical and +clearly caused by the pick batch; otherwise open the Tier-4 issue and +exit. Do **not** pile up multiple fix commits — the one-fix-per-PR rule +exists so the cherry-pick PR stays auditable as "upstream batch + at +most one mechanical fix". + +## When the build fails for fork-unrelated reasons + +If a flaky build (transient toolchain glitch, env issue, missing +PlatformToolset, ...) trips the gate: + +1. The Tier-4 stuck issue gives a clear log tail. +2. A human can re-run the build locally, confirm it's transient or fix + the host, then **close the stuck issue** to clear the lock. +3. The next scheduler tick re-attempts the same pick range from scratch. + +Distinguishing transient-build from real-pick-broke-build is left to +the human reviewing the issue — too noisy to automate, and the cost +of a manual cross-check is small (~once per N runs). + +## Build artifacts + +`Generated Files/upstream-sync//build-logs/` is **not** +committed — the repo root's `**/Generated Files/` gitignore rule +covers it. Build outputs under `bin/`, `obj/`, etc. follow the repo's +existing `.gitignore`. diff --git a/.github/skills/upstream-sync/references/build-verification.md b/.github/skills/upstream-sync/references/build-verification.md deleted file mode 100644 index 4cced5ed1..000000000 --- a/.github/skills/upstream-sync/references/build-verification.md +++ /dev/null @@ -1,104 +0,0 @@ -# Build verification - -Post-batch hard gate. Runs after the static scan passes and **before** -push / PR creation. If the build fails, the run is marked Tier-4 stuck. - -## Why this exists - -Static scan catches a specific set of content drifts. The compiler -catches everything else (missing includes, type mismatches, -vcxproj drift, MIDL/winrt projection errors, ...) with zero false -positives. A scheduler that opens PRs without proof the codebase still -builds is opening broken PRs — exactly what PR #220 risked. - -## Pipeline - -``` -toolchain preflight ─→ static scan ─→ try-build ─→ push / PR - │ │ │ - (infra-stuck) (Tier-4) (Tier-4) -``` - -## Toolchain preflight (`scripts/09-toolchain-preflight.ps1`) - -Runs first. Discovers the required `PlatformToolset` from -`src/common.build.pre.props` (and other props files) and verifies it -is installed under any Visual Studio install on the host. - -Outcomes: - -| Outcome | Behavior | -|---------------------|--------------------------------------------------------| -| All toolsets found | Continue to static scan + build. | -| Required missing | Tier-4 **infra-stuck** — separate kind from code-stuck. Does NOT open a stuck issue, does NOT set a lock (no labeled issue to gate on). The next scheduler tick simply retries; provision the host before then. | -| Skipped (`-SkipBuild`) | Preflight not run. Caller accepts risk. | - -**The preflight does NOT auto-bump v143→v145.** That recipe is -intentionally kept as a *local-only* developer workaround (see the -`v143-v145` memory notes). Auto-bump risks silently changing the -toolset for everyone, which would break the rest of the team. -Schedulers should be provisioned with the correct VS install instead. - -## Try-build (`scripts/10-try-build.ps1`) - -Default invocation: - -```cmd -cmd.exe /c "tools\razzle.cmd && bz no_clean" -``` - -(`bz no_clean` = incremental Debug build of the full solution.) - -Configurable via the orchestrator's `-BuildCommand` parameter. The -default is verified on the maintainer host; if validation blocks the run, -the generated Tier-4 diagnostics include the build log path and tail. - -Output: - -- Full build log → `Generated Files/upstream-sync//build-logs/.log` - (gitignored — these are big and noisy; the gitignore is the repo root's - `**/Generated Files/` rule). -- Last ~200 lines → captured into the run report and any Tier-4 stuck - issue body. -- Exit code + duration → embedded in the Tier-4 stuck-issue YAML block - (and the run report) when the build is the failing gate. - -Timeout: - -- Default 45 minutes (cold builds on a new sync branch with many - picks can hit ~30 min; 45 gives headroom). -- Configurable via `-BuildTimeoutMinutes N`. -- On timeout the build is killed and classified as - **build-inconclusive**. - -| Outcome | Default scheduler behavior | Dev opt-out | -|--------------------|--------------------------------|--------------------------------------| -| Build succeeded | Continue to push / PR. | n/a | -| Build failed | Tier-4 stuck, open issue. | n/a | -| Build inconclusive | Tier-4 stuck (be safe). | `-AllowInconclusiveBuild` → proceed with warning in report. | -| Skipped | Reject in scheduler context. | `-SkipBuild` for fast dev iteration. | - -The "be safe" default for inconclusive is deliberate. A hung build is -indistinguishable from a real failure in scheduler mode; opening a PR -on an unproven sync defeats the whole point of this gate. - -## When the build fails for fork-unrelated reasons - -If a flaky build (unrelated env issue, transient toolchain glitch) -trips the gate: - -1. The stuck issue gives a clear log tail. -2. A human can re-run the build locally to confirm it's a transient, - then **close the stuck issue** to clear the lock. -3. The next scheduler tick will re-attempt the same pick range. - -Distinguishing transient-build from real-pick-broke-build is left to -the human reviewing the issue — too noisy to automate, and the cost -of a manual cross-check is small (~once per N runs). - -## Build artifacts - -`Generated Files/upstream-sync//build-logs/` is **not** -committed — the repo root's `**/Generated Files/` gitignore rule -covers it. Build artifacts under `bin/`, `obj/`, etc. follow the -repo's existing `.gitignore`. diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index 521cb71be..7486d269c 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -26,7 +26,7 @@ follow-up PR. | Comment / failure | Fix where | Why | |---|---|---| | Build error introduced by the cherry-pick batch | **Sync PR** — one focused commit | The sync PR is broken; the sync PR must be made buildable before merge. | -| Duplicate `.resw` keys / manifest collisions surfaced by `bz` but missed by the static-scan baseline | **Sync PR** — one focused commit | Same reason — without it the sync PR can't merge. | +| Duplicate `.resw` keys / manifest collisions surfaced by `bz` only at build time | **Sync PR** — one focused commit | Same reason — without it the sync PR can't merge. | | CI gate failure on the sync PR (check-spelling, lint, format) that is genuinely caused by the cherry-picked content | **Sync PR** — one focused commit | Same reason. | | Copilot reviewer correctness finding (e.g. accumulator drift past clamp floor, missing cleanup on exception path) | **Follow-up PR** | Reviewer added value but the change is a *refactor* of upstream code, not a build fix. Cherry-pick PR stays faithful. | | Translation error (`zh-TW` 工作模式 vs 工作階段) | **Follow-up PR** | The cherry-pick faithfully imported what upstream shipped. Correcting it is fork-local improvement. | diff --git a/.github/skills/upstream-sync/references/fork-invariants.json b/.github/skills/upstream-sync/references/fork-invariants.json deleted file mode 100644 index cda885126..000000000 --- a/.github/skills/upstream-sync/references/fork-invariants.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "$schema_comment": "Invariants the fork must preserve after every upstream pick. The static-scan step ('scripts/08-static-scan.ps1') fails the run if any of these patterns is missing from the post-pick worktree. Use this for fork-specific content that an upstream cherry-pick can silently strip (e.g. take-upstream conflict resolution removing a fork-added warning suppression).", - "version": 1, - "invariants": [ - { - "id": "common-build-c4459-suppression", - "path": "src/common.build.pre.props", - "must_contain_regex": "[^<]*\\b4459\\b[^<]*", - "severity": "high", - "reason": "Fork added C4459 (declaration of 'X' hides class member) suppression because TreatWarningAsError=true and fork-specific code triggers it. Upstream removed C4459 in the ATL/MFC cleanup; a take-upstream resolution drops it and the next build fails. See PR #220 audit (2026-06-04).", - "introduced_in_fork_sha": "4bf2b6a45", - "added_after_pr_220_audit": true - } - ] -} diff --git a/.github/skills/upstream-sync/references/static-scan.md b/.github/skills/upstream-sync/references/static-scan.md deleted file mode 100644 index 71046a47b..000000000 --- a/.github/skills/upstream-sync/references/static-scan.md +++ /dev/null @@ -1,135 +0,0 @@ -# Static breakage scan - -Post-batch hard gate. Runs after all cherry-picks succeed and **before** -push / PR creation. If any finding is `critical` or `high`, the run is -marked Tier-4 stuck (see [conflict-triage.md](./conflict-triage.md)). - -## Why this exists - -A clean cherry-pick is not a working cherry-pick. PR #220 demonstrated -two failure modes that git-level conflict detection misses: - -1. **Duplicate `.resw` keys** — fork had appended translations for new - keys, upstream loc-bot pick renamed old keys to the same names → - duplicates in 26 files / 216 keys. -2. **Dropped fork-specific build suppressions** — `take-upstream` - resolution of `src/common.build.pre.props` removed the fork-added - `C4459` warning suppression. - -Both were caught post-hoc by audit. This scan catches them pre-PR. - -## What v1 checks - -### 1. Duplicate `.resw` keys (baseline-diff) - -For every `*.resw` file modified anywhere in the pick range, count -duplicate `` entries in the **pre-pick** state vs the -**post-pick** worktree. Gate on **newly-introduced** duplicates only — -preexisting duplicates are reported as `info`, not blocking. - -Pre-pick state = the file content at `origin/main` (the orchestrator's -base before the picks). Post-pick = the worktree on the sync branch. - -Severity: -- `critical` — any new duplicate key introduced by the pick range. -- `info` — preexisting duplicates carried forward. - -> **Implementation note (PR #220 follow-up).** Both `Get-FileTextAtRef` -> and `Get-FileTextOnDisk` MUST use absolute paths when calling -> `[System.IO.File]::ReadAllText`. .NET resolves relative paths against -> `[System.Environment]::CurrentDirectory`, NOT PowerShell's `$PWD`, so -> a relative path silently reads from a different worktree. PR #220's -> first dedup pass returned `blocking=false` while the build was still -> broken because of this exact bug — the scan was reading the dup-free -> `intelligent-terminal` main worktree instead of the dup-laden sync -> worktree. Same caveat for `git show` — capture via `cmd /c "git show -> ref:path > tmp 2>nul"` instead of PowerShell-pipeline join; otherwise -> high-Unicode (pseudo-locale) bytes get truncated by the PSObject -> formatter. - -> **Operator note: manual dedup regex.** When fixing duplicates by hand -> (the orchestrator does NOT auto-dedup), the matching regex must be -> CRLF-agnostic: `(?s)([ \t]*)]*>.*?(\r?\n)?`. -> A `\r?\n`-anchored regex misses inline-concatenated entries -> (`......`) that the loc-bot -> commits as a single appended line for fork-only keys in -> `qps-ploc/qps-ploca/qps-plocm` — exactly the second failure mode that -> blocked PR #220 build retry #4. - -### 2. Fork invariants (regex must-match) - -Reads `references/fork-invariants.json`. For each entry: -- Load the file at `path` from the post-pick worktree. -- Test `must_contain_regex` (case-sensitive, single-line). -- If it doesn't match → finding at the configured `severity`. - -Seed: `C4459` suppression in `src/common.build.pre.props`. - -Add new invariants whenever an audit finds another fork-specific item -that an upstream pick could silently strip. Keep the regex narrow. - -## What v1 does NOT check (deferred to v2) - -Documented here so contributors don't re-invent them and so v2 has a -clear scope: - -- **Missing `#include`s** — narrow scope (newly-added quote includes - in modified .h/.cpp/.idl, skipping `<...>`, `*.g.h`, `*.g.cpp`, - `precomp.h`, `winrt/.*`) is technically straightforward but each - exclusion list is a tail of false-positives. The try-build step (see - [build-verification.md](./build-verification.md)) catches missing - includes as compile errors with zero false positives, so v1 relies - on the build step for this class. -- **vcxproj / vcxproj.filters drift** — same reasoning: a missing - `ClCompile Include=` reference fails the build with a clear error. -- **MTSMSettings / GlobalAppSettings drift** — relies on the build - step for now; a v2 addition could be a focused cross-check if a real - miss slips past the build. - -## Output - -The scan emits a JSON document on stdout: - -```json -{ - "findings": [ - { - "check": "resw-duplicate-keys", - "severity": "critical", - "path": "src/cascadia/TerminalApp/Resources/zh-CN/Resources.resw", - "detail": "15 newly-duplicated entries", - "examples": ["ConfirmCloseDialog_Cancel", "..."] - }, - { - "check": "fork-invariant", - "severity": "high", - "id": "common-build-c4459-suppression", - "path": "src/common.build.pre.props", - "detail": "regex '\\b4459\\b' did not match" - } - ], - "summary": { - "critical": 13, - "high": 1, - "medium": 0, - "low": 0, - "info": 0 - }, - "blocking": true -} -``` - -`blocking` = true ⇔ any `critical` or `high` finding present. - -## Extending the scan - -To add a new check: - -1. Add a function to `scripts/08-static-scan.ps1` that returns a list - of finding hashtables (same shape as above). -2. Wire it into the main loop in `08-static-scan.ps1`. -3. Document the check here. -4. Add a baseline-aware test (compare pre-pick vs post-pick) **only - when** the check would otherwise gate on preexisting issues. -5. Wire its severities into the `blocking` calculation if it should - block the run. diff --git a/.github/skills/upstream-sync/references/workflow.md b/.github/skills/upstream-sync/references/workflow.md deleted file mode 100644 index ac7fc82ac..000000000 --- a/.github/skills/upstream-sync/references/workflow.md +++ /dev/null @@ -1,327 +0,0 @@ -# Upstream Sync — Full Workflow - -This is the authoritative per-step procedure. The orchestrator is -[`scripts/04-run-batch.ps1`](../scripts/04-run-batch.ps1); each step below -maps to a script or an in-orchestrator function. - -## State model — derived, not stored - -There is **no `state.json`**. Every persistent fact is derived from an -authoritative source on demand: - -| Fact | Source | Helper in [`scripts/Common.ps1`](../scripts/Common.ps1) | -|---|---|---| -| Last-synced upstream SHA | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main` | `Get-LastSyncedUpstreamSha` | -| Pending list | `git log --cherry-pick --right-only --no-merges ...upstream/main` (patch-id-aware; reverted picks reappear) | `Get-PendingUpstreamShas -Since ` | -| Stuck-lock | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal` | `Get-StuckIssues` | -| Stuck-lock metadata | Fenced ```yaml # wta-state``` block inside the issue body | `Get-StuckMetaFromIssue` | -| Transient artifacts (reports, build logs) | `Generated Files/upstream-sync//` (gitignored at repo root) | `Get-GeneratedDir [-Sub]` | - -## Entry Conditions - -- Working tree is clean (`git status --porcelain` empty). -- We are on `main` (or the script will `git switch main` and `git pull --ff-only origin main`). -- `Get-StuckIssues` returns empty (otherwise exit early — see "Stuck-lock" below). - -## Steps - -### 1. Fetch upstream - -```pwsh -git remote get-url upstream 2>$null || git remote add upstream https://github.com/microsoft/terminal.git -git fetch upstream main --no-tags -``` - -Script: [`01-fetch-upstream.ps1`](../scripts/01-fetch-upstream.ps1). - -If `git rev-parse upstream/main` equals `Get-LastSyncedUpstreamSha`, the -orchestrator writes a local no-op report and exits 0. - -### 2. Compute pending range - -```pwsh -$since = Get-LastSyncedUpstreamSha -git log --cherry-pick --right-only --no-merges --format='%H' --reverse "$since...upstream/main" -``` - -Oldest-first ordering is mandatory. Cherry-picking newest-first inverts -dependencies and creates spurious conflicts. `--cherry-pick` compares -patch IDs, so a commit that was picked then reverted on `origin/main` -correctly re-appears here as pending. - -Script: [`02-compute-pending.ps1`](../scripts/02-compute-pending.ps1) emits -a JSON object on stdout — see step 3 below for the full shape. - -### 3. Detect & drop revert pairs - -A commit is a revert if its **first line** matches `^Revert "..."$` **or** -its body contains `This reverts commit <40-hex>`. - -- If `<40-hex>` is **inside** the pending range AND has not been picked - yet → drop **both** the original and the revert; record the pair. -- If `<40-hex>` is **outside** the pending range (already synced earlier) - → keep the revert; it must land as a normal pick. - -Script: same `02-compute-pending.ps1`. Full return shape: -`{ from: "", to: "", pending: [...], dropped_pairs: [[A,B],...], skipped_empty: [...] }`. - -### 4. Drop upstream-empty commits - -Before picking, check `git diff-tree --no-commit-id --name-only -r `. -If empty, mark skipped and record. (Cheaper to detect upfront than to -pick and reset.) - -### 5. Create the sync branch - -```pwsh -# Branch name is per-run, never reused: date + UTC HHmmss + 4 random hex chars. -# This guarantees a fresh branch for every run (no risk of replaying onto a -# stale branch that GitHub didn't auto-delete after a rebase-merge). -$branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))-$((Get-Date).ToUniversalTime().ToString('HHmmss'))-$(([guid]::NewGuid().ToString('N').Substring(0,4)))" -git switch -c $branch -``` - -Resume = pick up the branch name from the run report or the open -`upstream-sync-stuck` issue body, not by deriving from the date. - -### 6. Cherry-pick loop - -For each commit in the (now-filtered) pending list: - -```pwsh -git cherry-pick --keep-redundant-commits -x -``` - -- `-x` adds `(cherry picked from commit )` to the message — critical - for audit trail, for the next-run revert-pair detector, **and for - `Get-LastSyncedUpstreamSha` to derive the next watermark.** Never strip it. -- `--keep-redundant-commits` lets us preserve no-op picks for traceability - (we then `git reset --hard HEAD~1` if Tier-1 fires). - -**On conflict, apply resolution tiers in order:** - -1. **Tier 0 — known take-upstream / take-ours files.** Read - [`known-conflicts.md`](./known-conflicts.md). For every conflicting - path in the Tier-0 list, run `git checkout --theirs ` (or - `--ours`), then `git add `. If **all** conflicting paths are - resolved, run `git cherry-pick --continue` and move on. -2. **Tier 1 — empty after pick.** If `git diff --cached --quiet` returns - zero exit code (no staged changes), the commit was already applied or - is a no-op against fork: `git cherry-pick --skip` and record. -3. **Tier 2 — trivial textual (opt-in via `-TryTier2`).** Delegate to a - fresh sub-agent with the conflict text. Accept only `high` confidence. - See [conflict-triage.md](./conflict-triage.md#tier-2-llm-assisted). -4. **Tier 3 — semantic conflict.** Run `git cherry-pick --abort`. Open - the labeled stuck issue, write report, exit 10. The next scheduler - tick sees the open labeled issue and skips. - -Script: [`03-cherry-pick-one.ps1`](../scripts/03-cherry-pick-one.ps1) -handles one commit, returns a JSON status object. The orchestrator loops. - -### 7. Post-pick validation gates (Tier-4) - -After all cherry-picks complete cleanly, the orchestrator runs three -hard gates **before** writing the report or pushing anything. The order -matters: cheapest infra check first, then content, then full build. - -#### 7a. Toolchain preflight - -```pwsh -pwsh .github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 -# Emits JSON { required_toolsets, available_toolsets, missing, vs_installs, ok } -``` - -Detects required `` values from `src/common.build.*.props` -and checks they exist under `\MSBuild\Microsoft\VC\\Platforms\x64\PlatformToolsets\`. -If `ok=false`, this is **Tier-4d infra-stuck**: NO GitHub issue, NO lock -(PR review cannot fix host provisioning; the next scheduler tick simply -retries from a properly provisioned host). Skipped when `-SkipBuild` is set. - -#### 7b. Static breakage scan - -```pwsh -pwsh .github/skills/upstream-sync/scripts/08-static-scan.ps1 -BaseSha $preBase -# Emits JSON { base, head, findings: [...], summary: { critical, high, ... }, blocking } -``` - -`$preBase` is `git rev-parse origin/main` captured BEFORE the cherry-pick -loop. The scan: - -- Baseline-diffs every changed `.resw` file for NEW duplicate `` - keys (preexisting dups are reported as `info`, not blocking). -- Runs regex assertions from [`fork-invariants.json`](./fork-invariants.json) - against the post-pick worktree. - -If `blocking=true` (any `critical` or `high` finding), this is **Tier-4a -stuck**: opens labeled issue + exit 10. Skipped when `-SkipStaticScan`. - -#### 7c. Try-build - -```pwsh -pwsh .github/skills/upstream-sync/scripts/10-try-build.ps1 -BuildCommand 'tools\razzle.cmd && bz no_clean' -TimeoutMinutes 45 -# Emits JSON { kind, exit_code, duration_ms, log_path, log_tail } -``` - -- `kind = build-ok` → continue to step 8. -- `kind = build-failed` → **Tier-4b stuck**. -- `kind = build-inconclusive` (timeout) → **Tier-4c stuck**, unless - `-AllowInconclusiveBuild` (dev opt-in; never in a scheduler). - -Skipped when `-SkipBuild`. Logs land in -`Generated Files/upstream-sync//build-logs/` (gitignored). - -### 8. Write report (always) - -Regardless of outcome (ok / no-op / dry-run / stuck / stuck-static-scan / -stuck-build-failed / stuck-build-inconclusive / stuck-toolchain-missing), -write `Generated Files/upstream-sync//-.md` -with: - -- Run metadata (start, end, duration, host, status) -- Counts: picked / dropped-pair / empty / known-conflict-resolved / stuck-at -- For each picked commit: SHA, subject, author -- For dropped pairs: the two SHAs and their subjects -- If stuck (Tier-3): the conflicting commit, the conflicting paths, what was attempted, the exact resume command -- If stuck (Tier-4): the validation findings, the build log tail, the exact resume command - -Reports are **transient** — never committed. The stuck issue body -(step 9b/9c) inlines the parts of the report a reviewer needs without -fetching the local file. - -Script: [`05-write-report.ps1`](../scripts/05-write-report.ps1). - -### 9a. Success path — push + open PR - -```pwsh -git push -u origin $branch -gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title "chore(upstream): sync up to $shortSha" --body-file $reportPath -``` - -No state-file commit. The `(cherry picked from commit )` trailer on -each cherry-pick IS the watermark — once the PR merges, the trailer is -on `origin/main` and the next run's `Get-LastSyncedUpstreamSha` finds it. - -Script: [`06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1). - -### 9b. Stuck path (Tier-3) — open labeled issue - -```pwsh -gh issue create -R microsoft/intelligent-terminal --label upstream-sync-stuck ` - --title "Upstream sync stuck at : " ` - --body-file $reportPath -``` - -The issue body carries a fenced ```yaml # wta-state``` block with -`tier`, `kind=cherry-pick-conflict`, `stuck_on_sha`, `branch`, `at`, -`host` so a future run's `Get-StuckMetaFromIssue` can read the lock -context. Nothing is committed to `main`. - -Script: [`07-open-stuck-issue.ps1`](../scripts/07-open-stuck-issue.ps1). - -### 9c. Stuck path (Tier-4) — open labeled issue - -For Tier-4a/b/c, same flow as 9b — open the labeled issue with a -`# wta-state` block carrying `tier=4`, `kind`, `findings_hash`, -`picked_count`. For Tier-4d (toolchain-missing), NO issue is opened -(infra problem); the next scheduler tick simply retries. - -Script: [`07b-open-validation-stuck-issue.ps1`](../scripts/07b-open-validation-stuck-issue.ps1). - -### 10. After-PR review handling (post-merge gate) - -Once the sync PR is open and reviewers (the GitHub Copilot bot, then -humans) start leaving comments, route the response by **comment kind**, -not by reviewer: - -| Comment kind | Where to fix | -|---|---| -| Build-blocking on the sync PR — compile errors, dedup of conflicts surfaced only at build time, sync-PR CI gate failures (check-spelling, lint, format) genuinely caused by the cherry-picked content | **Sync PR**, in **one** focused extra commit. Anything more than one extra commit is a smell. | -| Everything else — Copilot correctness findings, logic suggestions, translation corrections, spelling allow/expect migrations, doc/comment typos, design feedback | **Follow-up PR** based on the sync PR's HEAD. | - -The cherry-pick PR's value to a reviewer is "N small commits, each -faithful to one upstream commit, plus the bare minimum to make CI -green". Bundling substantive review fixes destroys that audit -property and forces the reviewer to mentally subtract those commits -from every upstream-comparison check. - -**Follow-up PR mechanics** (full procedure in -[follow-up-pr.md](./follow-up-pr.md)): - -1. Open a sibling worktree on a new branch off the sync PR's head: - ```pwsh - git fetch origin - git worktree add ..\it-fix -b dev//sync--review-fixes "origin/" - ``` -2. Apply fixes as **one focused commit per concern** (code-bugs / - translations / spelling-cleanup / etc.) — same "one commit per - round" rule as - [`copilot-pr-review-loop`](../../copilot-pr-review-loop/SKILL.md). -3. `gh pr create --base --head dev//sync--review-fixes` — - base is the **sync branch**, not `main`, so only the fix commits - show in the diff. -4. Walk every deferred review thread on the sync PR and reply + - resolve pointing to the follow-up PR number, via - [`copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1`](../../copilot-pr-review-loop/scripts/06-reply-and-resolve.ps1). -5. If the sync PR merges first, rebase the follow-up onto `main` and - `gh pr edit --base main`. - -The PR banner emitted by -[`scripts/06-finalize-pr.ps1`](../scripts/06-finalize-pr.ps1) spells -this policy out so the first reviewer doesn't push back on deferred -fixes. - -## Stuck-Lock - -When `Get-StuckIssues` returns any OPEN `upstream-sync-stuck` labeled -issue, the orchestrator: - -1. Logs `"stuck-lock set ( at ); skipping run"`. -2. Writes a transient `-skipped.md` under - `Generated Files/upstream-sync//` noting the skip. -3. Exits 0 (the scheduler should not alarm). - -To clear the lock after the human has resolved the underlying issue: - -``` -1. Resolve the conflict on the stuck branch (the exact name is in the - stuck-issue body, e.g. `upstream-sync/2026-06-04-091512-a3f1`), - keeping every `(cherry picked from commit )` trailer intact. -2. Open a PR for the fix, merge it (rebase or merge — NOT squash). -3. CLOSE the stuck issue. That's the lock-clear signal — no script. -``` - -The next scheduler tick: -- `Get-LastSyncedUpstreamSha` re-derives the watermark from the merged - PR's trailers (advancing past the resolved batch — for Tier-3 by the - exact resolved commit; for Tier-4 by whatever extra trailers the fix - PR carried). -- `Get-StuckIssues` returns empty (the issue is closed). -- The run proceeds from the new watermark. - -For Tier-4 where the operator wants to **re-attempt the same range** -(e.g. because the fix landed as a separate PR on `main` that doesn't -itself carry trailers), simply close the issue without merging a sync -fix: the next run will recompute pending against the same watermark and -re-validate. - -## Sub-Agent Delegation Map - -| Step | Delegate to fresh sub-agent? | Why | -|---|---|---| -| 1–2 (fetch, compute) | No | Pure git plumbing, deterministic. | -| 3 (revert-pair detection) | No | Mechanical; the script does it. | -| 6 / Tier-2 (LLM-assisted textual resolution) | **Yes — required** | Implementer bias risk; require `high` confidence and a different agent to verify before staging. | -| 7 (write report) | No | Template fill. | -| 8a (PR body polish) | Optional | If picked > 20 commits, a sub-agent can group them by area for the PR body. | -| 8b (issue summary) | **Yes** | A fresh agent writes a clearer "what's hard about this conflict" summary than the loop that aborted. | - -## Exit Codes (from `04-run-batch.ps1`) - -| Code | Meaning | -|---|---| -| 0 | Success (PR opened) **or** no-op **or** skipped because lock is set | -| 10 | Stuck — issue opened, lock set (this is **not** an error; scheduler should not alarm) | -| 20 | Hard failure (git command failed unexpectedly, network down, gh auth missing) — scheduler **should** alarm | - -Wrap the scheduler invocation accordingly: treat 0 and 10 as healthy, -20 as paging-worthy. diff --git a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 index 48e185650..4bde1869d 100644 --- a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +++ b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 @@ -5,11 +5,22 @@ Writes the current upstream/main SHA to stdout. #> [CmdletBinding()] -param() +param( + [string] $UpstreamUrl = 'https://github.com/microsoft/terminal.git' +) . "$PSScriptRoot/Common.ps1" -Ensure-UpstreamRemote +# Ensure-UpstreamRemote (inlined — single-use). Adds the `upstream` remote +# if missing; bails if it points somewhere unexpected. +$existing = git remote get-url upstream 2>$null +if ($LASTEXITCODE -ne 0) { + git remote add upstream $UpstreamUrl | Out-Null + if ($LASTEXITCODE -ne 0) { throw "Failed to add 'upstream' remote." } +} elseif ($existing.Trim() -ne $UpstreamUrl) { + throw "Remote 'upstream' points at '$($existing.Trim())' (expected '$UpstreamUrl'). Fix the remote before running upstream-sync." +} + git fetch upstream main --no-tags 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 7e34b12d6..c7460dcea 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -4,17 +4,20 @@ .DESCRIPTION Reads the last-synced upstream watermark from origin/main's - `cherry picked from commit ` trailers, lists commits in - watermark..upstream/main (oldest first), detects revert pairs within the - range and drops them, detects upstream-empty commits and drops them, and - emits the final pending list as JSON. + `cherry picked from commit ` trailers, lists commits patch-id-aware + between watermark and upstream/main (oldest first), detects revert pairs + within the range and drops them, detects upstream-empty commits and drops + them, and emits the final pending list as JSON. + + `01-fetch-upstream.ps1` must have been run first (we need upstream/main + in this clone). .OUTPUTS JSON object on stdout: { "from": "", "to": "", - "pending": [ "", ... ], # in pick order + "pending": [ "", ... ], "dropped_pairs": [ ["", ""], ... ], "skipped_empty": [ "", ... ] } @@ -24,10 +27,57 @@ param() . "$PSScriptRoot/Common.ps1" -# `git fetch upstream main` must have been run already (orchestrator calls -# 01-fetch-upstream.ps1 before us). Get-LastSyncedUpstreamSha walks the -# `cherry picked from commit ` trailers on origin/main back to the most -# recent one that resolves to a commit on upstream/main — no state.json. +# --- Inlined helpers (single-use; see Common.ps1 comment for why) ---------- + +function Resolve-FullCommitSha { + param([Parameter(Mandatory)] [string] $Sha) + $full = (git rev-parse "$Sha^{commit}" 2>$null) + if ($LASTEXITCODE -ne 0 -or -not $full) { throw "Could not resolve commit SHA '$Sha'." } + return $full.Trim() +} + +function Get-LastSyncedUpstreamSha { + # Walks origin/main newest-first, returns the first `(cherry picked from + # commit )` trailer whose target is reachable from upstream/main. + # Capped at 5000 most-recent commits — well beyond any realistic + # weekly-sync deployment. Resolves the trailer SHA to its full 40-char + # form before the ancestry check so an abbreviated/ambiguous trailer + # doesn't silently skip an otherwise-valid watermark candidate. + $commits = @(git log origin/main --max-count=5000 --grep='cherry picked from commit' --format='%H' 2>$null) + if ($LASTEXITCODE -ne 0) { throw "git log on origin/main failed while deriving last-synced SHA." } + foreach ($c in $commits) { + $body = git log -1 --format='%B' $c 2>$null + if ($body -match '\(cherry picked from commit ([0-9a-f]{7,40})\)') { + $fullSha = $null + try { $fullSha = Resolve-FullCommitSha $matches[1] } catch { continue } + $null = git merge-base --is-ancestor $fullSha upstream/main 2>$null + if ($LASTEXITCODE -eq 0) { return $fullSha } + } + } + throw "No 'cherry picked from commit' trailer pointing at upstream/main was found on origin/main (scanned the most recent 5000 commits). The very first sync needs an operator to seed the watermark commit (see SKILL.md - 'First-time sync')." +} + +function Get-PendingUpstreamShas { + # Patch-id-aware oldest-first list of upstream/main commits not yet on + # origin/main. `--cherry-pick` drops picked-then-reverted pairs by patch + # ID; -Since further trims by ancestry to keep the walk fast. + param([string] $Since) + $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) + if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } + $shas = @($out | Where-Object { $_ -match '^[0-9a-f]{40}$' }) + if ($Since) { + $filtered = New-Object 'System.Collections.Generic.List[string]' + foreach ($sha in $shas) { + $null = git merge-base --is-ancestor $sha $Since 2>$null + if ($LASTEXITCODE -ne 0) { [void] $filtered.Add($sha) } + } + $shas = @($filtered) + } + return ,$shas +} + +# --- Main logic ------------------------------------------------------------ + $from = Get-LastSyncedUpstreamSha $to = (git rev-parse upstream/main).Trim() if ($LASTEXITCODE -ne 0) { throw "git rev-parse upstream/main failed." } @@ -37,18 +87,10 @@ if ($from -eq $to) { return } -# Patch-id-aware list of full SHAs (oldest-first). Uses Get-PendingUpstreamShas -# from Common.ps1, which wraps `git log --cherry-pick --right-only --no-merges`: -# any upstream commit whose patch ID matches a commit already on origin/main is -# excluded (so picked-then-reverted commits stay out unless their patch is no -# longer on origin/main, in which case they correctly re-appear as pending). -# The revert-pair detection below stays as defense-in-depth and as the source -# of the `dropped_pairs` report field; in practice --cherry-pick already drops -# most pairs, but a same-batch original+revert that wasn't yet on origin/main -# at the time of computation is still useful to surface. $all = @(Get-PendingUpstreamShas -Since $from) -# Build sha -> first line and body map (single git invocation per commit is fine for typical batch sizes). +# Build sha -> first-line / body lookup (one git invocation per commit is +# fine at typical batch sizes). $info = @{} foreach ($sha in $all) { $subj = git log -1 --format='%s' $sha @@ -56,11 +98,9 @@ foreach ($sha in $all) { $info[$sha] = @{ subject = $subj; body = $body } } -# Detect revert pairs. Note: when a revert body lists multiple SHAs -# (e.g. "This reverts commit A. This also undoes parts of B"), the first -# match wins — that is, the SHA following the canonical "This reverts -# commit " line introduced by `git revert`. Bodies that list a -# secondary SHA outside the canonical form are ignored on purpose. +# Detect revert pairs WITHIN the range. `--cherry-pick` already handled +# pairs that crossed the watermark boundary; this catches same-batch +# original+revert that wasn't on origin/main yet at computation time. $dropped = New-Object 'System.Collections.Generic.HashSet[string]' $pairs = @() foreach ($sha in $all) { @@ -72,14 +112,10 @@ foreach ($sha in $all) { if ($body -match 'This reverts commit ([0-9a-f]{40})\b') { $targetSha = $Matches[1] } elseif ($subj -match '^Revert "') { - # Best-effort fallback: match the quoted original subject. To - # avoid pairing the revert with a *later* unrelated commit - # that happens to share the subject, search only the prefix of - # $all up to (but not including) the current revert — the - # original must precede its revert in oldest-first order. - # Also require exactly one match; if subjects repeat, fall - # through and let the revert land as a normal pick (safer - # than dropping the wrong commit). + # Fallback: match by quoted subject, but only against commits + # earlier in the oldest-first range, and only when the match is + # unique (otherwise leave the revert as a normal pick rather + # than risk dropping the wrong commit). $origSubject = $subj -replace '^Revert "', '' -replace '"\s*$', '' -replace '"\.?\s*$','' $prefix = @() foreach ($candidateSha in $all) { @@ -112,11 +148,10 @@ foreach ($sha in $all) { $pending = $all | Where-Object { -not $dropped.Contains($_) } -$result = [ordered] @{ +[ordered] @{ from = $from to = $to pending = @($pending) dropped_pairs = @($pairs) skipped_empty = @($empty) -} -$result | ConvertTo-Json -Depth 5 +} | ConvertTo-Json -Depth 5 diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 73abfb8dd..7e7dcfc63 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -24,7 +24,7 @@ param( . "$PSScriptRoot/Common.ps1" function Get-KnownConflicts { - $md = Join-Path (Split-Path $PSScriptRoot -Parent) 'references/known-conflicts.md' + $md = Join-Path (Split-Path $PSScriptRoot -Parent) 'references/03-known-conflicts.md' if (-not (Test-Path $md)) { return @() } $lines = Get-Content -LiteralPath $md $entries = @() diff --git a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 b/.github/skills/upstream-sync/scripts/04-run-batch.ps1 deleted file mode 100644 index 3ed563b6d..000000000 --- a/.github/skills/upstream-sync/scripts/04-run-batch.ps1 +++ /dev/null @@ -1,359 +0,0 @@ -<# -.SYNOPSIS - Orchestrator: run one upstream-sync pass. Safe to invoke from a - scheduler on a weekly/daily cadence. - -.DESCRIPTION - No state.json. Everything is derived from authoritative sources: - * last-synced watermark -> Get-LastSyncedUpstreamSha (origin/main trailers) - * pending list -> Get-PendingUpstreamShas (git log --cherry-pick) - * stuck-lock -> Get-StuckIssues (open upstream-sync-stuck labeled issues) - - If any open `upstream-sync-stuck` labeled issue exists, this run skips - with a `skipped-locked` report and exits 0. Otherwise: - 1. Fetches upstream/main. - 2. Computes pending commits, dropping revert pairs and empties. - 3. Creates a fresh sync branch upstream-sync/-- (per-run; no reuse). - 4. Cherry-picks one-by-one with Tier-0/Tier-1 auto-resolution. - On cherry-pick conflict -> Tier-3 stuck path (07). - 5. Post-batch HARD GATES (in order, before any push/PR): - a. Toolchain preflight (09) - missing toolset = infra stuck. - b. Static breakage scan (08) - duplicate resw / fork invariants. - c. Try-build (10) - razzle + bz no_clean. - Any failure -> Tier-4 stuck path (07b). - 6. Writes a transient report under `Generated Files/upstream-sync//` - (gitignored; never committed). - 7. On success -> pushes branch, opens PR (exit 0). - On Tier-3 -> pushes branch, opens labeled issue (exit 10). - On Tier-4 -> pushes branch, opens labeled issue (except infra), (exit 10). - On no-op -> exits 0 with a "no-op" report. - -.PARAMETER DryRun - Compute & report only; do not create the branch or pick anything. - -.PARAMETER TryTier2 - Reserved: enable LLM-assisted Tier-2 conflict resolution (NOT YET IMPLEMENTED). - -.PARAMETER Force - Override the stuck-lock (Tier-3 OR Tier-4). DANGEROUS - clobbers the - in-progress branch. Use only when you know the lock is stale. - -.PARAMETER MaxPicks - Cap the number of cherry-picks per run (default: unlimited). - -.PARAMETER PushDirectToMain - Skip the PR and fast-forward main directly to the sync branch tip. - Requires push permission on main. - -.PARAMETER AutoMergeStrategy - PR mode only. After opening the PR, run `gh pr merge -- --auto`. - Allowed: 'rebase' (recommended), 'merge', or 'none' (default). - -.PARAMETER SkipStaticScan - Skip step 5b. Default: scan. Schedulers MUST run the scan. - -.PARAMETER SkipBuild - Skip steps 5a + 5c. Default: build. Schedulers MUST build. - -.PARAMETER AllowInconclusiveBuild - Don't treat a build timeout as Tier-4 stuck - proceed with a warning - in the report. Dev opt-in only; schedulers should leave it off so - hung builds don't escape into unproven PRs. - -.PARAMETER BuildTimeoutMinutes - Wall-clock cap for try-build. Default 45. - -.PARAMETER BuildCommand - Override the default build command (passed to cmd.exe). Default: - 'tools\razzle.cmd && bz no_clean'. - -.OUTPUTS - Writes status to stdout. Exit codes: - 0 = success (PR opened) OR no-op OR skipped-locked - 10 = stuck (Tier-3 or Tier-4) - NOT an error - 20 = hard failure (git/gh broken) - alarm-worthy -#> -[CmdletBinding()] -param( - [switch] $DryRun, - [switch] $TryTier2, - [switch] $Force, - [int] $MaxPicks = 0, - [switch] $PushDirectToMain, - [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none', - [switch] $SkipStaticScan, - [switch] $SkipBuild, - [switch] $AllowInconclusiveBuild, - [int] $BuildTimeoutMinutes = 45, - [string] $BuildCommand = 'tools\razzle.cmd && bz no_clean' -) - -. "$PSScriptRoot/Common.ps1" - -function Exit-Hard([string] $msg) { - Write-Error $msg - exit 20 -} - -function Invoke-Tier4Stuck { - param( - $Ctx, - [string] $Kind, - [string] $FromSha, - [string] $ToSha - ) - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $Ctx -From $FromSha -To $ToSha -Status "stuck-$Kind" - $Ctx.ReportPath = $reportPath - Write-Host "Tier-4 stuck report: $reportPath" - $issueUrl = & "$PSScriptRoot/07b-open-validation-stuck-issue.ps1" -Ctx $Ctx -ReportPath $reportPath -Kind $Kind - if ($issueUrl) { Write-Host "Stuck issue: $issueUrl" -ForegroundColor Yellow } - exit 10 -} - -try { - $ctx = New-RunContext - - # Fast-forward local main from origin BEFORE any state-derivation calls - # so Get-LastSyncedUpstreamSha / Get-PendingUpstreamShas see the - # authoritative refs. A stale local clone would otherwise compute a - # wrong pending list (or repeat picks already on origin/main from a - # concurrent run on another host). Worktree cleanliness is checked - # first so an unrelated dirty file can't block the FF mid-script. - Assert-CleanWorktree - git switch main 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed." } - git pull --ff-only origin main 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git pull --ff-only origin main failed." } - - # --- Stuck-lock gate --- - # Derived from open `upstream-sync-stuck` labeled issues. Any open - # issue with that label blocks the scheduler until a human closes it - # (the close acts as the "lock cleared" signal - no clear-stuck.ps1 - # needed). The gate ALSO needs `upstream` fetched so that the report's - # range / watermark fields can be computed even when we skip. - Ensure-UpstreamRemote - git fetch upstream main --no-tags 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git fetch upstream main failed." } - - if (-not $Force) { - $stuck = Get-StuckIssues - if ($stuck.Count -gt 0) { - $first = $stuck[0] - $meta = Get-StuckMetaFromIssue -Issue $first - $lockDesc = if ($meta -and ($meta.PSObject.Properties.Name -contains 'tier')) { - "$($meta.tier) at $($first.url)" - } else { - "labeled issue $($first.url)" - } - Write-Host "Stuck-lock set ($lockDesc). Skipping. Close the issue to clear the lock." -ForegroundColor Yellow - $fromSha = try { Get-LastSyncedUpstreamSha } catch { '(unknown)' } - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $fromSha -Status 'skipped-locked' - Write-Host "Skip report: $reportPath" - exit 0 - } - } - - # --- Existing-PR gate --- - # Don't open a second concurrent upstream-sync PR. Same stderr-temp-file - # pattern as everywhere else: a gh banner on stderr must not be merged - # into stdout (would break ConvertFrom-Json on the JSON payload). - if (-not $Force -and -not $PushDirectToMain -and -not $DryRun) { - $errFile = [System.IO.Path]::GetTempFileName() - $existingJson = $null - try { - $existingJson = gh pr list --repo microsoft/intelligent-terminal --state open --limit 200 --json number,headRefName,url 2>$errFile - $ghExit = $LASTEXITCODE - if ($ghExit -ne 0) { - $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } - Exit-Hard "gh pr list failed (exit $ghExit): $errText. The existing-PR gate requires gh to be installed and authenticated. Re-run with -Force to bypass (at your own risk), or with -DryRun / -PushDirectToMain to skip the gate." - } - } - finally { - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue - } - if ($existingJson) { - $existing = @($existingJson | ConvertFrom-Json) | Where-Object { $_.headRefName -like 'upstream-sync/*' } - if ($existing.Count -gt 0) { - $first = $existing[0] - Write-Host "An upstream-sync PR is already open: #$($first.number) ($($first.headRefName)) -> $($first.url). Skipping until it merges or is closed (use -Force to override)." -ForegroundColor Yellow - $fromSha = try { Get-LastSyncedUpstreamSha } catch { '(unknown)' } - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $fromSha -Status 'skipped-pr-open' - Write-Host "Skip report: $reportPath" - exit 0 - } - } - } - - Assert-CleanWorktree - - # --- 1. Resolve from/to (upstream already fetched above) --- - $toSha = (git rev-parse upstream/main).Trim() - if ($LASTEXITCODE -ne 0) { Exit-Hard "git rev-parse upstream/main failed." } - $fromSha = Get-LastSyncedUpstreamSha - - if ($toSha -eq $fromSha) { - Write-Host "Already at upstream HEAD ($toSha). No-op." -ForegroundColor Green - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' - Write-Host "No-op report: $reportPath" - exit 0 - } - - # --- 2. Compute pending --- - $pendingJson = & "$PSScriptRoot/02-compute-pending.ps1" - $pending = $pendingJson | ConvertFrom-Json - Write-Host ("Pending: {0} commits, {1} revert pairs dropped, {2} empties dropped." -f $pending.pending.Count, $pending.dropped_pairs.Count, $pending.skipped_empty.Count) - - $ctx.Pending = @($pending.pending) - $ctx.DroppedPairs = @($pending.dropped_pairs) - $ctx.SkippedEmpty = @($pending.skipped_empty) - - if ($pending.pending.Count -eq 0) { - Write-Host "Nothing to pick after filtering. Effective no-op." -ForegroundColor Green - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'no-op' - Write-Host "Report: $reportPath" - exit 0 - } - - if ($DryRun) { - Write-Host "DryRun: skipping branch creation and cherry-picks." -ForegroundColor Cyan - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'dry-run' - Write-Host "DryRun report: $reportPath" - exit 0 - } - - # Capture pre-pick base SHA (origin/main) - used as static-scan baseline. - $preBase = (git rev-parse origin/main).Trim() - if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not resolve origin/main for scan baseline." } - - # --- 3. Create / switch to sync branch --- - $branch = $ctx.Branch - git switch -c $branch 2>$null - if ($LASTEXITCODE -ne 0) { - git switch $branch 2>&1 | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "Could not create or switch to $branch." } - } - - # --- 4. Cherry-pick loop --- - $picks = $pending.pending - if ($MaxPicks -gt 0 -and $picks.Count -gt $MaxPicks) { $picks = $picks[0..($MaxPicks-1)] } - - foreach ($sha in $picks) { - Write-Host "" - Write-Host "=== Cherry-pick $sha ===" -ForegroundColor Cyan - $resJson = & "$PSScriptRoot/03-cherry-pick-one.ps1" -Sha $sha - $res = $resJson | ConvertFrom-Json - switch ($res.status) { - 'picked' { - $ctx.Picked += $sha - foreach ($p in @($res.tier0_paths)) { - $ctx.Tier0 += [pscustomobject] @{ Sha = $sha; Path = $p } - } - } - 'skipped-empty' { - $ctx.SkippedEmpty += $sha - } - 'stuck' { - $ctx.StuckSha = $sha - $ctx.StuckPaths = @($res.conflict_paths) - $ctx.StuckError = if ($res.PSObject.Properties.Name -contains 'error') { [string]$res.error } else { $null } - $ctx.Status = 'stuck' - $errSuffix = if ($ctx.StuckError) { " (error: $($ctx.StuckError))" } else { '' } - if ($ctx.StuckPaths.Count -gt 0) { - Write-Warning "Stuck at $sha on paths: $($ctx.StuckPaths -join ', ')$errSuffix" - } else { - Write-Warning "Stuck at $sha - no conflict paths reported$errSuffix" - } - break - } - default { Exit-Hard "Unknown cherry-pick-one status: $($res.status)" } - } - if ($ctx.Status -eq 'stuck') { break } - } - - # --- 5. Tier-3 short-circuit --- - if ($ctx.Status -eq 'stuck') { - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'stuck' - $ctx.ReportPath = $reportPath - Write-Host "Stuck report: $reportPath" - $issueUrl = & "$PSScriptRoot/07-open-stuck-issue.ps1" -Ctx $ctx -ReportPath $reportPath - Write-Host "Stuck issue: $issueUrl" -ForegroundColor Yellow - exit 10 - } - - # --- 5a. Toolchain preflight (Tier-4 gate: infra-missing) --- - if (-not $SkipBuild) { - Write-Host "" - Write-Host "=== Toolchain preflight ===" -ForegroundColor Cyan - $preflightJson = & "$PSScriptRoot/09-toolchain-preflight.ps1" - $ctx.Preflight = $preflightJson | ConvertFrom-Json - Write-Host "Required: $($ctx.Preflight.required_toolsets -join ', '); available: $($ctx.Preflight.available_toolsets -join ', ')" - if (-not $ctx.Preflight.ok) { - Write-Warning "Toolchain preflight FAILED - missing: $($ctx.Preflight.missing -join ', ')" - Invoke-Tier4Stuck -Ctx $ctx -Kind 'toolchain-missing' -FromSha $fromSha -ToSha $toSha - } - } - - # --- 5b. Static breakage scan (Tier-4 gate: scan-blocking) --- - if (-not $SkipStaticScan) { - Write-Host "" - Write-Host "=== Static breakage scan ===" -ForegroundColor Cyan - $scanJson = & "$PSScriptRoot/08-static-scan.ps1" -BaseSha $preBase -HeadRef 'HEAD' - $ctx.Scan = $scanJson | ConvertFrom-Json - $sm = $ctx.Scan.summary - Write-Host "Findings: critical=$($sm.critical), high=$($sm.high), medium=$($sm.medium), low=$($sm.low), info=$($sm.info); blocking=$($ctx.Scan.blocking)" - if ($ctx.Scan.blocking) { - Invoke-Tier4Stuck -Ctx $ctx -Kind 'static-scan' -FromSha $fromSha -ToSha $toSha - } - } - - # --- 5c. Try-build (Tier-4 gate: build-failed / build-inconclusive) --- - if (-not $SkipBuild) { - Write-Host "" - Write-Host "=== Try-build (timeout ${BuildTimeoutMinutes}m) ===" -ForegroundColor Cyan - $buildJson = & "$PSScriptRoot/10-try-build.ps1" -BuildCommand $BuildCommand -TimeoutMinutes $BuildTimeoutMinutes - $ctx.Build = $buildJson | ConvertFrom-Json - Write-Host "Build: $($ctx.Build.kind) (exit=$($ctx.Build.exit_code), duration=$([int]($ctx.Build.duration_ms / 1000))s)" - switch ($ctx.Build.kind) { - 'build-failed' { Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-failed' -FromSha $fromSha -ToSha $toSha } - 'build-inconclusive' { - if ($AllowInconclusiveBuild) { - Write-Warning "Build inconclusive - proceeding (--AllowInconclusiveBuild)." - } else { - Invoke-Tier4Stuck -Ctx $ctx -Kind 'build-inconclusive' -FromSha $fromSha -ToSha $toSha - } - } - 'build-ok' { Write-Host "Build OK." -ForegroundColor Green } - } - } - - # --- 6. Report + finalize --- - $ctx.Status = 'ok' - $reportPath = & "$PSScriptRoot/05-write-report.ps1" -Ctx $ctx -From $fromSha -To $toSha -Status 'ok' - $ctx.ReportPath = $reportPath - Write-Host "Report: $reportPath" - - if ($PushDirectToMain) { - # No more state.json -> no backfill commit needed. Just push the - # sync branch's commits directly onto main as a fast-forward. - git switch main | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git switch main failed before direct-push." } - git merge --ff-only $branch | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git merge --ff-only $branch failed (main moved during the run?)." } - git push origin main | Out-Host - if ($LASTEXITCODE -ne 0) { Exit-Hard "git push origin main failed; sync content is local only." } - $mainHead = (git rev-parse HEAD).Trim() - Write-Host "" - Write-Host ("[OK] Sync fast-forwarded onto main at " + $mainHead.Substring(0,9)) -ForegroundColor Green - exit 0 - } - - $prUrl = & "$PSScriptRoot/06-finalize-pr.ps1" -Ctx $ctx -To $toSha -ReportPath $reportPath -AutoMergeStrategy $AutoMergeStrategy - Write-Host "" - Write-Host "[OK] Sync PR opened: $prUrl" -ForegroundColor Green - exit 0 -} -catch { - Write-Error $_.Exception.Message - Write-Error $_.ScriptStackTrace - exit 20 -} diff --git a/.github/skills/upstream-sync/scripts/10-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 similarity index 57% rename from .github/skills/upstream-sync/scripts/10-try-build.ps1 rename to .github/skills/upstream-sync/scripts/04-try-build.ps1 index 66d1febe9..d9bb9f914 100644 --- a/.github/skills/upstream-sync/scripts/10-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -1,12 +1,17 @@ <# .SYNOPSIS - Try build. Runs the configured build command in a razzle environment - and captures the result. Default: `cmd /c "tools\razzle.cmd && bz no_clean"`. + Try-build. Runs the configured build command in a razzle environment and + captures the result. Default: `cmd /c "tools\razzle.cmd && bz no_clean"`. + +.DESCRIPTION + Run AFTER cherry-picking (03) and BEFORE finalizing the PR (05). If the + build fails, the agent commits a fix on the same sync branch so it lands + in the same PR — that is why try-build is step 04, not step 99. .PARAMETER BuildCommand - Override the default build command. Must be a string the cmd.exe - shell can execute (razzle is cmd-based; PowerShell-keyword chaining - like 'if' won't work — keep it cmd-shell-friendly with &&). + Override the default build command. Must be a string the cmd.exe shell + can execute (razzle is cmd-based; PowerShell-keyword chaining like 'if' + won't work — keep it cmd-shell-friendly with &&). .PARAMETER TimeoutMinutes Wall-clock cap. Default 45. On timeout, the build is killed and the @@ -23,36 +28,63 @@ "exit_code": , "duration_ms": , "command": "", - "log_path": "", + "log_path": "", "log_tail": "" } - - Exit / error model: - Stdout JSON on success (orchestrator path). - Throws on wrapper error (couldn't start the build at all). The - orchestrator (`04-run-batch.ps1`) catches and routes through its - own exit-code mapping (0 ok / 10 stuck / 20 error). When the script - is run standalone for debugging, an uncaught throw exits with - PowerShell's default code (1) and prints the stack trace. - `exit 20` is intentionally NOT used here: the script is invoked via - `& "$PSScriptRoot/10-try-build.ps1"`, and `exit` in that context - would terminate the orchestrator mid-pipeline. #> [CmdletBinding()] param( - [string] $BuildCommand = 'tools\razzle.cmd && bz no_clean', + [string] $BuildCommand = 'tools\razzle.cmd && bz no_clean', [int] $TimeoutMinutes = 45, [string] $LogDir ) . "$PSScriptRoot/Common.ps1" -try { - if (-not $LogDir) { - # Default: per-day, per-skill artifact dir under the gitignored - # `Generated Files/` root. Get-GeneratedDir creates it on demand. - $LogDir = Get-GeneratedDir -Sub 'build-logs' +# --- Inlined helpers (single-use; see Common.ps1 comment for why) ---------- + +function Get-RepoRoot { + $r = git rev-parse --show-toplevel 2>$null + if ($LASTEXITCODE -ne 0) { throw "Not inside a git repo." } + return $r.Trim() +} + +function Get-GeneratedDir { + # Per-skill, per-day artifact dir under the repo's gitignored + # `Generated Files/` root (matches the workspace convention; the repo's + # top-level .gitignore has `**/Generated Files/`). + param([string] $Sub) + $root = Get-RepoRoot + $date = (Get-Date).ToString('yyyy-MM-dd') + $path = Join-Path $root "Generated Files/upstream-sync/$date" + if ($Sub) { $path = Join-Path $path $Sub } + if (-not (Test-Path -LiteralPath $path)) { + New-Item -ItemType Directory -Path $path -Force | Out-Null } + return $path +} + +function ConvertTo-RepoRelativePath { + # Normalize to forward-slash, repo-relative form so callers can safely + # embed it in committed text without leaking machine-specific drive + # letters / user dirs. + param([Parameter(Mandatory)] [string] $Path) + $root = ((Get-RepoRoot) -replace '\\','/').TrimEnd('/') + $abs = $Path -replace '\\','/' + if ($abs.Equals($root, [System.StringComparison]::OrdinalIgnoreCase)) { + throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" + } + $prefix = "$root/" + if ($abs.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { + return $abs.Substring($prefix.Length) + } + throw "ConvertTo-RepoRelativePath: '$Path' is not under repo root '$root'." +} + +# --- Main logic ------------------------------------------------------------ + +try { + if (-not $LogDir) { $LogDir = Get-GeneratedDir -Sub 'build-logs' } $root = Get-RepoRoot if (-not (Test-Path -LiteralPath $LogDir)) { New-Item -ItemType Directory -Path $LogDir -Force | Out-Null } @@ -62,7 +94,6 @@ try { $cmdLine = "/c `"cd /d `"$root`" && $BuildCommand`"" $started = Get-Date - # Use Start-Process with redirection so we can both tail and tee. $psi = New-Object System.Diagnostics.ProcessStartInfo $psi.FileName = $env:ComSpec $psi.Arguments = $cmdLine @@ -72,12 +103,12 @@ try { $psi.UseShellExecute = $false $psi.CreateNoWindow = $true - $proc = [System.Diagnostics.Process]::Start($psi) + $proc = [System.Diagnostics.Process]::Start($psi) $baseWriter = $null $writer = $null try { - # Tee stdout/stderr into the log file as the build runs. The synchronized + # Tee stdout/stderr to the log file as the build runs. The synchronized # wrapper serializes concurrent stdout/stderr DataReceived callbacks. $baseWriter = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) $writer = [System.IO.TextWriter]::Synchronized($baseWriter) @@ -105,8 +136,6 @@ try { } } finally { - # Always release the log file handle and process — scheduler runs are - # unattended and a leaked handle would jam the next run's log write. if ($writer) { try { $writer.Flush() } catch {} try { $writer.Close() } catch {} } if ($baseWriter) { try { $baseWriter.Dispose() } catch {} } @@ -116,26 +145,20 @@ try { $ended = Get-Date $durationMs = [int]($ended - $started).TotalMilliseconds - # Capture the last ~200 lines for the report / stuck issue. $tailLines = if (Test-Path -LiteralPath $logPath) { @(Get-Content -LiteralPath $logPath -Tail 200) -join "`n" } else { '' } - # Emit a repo-relative log_path when the log lives inside the repo - # (the common case). Absolute paths leak machine-specific details - # like username + drive letter into GitHub issues/reports. Fall back - # to the absolute path when the user passed a custom -LogDir that - # sits outside the repo root. $logPathForReport = try { ConvertTo-RepoRelativePath $logPath } catch { $logPath } - $doc = [ordered] @{ + + [ordered] @{ kind = $kind exit_code = $exitCode duration_ms = $durationMs command = $BuildCommand log_path = $logPathForReport log_tail = $tailLines - } - $doc | ConvertTo-Json -Depth 4 + } | ConvertTo-Json -Depth 4 } catch { Write-Error $_.Exception.Message diff --git a/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 new file mode 100644 index 000000000..7d2fa2538 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 @@ -0,0 +1,111 @@ +<# +.SYNOPSIS + Push the sync branch and open a PR. No state file, no extra commits. + +.DESCRIPTION + The branch already carries the cherry-picked commits (each with its + `(cherry picked from commit )` trailer — that IS the watermark + the next run reads). We just push it and open the PR. + + Called by the agent after a clean cherry-pick batch (and build pass). + +.PARAMETER Branch + Sync branch name (must already exist locally and be checked out / pushable). + +.PARAMETER UpstreamHeadSha + Upstream/main SHA at fetch time. Used only in the PR title. + +.PARAMETER PickedCount + Number of commits cherry-picked in this batch. Used only in the banner. + +.PARAMETER PrBody + Full markdown body for the PR. The banner (squash-warning + review-fix + policy) is prepended automatically. + +.PARAMETER AutoMergeStrategy + rebase | merge | none. Passed to `gh pr merge --auto`. + +.OUTPUTS + PR URL on stdout. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $Branch, + [Parameter(Mandatory)] [string] $UpstreamHeadSha, + [Parameter(Mandatory)] [int] $PickedCount, + [Parameter(Mandatory)] [string] $PrBody, + [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' +) + +. "$PSScriptRoot/Common.ps1" + +$banner = @" +> [!WARNING] +> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked +> upstream commit into one, destroying per-commit attribution, original +> author dates, the ``(cherry picked from commit )`` trailers that the +> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. +> Merge with **"Rebase and merge"** (preferred — flat history, all +> $PickedCount commit(s) land individually) or **"Create a merge commit"**. + +> [!NOTE] +> **Review-fix policy.** Only build-blocking fixes belong on this branch +> as **one** focused extra commit. All other Copilot / human review +> feedback goes into a **follow-up PR** based on this PR's head. See +> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). + +--- + +"@ + +$bodyPath = New-TemporaryFile +$bodyContent = $banner + $PrBody +[System.IO.File]::WriteAllText($bodyPath, $bodyContent, (New-Object System.Text.UTF8Encoding($false))) + +$shortTo = $UpstreamHeadSha.Substring(0,9) +$title = "chore(upstream): sync microsoft/terminal up to $shortTo" + +git push -u origin $Branch | Out-Host +if ($LASTEXITCODE -ne 0) { + Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue + throw "git push failed for $Branch." +} + +# `gh pr create` on Windows occasionally fails with "Head sha can't be blank" +# right after a push — retry up to 3x with a short delay. stderr goes to a +# separate temp file so a `gh` version-update notice can't displace the +# URL as the "last line" of merged output. +$prUrl = $null +$errFile = [System.IO.Path]::GetTempFileName() +try { + for ($attempt = 1; $attempt -le 3; $attempt++) { + Set-Content -LiteralPath $errFile -Value '' -NoNewline + $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $Branch --title $title --body-file $bodyPath 2>$errFile | Select-Object -Last 1 + if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } + $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + Write-Warning "gh pr create attempt $attempt failed (exit $LASTEXITCODE): stdout='$prUrl' stderr='$errText'" + Start-Sleep -Seconds 5 + } + if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { + $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } + throw "gh pr create did not return a PR URL after 3 attempts. Last stdout: '$prUrl'. Last stderr: '$errText'." + } +} +finally { + Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue +} + +$prUrl = $prUrl.Trim() + +if ($AutoMergeStrategy -ne 'none') { + $strategyFlag = "--$AutoMergeStrategy" + gh pr merge -R microsoft/intelligent-terminal $prUrl $strategyFlag --auto --delete-branch | Out-Host + if ($LASTEXITCODE -ne 0) { + Write-Warning "gh pr merge --auto failed. PR is open at $prUrl; merge manually with '$AutoMergeStrategy' strategy (NOT squash)." + } else { + Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green + } +} + +return $prUrl diff --git a/.github/skills/upstream-sync/scripts/05-write-report.ps1 b/.github/skills/upstream-sync/scripts/05-write-report.ps1 deleted file mode 100644 index 485ead8fd..000000000 --- a/.github/skills/upstream-sync/scripts/05-write-report.ps1 +++ /dev/null @@ -1,277 +0,0 @@ -<# -.SYNOPSIS - Generate a sync run report markdown file under - `Generated Files/upstream-sync//` (gitignored). - -.DESCRIPTION - Reports are TRANSIENT artifacts: they live in the gitignored - `Generated Files/` workspace and are NEVER committed. They exist - so a human running the skill (or reviewing a stuck issue) can read - what happened in a single run. The stuck-issue body (07 / 07b) - inlines the relevant parts of the report so issue readers don't - need to fetch the local file. - -.PARAMETER Ctx - The run-context object built by 04-run-batch.ps1. - -.PARAMETER From - Baseline upstream SHA before the run (Get-LastSyncedUpstreamSha). - -.PARAMETER To - Upstream HEAD SHA at fetch time. - -.PARAMETER Status - ok | no-op | dry-run | stuck | skipped-locked | skipped-pr-open - | stuck-static-scan | stuck-build-failed | stuck-build-inconclusive - | stuck-toolchain-missing - -.OUTPUTS - Absolute path to the written report file. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] $Ctx, - [Parameter(Mandatory)] [string] $From, - [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [ValidateSet('ok','no-op','dry-run','stuck','skipped-locked','skipped-pr-open','stuck-static-scan','stuck-build-failed','stuck-build-inconclusive','stuck-toolchain-missing')] [string] $Status -) - -. "$PSScriptRoot/Common.ps1" - -$started = $Ctx.StartedAt -$ended = Get-Date -$dur = $ended - $started -$durStr = "{0}m {1}s" -f [int]$dur.TotalMinutes, ($dur.Seconds) - -function Get-Subj([string] $sha) { - if (-not $sha) { return '' } - try { return (git log -1 --format='%s' $sha 2>$null) } catch { return '' } -} - -$fromSubj = Get-Subj $From -$toSubj = Get-Subj $To - -$lines = New-Object System.Collections.Generic.List[string] -$lines.Add("# Upstream sync - $Status - $(Format-Iso8601 $started)") -$lines.Add("") -$lines.Add("**Status:** $Status ") -$lines.Add("**Host:** $($Ctx.Host) ") -$lines.Add("**Duration:** $durStr ") -$lines.Add("**Baseline (before run):** ``$From`` - $fromSubj ") -$lines.Add("**Upstream HEAD:** ``$To`` - $toSubj ") -$lines.Add("**Branch:** ``$($Ctx.Branch)`` ") -$lines.Add("") -$lines.Add("## Summary") -$lines.Add("") -$lines.Add("- Commits picked: **$($Ctx.Picked.Count)**") -$lines.Add("- Revert pairs dropped: **$($Ctx.DroppedPairs.Count)** (= $($Ctx.DroppedPairs.Count * 2) commits skipped, net zero)") -$lines.Add("- Upstream-empty commits skipped: **$($Ctx.SkippedEmpty.Count)**") -$lines.Add("- Tier-0 auto-resolutions: **$($Ctx.Tier0.Count)**") -$lines.Add("- Tier-2 LLM resolutions: **$($Ctx.Tier2.Count)**") -if ($Ctx.StuckSha) { - $lines.Add("- Tier-3 stuck at: ``$($Ctx.StuckSha)``") -} -$lines.Add("") - -if ($Status -eq 'dry-run' -and $Ctx.Pending.Count -gt 0) { - $lines.Add("## Pending commits (oldest -> newest)") - $lines.Add("") - $lines.Add("| # | SHA | Subject | Author |") - $lines.Add("|---|---|---|---|") - $i = 0 - foreach ($sha in $Ctx.Pending) { - $i++ - $s = (git log -1 --format='%s' $sha) -replace '\|','\|' - $a = git log -1 --format='%an' $sha - $lines.Add("| $i | ``$($sha.Substring(0,9))`` | $s | $a |") - } - $lines.Add("") -} - -if ($Ctx.Picked.Count -gt 0) { - $lines.Add("## Picked commits (oldest -> newest)") - $lines.Add("") - $lines.Add("| # | SHA | Subject | Author |") - $lines.Add("|---|---|---|---|") - $i = 0 - foreach ($sha in $Ctx.Picked) { - $i++ - $s = (git log -1 --format='%s' $sha) -replace '\|','\|' - $a = git log -1 --format='%an' $sha - $lines.Add("| $i | ``$($sha.Substring(0,9))`` | $s | $a |") - } - $lines.Add("") -} - -if ($Ctx.DroppedPairs.Count -gt 0) { - $lines.Add("## Dropped revert pairs") - $lines.Add("") - $lines.Add("| Original SHA | Original subject | Revert SHA |") - $lines.Add("|---|---|---|") - foreach ($pair in $Ctx.DroppedPairs) { - $os = (git log -1 --format='%s' $pair[0]) -replace '\|','\|' - $lines.Add("| ``$($pair[0].Substring(0,9))`` | $os | ``$($pair[1].Substring(0,9))`` |") - } - $lines.Add("") -} - -if ($Ctx.SkippedEmpty.Count -gt 0) { - $lines.Add("## Empty / no-op commits skipped") - $lines.Add("") - $lines.Add("| SHA | Subject |") - $lines.Add("|---|---|") - foreach ($sha in $Ctx.SkippedEmpty) { - $s = (git log -1 --format='%s' $sha) -replace '\|','\|' - $lines.Add("| ``$($sha.Substring(0,9))`` | $s |") - } - $lines.Add("") -} - -if ($Ctx.Tier0.Count -gt 0) { - $lines.Add("## Tier-0 auto-resolutions") - $lines.Add("") - $lines.Add("| Commit SHA | File |") - $lines.Add("|---|---|") - foreach ($r in $Ctx.Tier0) { - $lines.Add("| ``$($r.Sha.Substring(0,9))`` | ``$($r.Path)`` |") - } - $lines.Add("") -} - -if ($Status -eq 'stuck' -and $Ctx.StuckSha) { - $stuckSubj = Get-Subj $Ctx.StuckSha - $stuckAuthor = git log -1 --format='%an <%ae>' $Ctx.StuckSha - $lines.Add("## Conflict diagnostics") - $lines.Add("") - $lines.Add("**Conflicting commit:** [`$($Ctx.StuckSha)`](https://github.com/microsoft/terminal/commit/$($Ctx.StuckSha)) - $stuckSubj ") - $lines.Add("**Author:** $stuckAuthor") - $lines.Add("") - $stuckError = if ($Ctx.PSObject.Properties.Name -contains 'StuckError') { $Ctx.StuckError } else { $null } - if ($Ctx.StuckPaths -and $Ctx.StuckPaths.Count -gt 0) { - $lines.Add("**Files in conflict:**") - $lines.Add("") - foreach ($p in $Ctx.StuckPaths) { $lines.Add("- ``$p``") } - $lines.Add("") - } else { - $lines.Add("**No unmerged paths** - ``git cherry-pick`` failed for a reason other than a merge conflict (e.g. attempting to pick a merge commit without ``-m``, hook failure, or another non-conflict error).") - if ($stuckError) { - $lines.Add("") - $lines.Add("**Reported error:** ``$stuckError``") - } - $lines.Add("") - } - $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (push attempted - run ````git ls-remote --heads origin $($Ctx.Branch)```` to verify it landed)") - $lines.Add("") - $lines.Add("**How to resume:**") - $lines.Add("") - $lines.Add("1. ``git switch $($Ctx.Branch)``") - $lines.Add("2. Manually cherry-pick the stuck commit and resolve. Pin upstream identity/dates so the resolved commit matches the rest of this batch:") - $lines.Add(" ``````pwsh") - $lines.Add(" `$info = (git log -1 --pretty=format:'%an%x00%ae%x00%aI%x00%cn%x00%ce%x00%cI' $($Ctx.StuckSha)) -split [char]0") - $lines.Add(" `$env:GIT_AUTHOR_NAME=`$info[0]; `$env:GIT_AUTHOR_EMAIL=`$info[1]; `$env:GIT_AUTHOR_DATE=`$info[2]") - $lines.Add(" `$env:GIT_COMMITTER_NAME=`$info[3]; `$env:GIT_COMMITTER_EMAIL=`$info[4]; `$env:GIT_COMMITTER_DATE=`$info[5]") - $lines.Add(" git cherry-pick -x $($Ctx.StuckSha)") - $lines.Add(" # resolve conflicts, then:") - $lines.Add(" git add -A; git cherry-pick --continue --no-edit") - $lines.Add(" Remove-Item Env:GIT_AUTHOR_NAME,Env:GIT_AUTHOR_EMAIL,Env:GIT_AUTHOR_DATE,Env:GIT_COMMITTER_NAME,Env:GIT_COMMITTER_EMAIL,Env:GIT_COMMITTER_DATE -ErrorAction SilentlyContinue") - $lines.Add(" ``````") - $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual resolution for $($Ctx.StuckSha.Substring(0,9))``, merge it. The ``-x`` trailer is what the next sync run reads as the new watermark - **do not strip it.**") - $lines.Add("4. Close the stuck issue. That's the lock-cleared signal; no script needed.") - $lines.Add("5. The next scheduled sync derives the new watermark from your merged PR and resumes from the commit after this one.") - $lines.Add("") -} - -if ($Status -like 'stuck-*') { - $kind = $Status -replace '^stuck-','' - $lines.Add("## Validation diagnostics - Tier-4 ($kind)") - $lines.Add("") - $lines.Add("All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch validation step blocked the push.") - $lines.Add("") - switch ($kind) { - 'static-scan' { - $sm = $Ctx.Scan.summary - $lines.Add("**Static scan summary:** critical=$($sm.critical), high=$($sm.high), medium=$($sm.medium), low=$($sm.low), info=$($sm.info)") - $lines.Add("") - $blocking = @($Ctx.Scan.findings | Where-Object { $_.severity -in @('critical','high') }) - if ($blocking.Count -gt 0) { - $lines.Add("**Blocking findings:**") - $lines.Add("") - $lines.Add("| Severity | Kind | Where | Detail |") - $lines.Add("|---|---|---|---|") - foreach ($f in $blocking) { - $where = if ($f.path) { "``$($f.path)``" } else { '-' } - $findingKind = if ($f.check) { $f.check } elseif ($f.kind) { $f.kind } else { 'unknown' } - $detail = ($f | ConvertTo-Json -Compress -Depth 4) - $lines.Add("| $($f.severity) | $findingKind | $where | ``$detail`` |") - } - $lines.Add("") - } - } - 'build-failed' { - $lines.Add("**Build exit code:** $($Ctx.Build.exit_code) ") - $lines.Add("**Build duration:** $([int]($Ctx.Build.duration_ms / 1000))s ") - $lines.Add("**Build log:** ``$($Ctx.Build.log_path)``") - $lines.Add("") - $lines.Add("**Last lines of build log:**") - $lines.Add("") - $lines.Add('```') - $lines.Add(($Ctx.Build.log_tail -split "`n" | Select-Object -Last 80) -join "`n") - $lines.Add('```') - $lines.Add("") - } - 'build-inconclusive' { - $lines.Add("**Build hit timeout** after $([int]($Ctx.Build.duration_ms / 1000))s.") - $lines.Add("**Build log:** ``$($Ctx.Build.log_path)``") - $lines.Add("") - $lines.Add("If this was a legitimate hang, investigate. If it was a slow build host, re-run with ``-BuildTimeoutMinutes `` or ``-AllowInconclusiveBuild``.") - $lines.Add("") - } - 'toolchain-missing' { - $lines.Add("**Required toolsets:** $($Ctx.Preflight.required_toolsets -join ', ')") - $lines.Add("**Available toolsets:** $($Ctx.Preflight.available_toolsets -join ', ')") - $lines.Add("**Missing:** $($Ctx.Preflight.missing -join ', ')") - $lines.Add("") - $lines.Add("This is an **infrastructure** problem - provision the host with the required Visual Studio toolset(s). No GitHub issue was opened because PR review cannot fix it.") - $lines.Add("") - } - } - $lines.Add("**Pickup branch:** ``$($Ctx.Branch)`` (push attempted - run ````git ls-remote --heads origin $($Ctx.Branch)```` to verify it landed)") - $lines.Add("") - if ($kind -eq 'toolchain-missing') { - $lines.Add("**How to resume (infra-only - no PR needed):**") - $lines.Add("") - $lines.Add("1. Provision the host with the missing toolset(s) above, **or** rerun the sync from a correctly provisioned host.") - $lines.Add("2. No issue was opened, so there is no lock to clear - the next scheduler tick just retries.") - $lines.Add("3. The next scheduled sync re-runs the same range; nothing on ``$($Ctx.Branch)`` needs editing.") - $lines.Add("") - } else { - $lines.Add("**How to resume:**") - $lines.Add("") - $lines.Add("1. ``git switch $($Ctx.Branch)``") - $lines.Add("2. Fix the issue above (e.g. resw dedup, restored fork invariant, build fix).") - $lines.Add("3. Push and open a PR titled ``chore(upstream-sync): manual validation fix for $($Ctx.Branch)``, merge it (keep the ``-x`` trailers on every cherry-picked commit so the next run's watermark derivation still works).") - $lines.Add("4. Close the stuck issue. That's the lock-cleared signal; no script needed.") - $lines.Add("5. The next scheduled sync runs the same range - validation must pass before any PR is opened.") - $lines.Add("") - } -} - -$lines.Add("---") -$lines.Add("") -$lines.Add("_Generated by ``.github/skills/upstream-sync/scripts/05-write-report.ps1``. This file lives under `Generated Files/` and is gitignored - never committed._") - -$suffix = if ($Status -eq 'skipped-locked') { 'skipped' } - elseif ($Status -eq 'skipped-pr-open') { 'skipped' } - elseif ($Status -eq 'dry-run') { 'dry-run' } - elseif ($Status -like 'stuck-*') { $Status } - elseif ($Status -eq 'stuck') { 'stuck' } - elseif ($Status -eq 'no-op') { 'noop' } - else { 'ok' } - -# Filenames are sortable by run timestamp + disambiguated by status. The -# parent directory (Generated Files/upstream-sync//) is already per-day. -$stamp = $started.ToString('yyyy-MM-ddTHHmmss') -$name = "$stamp-$suffix.md" -$path = Join-Path (Get-GeneratedDir) $name -[System.IO.File]::WriteAllText($path, ($lines -join "`n"), (New-Object System.Text.UTF8Encoding($false))) -return $path diff --git a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 deleted file mode 100644 index fe131a287..000000000 --- a/.github/skills/upstream-sync/scripts/06-finalize-pr.ps1 +++ /dev/null @@ -1,121 +0,0 @@ -<# -.SYNOPSIS - Push the sync branch and open a PR. No state file, no extra commits. - -.DESCRIPTION - The branch already carries the cherry-picked commits (each with its - `(cherry picked from commit )` trailer - that IS the watermark - the next run reads). We just push it and open the PR. No state.json - to commit, no pr_url backfill commit, no extra round-trip after PR - creation. - -.PARAMETER Ctx - Run context from 04-run-batch.ps1. - -.PARAMETER To - Upstream HEAD SHA at fetch time (used only in the PR title). - -.PARAMETER ReportPath - Absolute path to the report markdown to use as the PR body. The report - itself is NOT committed - just inlined into the PR body text. - -.PARAMETER AutoMergeStrategy - rebase | merge | none. Passed to `gh pr merge --auto`. - -.OUTPUTS - PR URL on stdout (and writes Ctx.PrUrl). -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] $Ctx, - [Parameter(Mandatory)] [string] $To, - [Parameter(Mandatory)] [string] $ReportPath, - [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' -) - -. "$PSScriptRoot/Common.ps1" - -# Prepend the squash-warning + review-policy banner to the report so it -# lands as the first thing reviewers see. -$banner = @" -> [!WARNING] -> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked -> upstream commit into one, destroying per-commit attribution, original -> author dates, the ``(cherry picked from commit )`` trailers that the -> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. -> Merge with **"Rebase and merge"** (preferred - flat history, all -> $($Ctx.Picked.Count) commits land individually) or **"Create a merge -> commit"** (also preserves per-commit content). - -> [!NOTE] -> **Review-fix policy.** Only build-blocking fixes (compile errors, dedup -> of conflicts surfaced at build time, CI gate failures on this PR itself) -> belong here - as **one** focused extra commit on this branch. All other -> Copilot / human review feedback (code-quality, logic, translation, -> spelling-list migrations, doc nits) goes into a **follow-up PR** based on -> this PR's head. Rationale and mechanics: -> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). - ---- - -"@ -$bodyPath = New-TemporaryFile -$bodyContent = $banner + (Get-Content -Raw -LiteralPath $ReportPath) -[System.IO.File]::WriteAllText($bodyPath, $bodyContent, (New-Object System.Text.UTF8Encoding($false))) - -$branch = $Ctx.Branch -$shortTo = $To.Substring(0,9) - -# Push the sync branch (cherry-pick commits already have their `-x` -# trailers; those trailers ARE the watermark - nothing else to commit). -git push -u origin $branch | Out-Host -if ($LASTEXITCODE -ne 0) { - Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue - throw "git push failed for $branch." -} - -$title = "chore(upstream): sync microsoft/terminal up to $shortTo" - -# Same-repo PR: `--head` takes the bare branch name. Retry up to 3 times -# with a short delay - `gh pr create` on Windows occasionally fails with -# "Head sha can't be blank" right after a push. -$prUrl = $null -$errFile = [System.IO.Path]::GetTempFileName() -try { - for ($attempt = 1; $attempt -le 3; $attempt++) { - # Capture stderr to a separate temp file: a `gh` version-update / - # deprecation notice on stderr can otherwise become the last line - # of merged output, breaking URL match. - Set-Content -LiteralPath $errFile -Value '' -NoNewline - $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $branch --title $title --body-file $bodyPath 2>$errFile | Select-Object -Last 1 - if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } - $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } - Write-Warning "gh pr create attempt $attempt failed (exit $LASTEXITCODE): stdout='$prUrl' stderr='$errText'" - Start-Sleep -Seconds 5 - } - if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { - $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } - throw "gh pr create did not return a PR URL after 3 attempts. Last stdout: '$prUrl'. Last stderr: '$errText'." - } -} -finally { - Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} - -$Ctx.PrUrl = $prUrl.Trim() - -# Optional: arm GitHub auto-merge with a strategy that preserves per-commit -# history. 'rebase' is the recommended default - it lands all N commits -# flatly on main once CI + approvals pass. Never squash. -if ($AutoMergeStrategy -ne 'none') { - $strategyFlag = "--$AutoMergeStrategy" - gh pr merge -R microsoft/intelligent-terminal $Ctx.PrUrl $strategyFlag --auto --delete-branch | Out-Host - if ($LASTEXITCODE -ne 0) { - Write-Warning "gh pr merge --auto failed. PR is open at $($Ctx.PrUrl); merge manually with '$AutoMergeStrategy' strategy (NOT squash)." - } else { - Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green - } -} - -return $Ctx.PrUrl diff --git a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 new file mode 100644 index 000000000..f733c5660 --- /dev/null +++ b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 @@ -0,0 +1,116 @@ +<# +.SYNOPSIS + Open a Tier-3 stuck issue (cherry-pick stopped at a real merge conflict). + +.DESCRIPTION + The "lock" IS this open issue — the next sync run queries + `gh issue list --label upstream-sync-stuck --state open` and bails when + one exists. The issue body carries a fenced ```yaml # wta-state``` block + with enough metadata to recognize the same failure on a re-run. + + Cleared by: a human closes the issue. That's it. + +.PARAMETER Branch + Stuck sync branch name. + +.PARAMETER StuckSha + The upstream SHA the pick got stuck on. + +.PARAMETER ConflictPaths + Files that 03-cherry-pick-one.ps1 left unmerged. + +.PARAMETER StuckError + Optional error string captured from 03 (e.g. "git cherry-pick exited 1 + with no conflict paths" for a non-conflict failure). + +.PARAMETER ExtraBody + Optional markdown to append after the YAML block (resume instructions, + log excerpts, etc.). + +.OUTPUTS + Issue URL on stdout. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $Branch, + [Parameter(Mandatory)] [string] $StuckSha, + [string[]] $ConflictPaths = @(), + [string] $StuckError = '', + [string] $ExtraBody = '' +) + +. "$PSScriptRoot/Common.ps1" + +# Push the stuck branch so the human can resume on it. +git push -u origin $Branch 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } + +$shortSha = $StuckSha.Substring(0,9) +$subj = (git log -1 --format='%s' $StuckSha).Trim() +$title = "Upstream sync stuck at ${shortSha}: $subj" + +$yamlBlock = Format-StuckYamlBlock @{ + tier = '3' + kind = 'cherry-pick-conflict' + stuck_on_sha = $StuckSha + branch = $Branch + at = Format-Iso8601 + host = $env:COMPUTERNAME + conflict_count = $ConflictPaths.Count + error = $StuckError +} + +$conflictList = if ($ConflictPaths.Count -gt 0) { + ($ConflictPaths | ForEach-Object { "- ``$_``" }) -join "`n" +} else { + '_(no conflict paths reported — see error)_' +} + +$body = @" +> [!CAUTION] +> **Upstream sync stopped at a conflict that needs human judgment.** +> +> The scheduler will keep skipping its runs until this issue is **closed**. +> Closing the issue IS the lock-clear signal — no separate script needed. + +**Stuck on:** ``$StuckSha`` — $subj +**Sync branch:** ``$Branch`` (push attempted — run ``git ls-remote --heads origin $Branch`` to verify it landed) + +**Conflicting paths:** +$conflictList + +$yamlBlock + +--- + +$ExtraBody +"@ + +$tmp = New-TemporaryFile +[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) + +# Ensure label exists (best-effort). -R pinned because an `upstream` remote +# can make `gh` default to microsoft/terminal where this account has no +# label-create permission. +gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null + +# Capture stderr to a separate temp file so a `gh` warning on stderr can't +# displace the URL as the last line of merged output. +$errFile = [System.IO.Path]::GetTempFileName() +$errText = '' +$issueUrl = $null +$ghExit = 0 +try { + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } +} +finally { + Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue +} +if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" +} + +return $issueUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 new file mode 100644 index 000000000..444f472ac --- /dev/null +++ b/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 @@ -0,0 +1,141 @@ +<# +.SYNOPSIS + Open a Tier-4 stuck issue (build failed after a clean cherry-pick batch). + +.DESCRIPTION + Counterpart to 06-open-stuck-issue.ps1 (Tier-3 = mid-pick conflict). + Tier-4 means all picks completed cleanly but `04-try-build.ps1` said NO + and the agent couldn't auto-fix. + + Same lock model: the open issue IS the lock; closing it clears. + +.PARAMETER Branch + Sync branch name. + +.PARAMETER Kind + 'build-failed' or 'build-inconclusive'. + +.PARAMETER PickedCount + How many commits landed cleanly before the build failed. + +.PARAMETER BuildExitCode + Exit code from 04-try-build.ps1. + +.PARAMETER BuildLogTail + Tail of the build log to embed in the issue body (last ~200 lines). + +.PARAMETER BuildLogPath + Repo-relative path to the full build log (gitignored — for operator + reference, not committed). + +.OUTPUTS + Issue URL on stdout. +#> +[CmdletBinding()] +param( + [Parameter(Mandatory)] [string] $Branch, + [Parameter(Mandatory)] [ValidateSet('build-failed','build-inconclusive')] [string] $Kind, + [Parameter(Mandatory)] [int] $PickedCount, + [Parameter(Mandatory)] [int] $BuildExitCode, + [string] $BuildLogTail = '', + [string] $BuildLogPath = '' +) + +. "$PSScriptRoot/Common.ps1" + +# Findings hash — stable across runs of the same broken batch so a future +# scheduler tick can recognize "same failure as last time" and skip +# re-opening. Inlined (single use). +function Get-FindingsHash { + param([Parameter(Mandatory)] $Findings) + $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) + $sha = [System.Security.Cryptography.SHA256]::Create() + try { + $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) + return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) + } finally { $sha.Dispose() } +} + +$findingsForHash = if ($Kind -eq 'build-failed') { + @([ordered] @{ exit_code = $BuildExitCode; tail_excerpt = ($BuildLogTail -split "`n" | Select-Object -Last 20) -join "`n" }) +} else { + @([ordered] @{ kind = 'inconclusive'; exit_code = $BuildExitCode }) +} +$findingsHash = Get-FindingsHash $findingsForHash + +# Push the sync branch so the human can resume on it. +git push -u origin $Branch 2>&1 | Out-Host +if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push sync branch — issue still being filed for visibility." } + +$titleKindLabel = if ($Kind -eq 'build-failed') { 'build failure' } else { 'build inconclusive (timeout)' } +$title = "Upstream sync stuck after $PickedCount clean picks: $titleKindLabel ($findingsHash)" + +$yamlBlock = Format-StuckYamlBlock @{ + tier = '4' + kind = $Kind + branch = $Branch + findings_hash = $findingsHash + picked_count = $PickedCount + at = Format-Iso8601 + host = $env:COMPUTERNAME +} + +$logSection = if ($BuildLogTail) { + @" +

Build log tail (last ~200 lines) + +`````` +$BuildLogTail +`````` + +
+"@ +} else { '' } + +$logPathLine = if ($BuildLogPath) { + "**Full log:** ``$BuildLogPath`` (gitignored; on the host that ran the build)" +} else { '' } + +$body = @" +> [!CAUTION] +> **Upstream sync stopped after build failed.** +> +> All $PickedCount cherry-pick(s) applied cleanly, but ``04-try-build.ps1`` +> said NO before the PR could be finalized. Stop reason: **$Kind** (exit $BuildExitCode). +> +> The scheduler will keep skipping its runs until this issue is **closed**. + +**Sync branch:** ``$Branch`` (push attempted — run ``git ls-remote --heads origin $Branch`` to verify it landed) +**Findings hash:** ``$findingsHash`` (re-runs of the same broken batch will match) +$logPathLine + +$yamlBlock + +--- + +$logSection +"@ + +$tmp = New-TemporaryFile +[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) + +gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null + +$errFile = [System.IO.Path]::GetTempFileName() +$errText = '' +$issueUrl = $null +$ghExit = 0 +try { + $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 + $ghExit = $LASTEXITCODE + if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } +} +finally { + Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue + Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue +} +if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { + throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" +} + +return $issueUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 deleted file mode 100644 index c99e1f741..000000000 --- a/.github/skills/upstream-sync/scripts/07-open-stuck-issue.ps1 +++ /dev/null @@ -1,105 +0,0 @@ -<# -.SYNOPSIS - Tier-3 (cherry-pick stopped at a real merge conflict) stuck-issue opener. - -.DESCRIPTION - No state.json. The "lock" is the OPEN labeled issue itself - the next - scheduler run calls Get-StuckIssues and bails when this issue is found. - The issue body carries a fenced ```yaml # wta-state``` block with the - same metadata the old state.json held (tier, stuck_on_sha, branch, - findings_hash) so re-runs can reason about it. - - Cleared by: a human closes the issue. That's it - no separate script. - -.PARAMETER Ctx - Run context (must have StuckSha, StuckPaths, Branch set). - -.PARAMETER ReportPath - Absolute path to the stuck report markdown. - -.OUTPUTS - Issue URL on stdout (and writes Ctx.IssueUrl). -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] $Ctx, - [Parameter(Mandatory)] [string] $ReportPath -) - -. "$PSScriptRoot/Common.ps1" - -if (-not $Ctx.StuckSha) { throw "Ctx.StuckSha is empty - nothing to escalate." } - -# Push the stuck branch so the human can resume on it. -git push -u origin $Ctx.Branch 2>&1 | Out-Host -if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch - issue still being filed for visibility." } - -$shortSha = $Ctx.StuckSha.Substring(0,9) -$subj = (git log -1 --format='%s' $Ctx.StuckSha).Trim() -$title = "Upstream sync stuck at ${shortSha}: $subj" - -# Build the lock-state YAML block. Schedulers read this on the next run -# via Get-StuckMetaFromIssue - it carries enough metadata to recognize a -# re-issue of the same failure without re-spamming a new ticket. -$stuckErrorVal = if ($Ctx.PSObject.Properties.Name -contains 'StuckError' -and $Ctx.StuckError) { $Ctx.StuckError } else { '' } -$yamlBlock = Format-StuckYamlBlock @{ - tier = '3' - kind = 'cherry-pick-conflict' - stuck_on_sha = $Ctx.StuckSha - branch = $Ctx.Branch - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host - conflict_count = ($Ctx.StuckPaths | Measure-Object).Count - error = $stuckErrorVal -} - -$header = @" -> [!CAUTION] -> **Upstream sync stopped at a conflict that needs human judgment.** -> -> The scheduler will keep skipping its runs until this issue is **closed**. -> Closing the issue IS the lock-clear signal - no separate script needed. - -**How to unblock:** follow "How to resume" in the report excerpt below, -merge your manual-resolution PR (keeping the ``(cherry picked from commit -)`` trailer - that's what the next sync run reads as its watermark), -then close this issue. - -$yamlBlock - ---- - -"@ -$body = $header + (Get-Content -Raw -LiteralPath $ReportPath) -$tmp = New-TemporaryFile -[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - -# Ensure label exists (best-effort). -R pinned because an `upstream` remote -# can make `gh` default to microsoft/terminal where this account has no -# label-create permission. -gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null - -# Capture stderr to a separate temp file so a `gh` warning on stderr can't -# displace the URL as the "last line" of merged output. -$errFile = [System.IO.Path]::GetTempFileName() -$errText = '' -$issueUrl = $null -$ghExit = 0 -try { - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } -} -finally { - Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} -if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" -} -$Ctx.IssueUrl = $issueUrl.Trim() - -# That's it. The open labeled issue IS the lock - no state file to write, -# no main-branch commit, nothing to push. The next scheduler run will see -# the open issue via Get-StuckIssues and skip. -return $Ctx.IssueUrl diff --git a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 deleted file mode 100644 index 0b09818ca..000000000 --- a/.github/skills/upstream-sync/scripts/07b-open-validation-stuck-issue.ps1 +++ /dev/null @@ -1,147 +0,0 @@ -<# -.SYNOPSIS - Tier-4 (post-pick validation failure) stuck-issue opener. - -.DESCRIPTION - Counterpart to 07-open-stuck-issue.ps1 (which handles Tier-3 = cherry-pick - stopped mid-pick on a real merge conflict). Tier-4 means all picks - completed cleanly but the static scan, toolchain preflight, or try-build - step said NO. - - No state.json - the lock is the open labeled issue itself (cleared by - the human closing the issue). The fenced ```yaml # wta-state``` block - in the body carries findings_hash so re-runs of the same broken batch - can be matched against the same issue. - - `toolchain-missing` is special: it's an INFRA problem (this host lacks - the required VS toolset), not a CODE problem. We do not open a GitHub - issue for it - issues are noise for things humans can't fix via the PR - surface. The scheduler simply retries next tick from another host (or - after provisioning). - -.PARAMETER Ctx - Run context. Must have Branch set; uses Picked, Preflight, Scan, Build. - -.PARAMETER ReportPath - Absolute path to the stuck report markdown. - -.PARAMETER Kind - One of: 'static-scan', 'build-failed', 'build-inconclusive', - 'toolchain-missing'. Determines the issue title and report header. - -.OUTPUTS - Issue URL on stdout (and writes Ctx.IssueUrl + Ctx.StuckValidation). - For toolchain-missing, returns $null. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] $Ctx, - [Parameter(Mandatory)] [string] $ReportPath, - [Parameter(Mandatory)] [ValidateSet('static-scan','build-failed','build-inconclusive','toolchain-missing')] [string] $Kind -) - -. "$PSScriptRoot/Common.ps1" - -# Compute a findings hash so re-runs of the same broken batch are detectable -# from a future run's gh issue body. Use [ordered] hashtables so JSON -# serialization (and therefore the hash) is stable across runs. -$findingsForHash = switch ($Kind) { - 'static-scan' { $Ctx.Scan.findings } - 'build-failed' { @([ordered] @{ exit_code = $Ctx.Build.exit_code; tail_excerpt = ($Ctx.Build.log_tail -split "`n" | Select-Object -Last 20) -join "`n" }) } - 'build-inconclusive' { @([ordered] @{ kind = 'inconclusive'; duration_ms = $Ctx.Build.duration_ms }) } - 'toolchain-missing' { @([ordered] @{ missing = @($Ctx.Preflight.missing | Sort-Object) }) } -} -$findingsHash = Get-FindingsHash $findingsForHash - -# Push the sync branch so the human can resume on it (even toolchain-missing - -# the picks are still useful artifacts for whoever owns the host). -git push -u origin $Ctx.Branch 2>&1 | Out-Host -if ($LASTEXITCODE -ne 0) { - Write-Warning "Could not push sync branch - issue will still be filed for visibility (when applicable)." -} - -# Tier-4 metadata stashed on the context for the orchestrator's logs. -$validation = [ordered] @{ - kind = $Kind - branch = $Ctx.Branch - range = @($Ctx.Picked) - findings_hash = $findingsHash - at = Format-Iso8601 $Ctx.StartedAt - issue_url = $null -} - -# For toolchain-missing we do NOT open an issue (infra problem, not code - -# and no lock either: the next tick simply retries from any properly -# provisioned host). -if ($Kind -eq 'toolchain-missing') { - $Ctx.StuckValidation = $validation - return $null -} - -$titleKindLabel = switch ($Kind) { - 'static-scan' { 'static scan' } - 'build-failed' { 'build failure' } - 'build-inconclusive' { 'build inconclusive (timeout)' } -} -$title = "Upstream sync stuck after $($Ctx.Picked.Count) clean picks: $titleKindLabel ($findingsHash)" - -$yamlBlock = Format-StuckYamlBlock @{ - tier = '4' - kind = $Kind - branch = $Ctx.Branch - findings_hash = $findingsHash - picked_count = $Ctx.Picked.Count - at = Format-Iso8601 $Ctx.StartedAt - host = $Ctx.Host -} - -$header = @" -> [!CAUTION] -> **Upstream sync stopped after validation failed.** -> -> All $($Ctx.Picked.Count) cherry-pick(s) applied cleanly, but the post-batch -> validation step said NO before any PR was opened. Stop reason: **$Kind**. -> -> The scheduler will keep skipping its runs until this issue is **closed**. -> Closing the issue IS the lock-clear signal - no separate script needed. - -Sync branch: ``$($Ctx.Branch)`` (push attempted - run ``git ls-remote --heads origin $($Ctx.Branch)`` to verify it landed). -Findings hash: ``$findingsHash`` (re-runs of the same broken batch will match). - -$yamlBlock - ---- - -"@ -$body = $header + (Get-Content -Raw -LiteralPath $ReportPath) -$tmp = New-TemporaryFile -[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - -# Ensure label exists (best-effort). -R pinned for the same reason as the -# issue-create call below (avoid the `upstream` remote tricking gh into -# microsoft/terminal). -gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null - -# Capture stderr to a separate temp file so a `gh` warning on stderr can't -# displace the URL as the "last line" of merged output. -$errFile = [System.IO.Path]::GetTempFileName() -$errText = '' -$issueUrl = $null -$ghExit = 0 -try { - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } -} -finally { - Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} -if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" -} -$validation.issue_url = $issueUrl.Trim() -$Ctx.IssueUrl = $validation.issue_url -$Ctx.StuckValidation = $validation - -return $validation.issue_url diff --git a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 b/.github/skills/upstream-sync/scripts/08-static-scan.ps1 deleted file mode 100644 index 8f3fce73f..000000000 --- a/.github/skills/upstream-sync/scripts/08-static-scan.ps1 +++ /dev/null @@ -1,195 +0,0 @@ -<# -.SYNOPSIS - Static breakage scan. Runs AFTER all cherry-picks succeed and BEFORE - push / PR creation. Catches "clean cherry-pick but broken content" - failures that git-level conflict detection misses (PR #220 audit). - -.DESCRIPTION - Two v1 checks (see references/static-scan.md for v2 deferred items): - - 1. Duplicate entries in *.resw files (baseline-diff — - only gates on NEW duplicates introduced by the pick range). - - 2. Fork invariants — regex patterns from references/fork-invariants.json - that must still match in the post-pick worktree. - -.PARAMETER BaseSha - Pre-pick base commit (usually origin/main at orchestrator start). Used - to compute baseline-diff for the resw check. Required. - -.PARAMETER HeadRef - Post-pick worktree ref (default: HEAD). - -.OUTPUTS - Emits a single JSON document on stdout. - - Error model: - Throws on wrapper error (broken script, missing files, etc.). The - orchestrator (`04-run-batch.ps1`) catches and routes through its - own exit-code mapping (0 ok / 10 stuck / 20 error). Run standalone, - an uncaught throw exits with PowerShell's default code (1) plus a - stack trace. `exit 20` is intentionally NOT used here because this - script is invoked via `&` from the orchestrator — `exit` in that - context would terminate the orchestrator mid-pipeline. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] [string] $BaseSha, - [string] $HeadRef = 'HEAD' -) - -. "$PSScriptRoot/Common.ps1" - -function Get-ReswDuplicateNames { - param([string] $Text) - if (-not $Text) { return @() } - $names = [System.Collections.Generic.List[string]]::new() - $re = [regex]'$null - if ($LASTEXITCODE -ne 0) { throw "git diff failed computing changed resw files." } - return @($out | Where-Object { $_ -like '*.resw' }) -} - -function Get-FileTextAtRef { - param([string] $Ref, [string] $Path) - # Capture via a temp file to avoid PowerShell mangling binary-ish output - # (UTF-8 BOM, mixed CRLF/LF, high-Unicode pseudo-locale glyphs) when the - # subprocess's stdout is bound to a PSObject pipeline. - $tmp = [System.IO.Path]::GetTempFileName() - try { - & cmd /c "git show ""${Ref}:$Path"" > ""$tmp"" 2>nul" - if ($LASTEXITCODE -ne 0) { return $null } - return [System.IO.File]::ReadAllText($tmp) - } finally { - Remove-Item -LiteralPath $tmp -ErrorAction SilentlyContinue - } -} - -function Get-FileTextOnDisk { - param([string] $Path) - # IMPORTANT: [System.IO.File]::* APIs resolve relative paths against - # [Environment]::CurrentDirectory, NOT PowerShell's $PWD. Any relative - # path passed in here would silently read from the wrong worktree (the - # PR #220 audit miss). Resolve to absolute against the repo root. - if ([System.IO.Path]::IsPathRooted($Path)) { - $abs = $Path - } else { - $abs = Join-Path (Get-RepoRoot) $Path - } - if (-not (Test-Path -LiteralPath $abs)) { return $null } - return [System.IO.File]::ReadAllText($abs) -} - -function Scan-ReswDuplicates { - param([string] $Base, [string] $Head) - $findings = @() - foreach ($f in (Get-ChangedReswFiles -Base $Base -Head $Head)) { - $baseText = Get-FileTextAtRef -Ref $Base -Path $f - $headText = if ($Head -eq 'HEAD') { Get-FileTextOnDisk -Path $f } else { Get-FileTextAtRef -Ref $Head -Path $f } - $baseDups = @(Get-ReswDuplicateNames -Text $baseText) - $headDups = @(Get-ReswDuplicateNames -Text $headText) - $newDups = @($headDups | Where-Object { $baseDups -notcontains $_ }) - $oldStill = @($headDups | Where-Object { $baseDups -contains $_ }) - if ($newDups.Count -gt 0) { - $findings += [ordered] @{ - check = 'resw-duplicate-keys' - severity = 'critical' - path = $f - detail = "$($newDups.Count) newly-duplicated entries (was $($baseDups.Count) at base)" - examples = @($newDups | Select-Object -First 5) - } - } - if ($oldStill.Count -gt 0) { - $findings += [ordered] @{ - check = 'resw-duplicate-keys' - severity = 'info' - path = $f - detail = "$($oldStill.Count) duplicate entries also present pre-pick (not blocking)" - examples = @($oldStill | Select-Object -First 5) - } - } - } - return ,$findings -} - -function Scan-ForkInvariants { - $findings = @() - $invPath = Join-Path (Split-Path -Parent $PSScriptRoot) 'references/fork-invariants.json' - if (-not (Test-Path -LiteralPath $invPath)) { - return ,@([ordered] @{ - check = 'fork-invariants' - severity = 'medium' - path = $invPath - detail = 'fork-invariants.json missing — cannot check fork-protected items' - }) - } - $doc = Get-Content -Raw -LiteralPath $invPath | ConvertFrom-Json - foreach ($inv in @($doc.invariants)) { - $absPath = Join-Path (Get-RepoRoot) $inv.path - if (-not (Test-Path -LiteralPath $absPath)) { - $findings += [ordered] @{ - check = 'fork-invariant' - severity = $inv.severity - id = $inv.id - path = $inv.path - detail = "protected file does not exist in worktree" - reason = $inv.reason - } - continue - } - $text = [System.IO.File]::ReadAllText($absPath) - $re = [regex]::new($inv.must_contain_regex) - if (-not $re.IsMatch($text)) { - $findings += [ordered] @{ - check = 'fork-invariant' - severity = $inv.severity - id = $inv.id - path = $inv.path - detail = "regex '$($inv.must_contain_regex)' did not match in post-pick file" - reason = $inv.reason - } - } - } - return ,$findings -} - -try { - $findings = @() - $findings += Scan-ReswDuplicates -Base $BaseSha -Head $HeadRef - $findings += Scan-ForkInvariants - - $summary = [ordered] @{ - critical = @($findings | Where-Object { $_.severity -eq 'critical' }).Count - high = @($findings | Where-Object { $_.severity -eq 'high' }).Count - medium = @($findings | Where-Object { $_.severity -eq 'medium' }).Count - low = @($findings | Where-Object { $_.severity -eq 'low' }).Count - info = @($findings | Where-Object { $_.severity -eq 'info' }).Count - } - $blocking = ($summary.critical + $summary.high) -gt 0 - - $doc = [ordered] @{ - base = $BaseSha - head = $HeadRef - findings = @($findings) - summary = $summary - blocking = $blocking - } - $doc | ConvertTo-Json -Depth 8 -} -catch { - Write-Error $_.Exception.Message - Write-Error $_.ScriptStackTrace - throw -} diff --git a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 b/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 deleted file mode 100644 index fd31f264d..000000000 --- a/.github/skills/upstream-sync/scripts/09-toolchain-preflight.ps1 +++ /dev/null @@ -1,141 +0,0 @@ -<# -.SYNOPSIS - Toolchain preflight. Detects the required PlatformToolset from the - repo's build props and verifies it's installed on this host. - -.DESCRIPTION - Returns a JSON document. Used by the orchestrator BEFORE try-build - to distinguish "code broke the build" from "host doesn't have the - toolset". Critical for unattended schedulers — an infra problem - must not be filed as a code-stuck issue. - - Does NOT auto-bump v143→v145 or any other toolset. That recipe is - kept as a local-only developer workaround. - -.OUTPUTS - JSON to stdout: - { - "required_toolsets": ["v143"], - "available_toolsets": ["v143", "v145"], - "missing": [], - "vs_installs": ["C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise"], - "ok": true - } - - Error model: - Throws on wrapper error. The orchestrator (`04-run-batch.ps1`) - catches and routes through its own exit-code mapping (0 ok / 10 - stuck / 20 error). Run standalone, an uncaught throw exits with - PowerShell's default code (1) plus a stack trace. `exit 20` is - intentionally NOT used here because this script is invoked via - `&` from the orchestrator — `exit` in that context would terminate - the orchestrator mid-pipeline. -#> -[CmdletBinding()] -param() - -. "$PSScriptRoot/Common.ps1" - -function Get-RequiredToolsets { - $root = Get-RepoRoot - # PlatformToolset can live at either the repo root (e.g. fork-local - # common.openconsole.props) or under src/ (the upstream cascadia - # layout). Probe both so a future relocation doesn't silently make - # the preflight read a stale file. Tests/older layouts may differ — - # missing files are skipped. - $candidates = @( - 'common.openconsole.props', - 'src/common.openconsole.props', - 'src/common.build.pre.props', - 'src/common.build.post.props', - 'src/common.build.tests.props', - 'src/wap-common.build.pre.props', - 'src/wap-common.build.post.props' - ) - $found = [System.Collections.Generic.HashSet[string]]::new() - $probed = [System.Collections.Generic.List[string]]::new() - foreach ($rel in $candidates) { - $p = Join-Path $root $rel - if (-not (Test-Path -LiteralPath $p)) { continue } - $probed.Add($rel) | Out-Null - $text = [System.IO.File]::ReadAllText($p) - foreach ($m in ([regex]']*>([^<]+)').Matches($text)) { - $val = $m.Groups[1].Value.Trim() - # Skip MSBuild property references like $(DefaultPlatformToolset) — those resolve at build time. - if ($val -and $val -notmatch '^\$\(') { [void]$found.Add($val) } - } - } - return [pscustomobject] @{ - Toolsets = @($found) - ProbedFiles = @($probed) - } -} - -function Get-VsInstalls { - # ${env:ProgramFiles(x86)} can be unset on nonstandard hosts (e.g. a - # 32-bit-only container, or a misconfigured runner). Treat the - # absence as "no VS detected" rather than throwing — the orchestrator - # treats VS-missing as a Tier-4 infra stuck the same way. - $pf86 = ${env:ProgramFiles(x86)} - if (-not $pf86) { return ,@() } - $vswhere = Join-Path $pf86 'Microsoft Visual Studio\Installer\vswhere.exe' - if (-not (Test-Path -LiteralPath $vswhere)) { return ,@() } - $out = & $vswhere -all -products * -property installationPath 2>$null - if ($LASTEXITCODE -ne 0) { return ,@() } - return ,@($out | Where-Object { $_ }) -} - -function Get-AvailableToolsets { - param([string[]] $VsInstalls) - $found = [System.Collections.Generic.HashSet[string]]::new() - foreach ($inst in $VsInstalls) { - # The authoritative location for installed platform toolsets is - # MSBuild\Microsoft\VC\\Platforms\\PlatformToolsets\. - # The bare `MSBuild\Microsoft\VC\v???` directories are MSBuild target - # versions, not toolsets — older versions of this script confused the two. - $vcRoot = Join-Path $inst 'MSBuild\Microsoft\VC' - if (-not (Test-Path -LiteralPath $vcRoot)) { continue } - Get-ChildItem -Directory -LiteralPath $vcRoot -ErrorAction SilentlyContinue | ForEach-Object { - $ptRoot = Join-Path $_.FullName 'Platforms\x64\PlatformToolsets' - if (Test-Path -LiteralPath $ptRoot) { - Get-ChildItem -Directory -LiteralPath $ptRoot -ErrorAction SilentlyContinue | ForEach-Object { - if ($_.Name -match '^v\d{3}$') { [void]$found.Add($_.Name) } - } - } - } - } - return ,@($found) -} - -try { - $req = Get-RequiredToolsets - $required = $req.Toolsets - $probed = $req.ProbedFiles - $vsInstalls = Get-VsInstalls - $available = Get-AvailableToolsets -VsInstalls $vsInstalls - $missing = @($required | Where-Object { $available -notcontains $_ }) - - # Stale-probe guard: if none of the candidate build files even - # existed, the probe list itself is wrong (paths moved). That's a - # silent gap — fail closed so the orchestrator surfaces it as a - # Tier-4 stuck instead of waving the run through on an empty - # required-toolsets list. - $probeStale = ($probed.Count -eq 0) - $ok = (-not $probeStale) -and ($missing.Count -eq 0) -and ($vsInstalls.Count -gt 0) - - $doc = [ordered] @{ - required_toolsets = @($required | Sort-Object) - available_toolsets = @($available | Sort-Object) - missing = @($missing | Sort-Object) - vs_installs = @($vsInstalls) - probed_files = @($probed | Sort-Object) - probe_stale = $probeStale - ok = $ok - } - $doc | ConvertTo-Json -Depth 4 -} -catch { - Write-Error $_.Exception.Message - Write-Error $_.ScriptStackTrace - throw -} diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 782c45d99..449b0dda3 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -1,254 +1,24 @@ # Common.ps1 — shared helpers for upstream-sync scripts. # Dot-source from each script: . "$PSScriptRoot/Common.ps1" # -# State model -# ----------- -# This skill does NOT keep a state.json file. Every persistent fact lives in -# the authoritative source that owns it: +# Only contains helpers used by 2+ scripts. Single-use helpers live inline +# in the script that uses them. # -# * "What's already been picked?" -> the `cherry picked from commit ` -# trailers we write on every `git cherry-pick -x` (parsed from origin/main). -# * "What's pending?" -> `git log --cherry-pick --right-only`. -# * "Is the scheduler locked?" -> any OPEN gh issue carrying the -# `upstream-sync-stuck` label. Lock metadata (kind, tier, stuck_on_sha, -# findings_hash) is encoded in a fenced ```yaml # wta-state ... ``` block -# in the issue body so re-runs can recognize the same failure. -# * "Transient artifacts" (build logs, generated reports) -> written under -# `Generated Files/upstream-sync//` which is gitignored at the -# repo root (`**/Generated Files/`). Never committed. +# The skill keeps no `state.json`. Watermark is the most recent +# `(cherry picked from commit )` trailer on origin/main (read by +# 02-compute-pending.ps1). Stuck-lock is any OPEN gh issue with the +# `upstream-sync-stuck` label (agent queries `gh issue list` directly). $ErrorActionPreference = 'Stop' Set-StrictMode -Version Latest -# --------------------------------------------------------------------------- -# Repo + path helpers -# --------------------------------------------------------------------------- - -function Get-RepoRoot { - $r = git rev-parse --show-toplevel 2>$null - if ($LASTEXITCODE -ne 0) { throw "Not inside a git repo." } - return $r.Trim() -} - -function ConvertTo-RepoRelativePath { - # Normalize a path to forward-slash, repo-relative form so callers can - # safely embed it in committed text without leaking machine-specific - # drive letters / user dirs. - param([Parameter(Mandatory)] [string] $Path) - $root = ((Get-RepoRoot) -replace '\\','/').TrimEnd('/') - $abs = $Path -replace '\\','/' - if ($abs.Equals($root, [System.StringComparison]::OrdinalIgnoreCase)) { - throw "ConvertTo-RepoRelativePath: refusing to return empty (path == repo root): $Path" - } - $prefix = "$root/" - if ($abs.StartsWith($prefix, [System.StringComparison]::OrdinalIgnoreCase)) { - return $abs.Substring($prefix.Length) - } - throw "ConvertTo-RepoRelativePath: '$Path' is not under repo root '$root'." -} - -function Get-GeneratedDir { - # Per-skill, per-day artifact directory under the repo's gitignored - # `Generated Files/` root (matches the workspace convention used by other - # skills; the repo's top-level .gitignore has `**/Generated Files/`). - # Optional -Sub appends a subdirectory (e.g. 'build-logs'). - param([string] $Sub) - $root = Get-RepoRoot - $date = (Get-Date).ToString('yyyy-MM-dd') - $path = Join-Path $root "Generated Files/upstream-sync/$date" - if ($Sub) { $path = Join-Path $path $Sub } - if (-not (Test-Path -LiteralPath $path)) { - New-Item -ItemType Directory -Path $path -Force | Out-Null - } - return $path -} - -# --------------------------------------------------------------------------- -# Git + remote setup -# --------------------------------------------------------------------------- - -function Ensure-UpstreamRemote { - param( - [string] $Name = 'upstream', - [string] $Url = 'https://github.com/microsoft/terminal.git' - ) - $existing = git remote get-url $Name 2>$null - if ($LASTEXITCODE -ne 0) { - git remote add $Name $Url | Out-Null - if ($LASTEXITCODE -ne 0) { throw "Failed to add remote $Name." } - } elseif ($existing.Trim() -ne $Url) { - throw "Remote '$Name' points at '$($existing.Trim())' (expected '$Url'). Fix the remote before running upstream-sync." - } -} - -function Assert-CleanWorktree { - $dirty = git status --porcelain - if ($LASTEXITCODE -ne 0) { throw "git status failed." } - if ($dirty) { - throw "Working tree is not clean:`n$dirty`nCommit or stash first." - } -} - -function Resolve-FullCommitSha { - param([Parameter(Mandatory)] [string] $Sha) - $full = (git rev-parse "$Sha^{commit}" 2>$null) - if ($LASTEXITCODE -ne 0 -or -not $full) { throw "Could not resolve commit SHA '$Sha'." } - return $full.Trim() -} - -# --------------------------------------------------------------------------- -# Derived state — replaces the old Read-State/Write-State on state.json -# --------------------------------------------------------------------------- - -function Get-LastSyncedUpstreamSha { - # "How far have we already synced from upstream/main?" Derived from the - # `(cherry picked from commit )` trailers that `git cherry-pick -x` - # writes on every pick. We walk origin/main newest-first and return the - # FIRST trailer that points at a commit reachable from upstream/main. - # - # Why newest-first instead of "highest topological position": cherry-picks - # land in chronological order on origin/main, so the most recent trailer - # is the watermark. A picked-then-reverted upstream commit will appear in - # an OLDER trailer (with a corresponding `Revert "..."` commit later) - - # `git cherry` (used by Get-PendingUpstreamShas) will see the revert and - # correctly re-list it as pending if needed, so this watermark only needs - # to be the high-water-mark of progress, not the strict frontier. - # - # Known limitation: a HUMAN who manually cherry-picks an upstream hotfix - # onto origin/main jumps the watermark forward even though earlier - # upstream commits remain unsynced. Mitigation: Get-PendingUpstreamShas - # uses patch-id comparison against ALL of origin/main (not just the - # commits after the watermark), so the unsynced earlier commits will - # still be picked up on the next scheduler run. The watermark only - # narrows the `git log` walk for speed - it isn't load-bearing for - # correctness of the pending list. - # - # Performance: capped at the most recent 5000 origin/main commits via - # --max-count. Even years of fork history fits well under that cap; if - # a real deployment ever exceeds it we re-throw with the standard - # not-found message so the operator can re-seed with the fast path. - # Requires `upstream` remote fetched: caller must Ensure-UpstreamRemote + - # `git fetch upstream main --no-tags` first. - $commits = @(git log origin/main --max-count=5000 --grep='cherry picked from commit' --format='%H' 2>$null) - if ($LASTEXITCODE -ne 0) { throw "git log on origin/main failed while deriving last-synced SHA." } - foreach ($c in $commits) { - $body = git log -1 --format='%B' $c 2>$null - if ($body -match '\(cherry picked from commit ([0-9a-f]{7,40})\)') { - $rawSha = $matches[1] - # Resolve to full 40-char SHA first so the ancestry check below - # works against a canonical object name (an abbreviated or ambiguous - # prefix could cause `git merge-base --is-ancestor` to fail and - # silently skip an otherwise-valid watermark candidate). - $fullSha = $null - try { $fullSha = Resolve-FullCommitSha $rawSha } catch { continue } - $null = git merge-base --is-ancestor $fullSha upstream/main 2>$null - if ($LASTEXITCODE -eq 0) { - return $fullSha - } - } - } - throw "No 'cherry picked from commit' trailer pointing at upstream/main was found on origin/main (scanned the most recent 5000 commits). Either the fork hasn't synced via this skill before, or the trailer convention drifted. The very first sync needs an operator to seed the watermark commit (see SKILL.md - 'First-time sync')." -} - -function Get-PendingUpstreamShas { - # Returns upstream/main commits that don't have an equivalent on - # origin/main, in chronological (oldest-first) order. Uses - # `git log --cherry-pick --right-only` which compares patch IDs (not just - # trailers), so picked-and-reverted commits correctly re-appear. - # - # The range we walk is ALWAYS `origin/main...upstream/main` regardless of - # -Since. -Since (the watermark) is treated as a known floor: we drop any - # commit that is an ancestor of -Since. This way: - # * The patch-id filter still considers ALL of origin/main (so commits - # that landed on origin/main outside the scheduler's trailer trail - - # e.g. a manual cherry-pick - still get filtered out). - # * The watermark only trims the obviously-old tail at the bottom of - # the resulting list, which is what makes the walk fast. - param( - [string] $Since, - [int] $Limit = 0 - ) - $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) - if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } - $shas = @($out | Where-Object { $_ -match '^[0-9a-f]{40}$' }) - if ($Since) { - # Skip commits that are ancestors of the watermark. We can't use - # `..upstream/main` for the range because that would re-include - # commits filtered by --cherry-pick; explicit per-sha ancestry check - # is O(n) git calls but n is small (only the suffix matters). - $filtered = New-Object 'System.Collections.Generic.List[string]' - foreach ($sha in $shas) { - $null = git merge-base --is-ancestor $sha $Since 2>$null - if ($LASTEXITCODE -ne 0) { [void] $filtered.Add($sha) } - } - $shas = @($filtered) - } - if ($Limit -gt 0) { $shas = @($shas | Select-Object -First $Limit) } - return ,$shas -} - -# --------------------------------------------------------------------------- -# Stuck-lock — derived from open `upstream-sync-stuck` labeled issues -# --------------------------------------------------------------------------- - -$script:StuckLabel = 'upstream-sync-stuck' -$script:WtaStateFence = '# wta-state' # marker inside ```yaml ... ``` blocks - -function Get-StuckIssues { - # Returns all OPEN issues carrying the upstream-sync-stuck label. -R is - # pinned because an `upstream` remote can trick gh into defaulting to - # microsoft/terminal, where this account has no permission. Stderr goes - # to a temp file so a gh deprecation/version notice can't break the JSON. - $errFile = [System.IO.Path]::GetTempFileName() - $errText = '' - $ghExit = 0 - try { - $json = gh issue list --repo microsoft/intelligent-terminal --label $script:StuckLabel --state open --json number,title,body,url,labels,createdAt 2>$errFile - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } - } - finally { - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue - } - if ($ghExit -ne 0) { - throw "gh issue list failed (exit $ghExit): $errText" - } - if (-not $json) { return @() } - return @($json | ConvertFrom-Json) -} - -function Get-StuckMetaFromIssue { - # Parse a fenced ```yaml ... # wta-state ... ``` block out of an issue body. - # Accepts the single-quoted form Format-StuckYamlBlock emits; values are - # un-escaped (`''` -> `'`). Returns $null if no block is found (degraded - # but safe: callers should still treat the open issue as a lock - the - # metadata is for findings_hash compare and resume hints, not for the - # lock decision itself). - param([Parameter(Mandatory)] $Issue) - if (-not $Issue.body) { return $null } - $pattern = '(?ms)```yaml\s*\r?\n#\s*wta-state\s*\r?\n(.+?)\r?\n```' - if ($Issue.body -notmatch $pattern) { return $null } - $yaml = $matches[1] - $h = [ordered] @{} - foreach ($l in $yaml -split '\r?\n') { - if ($l -match "^\s*([a-z_][a-z0-9_]*)\s*:\s*'((?:[^']|'')*)'\s*$") { - $h[$matches[1]] = $matches[2] -replace "''", "'" - } elseif ($l -match '^\s*([a-z_][a-z0-9_]*)\s*:\s*(.+?)\s*$') { - # Tolerate bare scalars for backward compatibility with hand-edited - # issues; the writer always quotes, but a human edit might not. - $h[$matches[1]] = $matches[2] - } - } - return [pscustomobject] $h -} +$script:WtaStateFence = '# wta-state' function Format-StuckYamlBlock { - # Build the fenced YAML block that 07/07b embed in stuck-issue bodies. - # Values are always single-quoted with `'` -> `''` escaping so embedded - # colons, newlines, leading dashes, etc. round-trip without breaking the - # parser in Get-StuckMetaFromIssue or any other YAML reader. Multiline - # values are folded to spaces (we don't need full block-scalar support; - # the lock decision is just "is the list non-empty"). + # Fenced ```yaml ... # wta-state ... ``` block embedded in stuck-issue + # bodies. Values are single-quoted with `'` -> `''` escaping so colons, + # newlines, leading dashes round-trip through the issue body. Used by + # 06-open-stuck-issue.ps1 and 06b-open-build-stuck-issue.ps1. param([Parameter(Mandatory)] [hashtable] $Fields) $lines = @('```yaml', $script:WtaStateFence) foreach ($k in $Fields.Keys) { @@ -261,62 +31,8 @@ function Format-StuckYamlBlock { return ($lines -join "`n") } -# --------------------------------------------------------------------------- -# Misc helpers -# --------------------------------------------------------------------------- - -function Get-GhUserLogin { - $login = gh api user --jq '.login' 2>$null - if ($LASTEXITCODE -ne 0 -or -not $login) { throw "gh CLI is not authenticated. Run 'gh auth login'." } - return $login.Trim() -} - function Format-Iso8601 { + # Used by 06 + 06b for the `at` field of the stuck-issue YAML block. param([DateTime] $When = (Get-Date)) return $When.ToString('yyyy-MM-ddTHH:mm:sszzz') } - -function New-RunContext { - [pscustomobject] @{ - StartedAt = Get-Date - Host = $env:COMPUTERNAME - # Branch name carries date + UTC timestamp + 4 random hex chars so - # repeated runs on the same day - or two consecutive runs after a - # rebase-merge that didn't auto-delete the previous branch - never - # check out a stale branch and replay already-merged commits. - Branch = "upstream-sync/$((Get-Date).ToString('yyyy-MM-dd'))-$((Get-Date).ToUniversalTime().ToString('HHmmss'))-$(([guid]::NewGuid().ToString('N').Substring(0,4)))" - Picked = @() - Pending = @() - DroppedPairs = @() - SkippedEmpty = @() - Tier0 = @() - Tier2 = @() - StuckSha = $null - StuckPaths = @() - StuckError = $null - StuckValidation = $null - Preflight = $null - Scan = $null - Build = $null - Status = 'unknown' - ReportPath = $null - PrUrl = $null - IssueUrl = $null - } -} - -function Get-FindingsHash { - param([Parameter(Mandatory)] $Findings) - # Stable hash of a findings list — used as a stuck-issue findings_hash - # so repeat-runs of the same broken batch can detect "same failure as - # last time" and avoid re-opening duplicate issues. - $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) - $sha = [System.Security.Cryptography.SHA256]::Create() - try { - $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) - return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) - } - finally { - $sha.Dispose() - } -} From c3cce2b908ae4e49f861edc7befd923669dced0a Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 16:13:24 +0800 Subject: [PATCH 54/82] fix: address Copilot review on 38b831768 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight findings from the Copilot review after the orchestration-to-SKILL restructure — all docs-vs-code drift or stale links from the rename: * SKILL.md step 5: cherry-pick status names were 'applied'/'auto-resolved'/'empty' but 03-cherry-pick-one.ps1 actually emits 'picked'/'skipped-empty'/'stuck'. * SKILL.md State Model: 'pending' command needs origin/main in the rev range to match 02-compute-pending.ps1. * SKILL.md prereqs: explicitly require 'origin' to point at microsoft/intelligent-terminal (the scripts pin -R explicitly but the push still relies on origin pointing at the target repo). * 03-conflict-triage.md + 03-cherry-pick-one.ps1 doc comments still referenced 'known-conflicts.md' instead of '03-known-conflicts.md'. * follow-up-pr.md had two broken links to './conflict-triage.md' (now '03-conflict-triage.md') plus an anchor that never existed ('primary-worktree-clean-main convention'). Rewrote both to be self-contained. * SKILL.md troubleshooting row: 'returns \"empty\"' -> 'returns \"skipped-empty\"'. * 06-open-stuck-issue.ps1: Tier-3 issue body now includes the upstream commit URL and author per the contract in 03-conflict-triage.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 9 +++++---- .../upstream-sync/references/03-conflict-triage.md | 4 ++-- .../skills/upstream-sync/references/follow-up-pr.md | 8 ++++---- .../upstream-sync/scripts/03-cherry-pick-one.ps1 | 4 ++-- .../upstream-sync/scripts/06-open-stuck-issue.ps1 | 10 +++++++--- 5 files changed, 20 insertions(+), 15 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index a84b23d5b..4ea7bdae5 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -35,6 +35,7 @@ the operator can audit in your transcript. ## Prerequisites - `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches matching `upstream-sync/*`** and **issue + label create** on the same repo. +- **`origin` MUST point at `microsoft/intelligent-terminal`.** All scripts push to `origin` and the PR / stuck-issue creation passes `-R microsoft/intelligent-terminal` explicitly. If `origin` is a personal fork, the push will land on the fork and the PR will fail to find its head. Use `git remote -v` to verify before running. - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment (build is a hard gate before finalize — see [step 7](#7-build)). - Remote named `upstream` — the scripts create it if missing. @@ -50,7 +51,7 @@ Every persistent fact lives in the source that owns it: | Question | Source of truth | |---|---| | What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived inline by [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | -| What's pending? | `git log --cherry-pick --right-only --no-merges ...upstream/main`. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | +| What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, then drop SHAs older than (or equal to) the watermark above. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | | Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | | What does the lock mean? | A fenced ```yaml # wta-state``` block in the issue body carries `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. | | Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | @@ -132,8 +133,8 @@ $pick = $pickJson | ConvertFrom-Json Branch on `$pick.status`: -- `"applied"` or `"auto-resolved"` — record `$sha` in `$picked`, continue. -- `"empty"` — record `$sha` in `$skippedEmpty`, continue. +- `"picked"` — record `$sha` in `$picked`, continue. +- `"skipped-empty"` — record `$sha` in `$skippedEmpty`, continue. - `"stuck"` — **stop the loop** and go to step 5a. #### 5a. On stuck — open the Tier-3 issue and exit @@ -389,7 +390,7 @@ deferred fixes. |---|---| | `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [First-time sync](#first-time-sync). | | Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | -| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"empty"` and the agent's loop skips it. No action needed. | +| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"skipped-empty"` and the agent's loop skips it. No action needed. | | Same file conflicts every run | Add it to the Tier-0 list in [references/03-known-conflicts.md](./references/03-known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | | `gh pr create` returns "Head sha can't be blank" | `05-finalize-pr.ps1` retries 3× automatically. On slow networks may need a manual second run. | diff --git a/.github/skills/upstream-sync/references/03-conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md index 1e9ba00d9..01a4043f6 100644 --- a/.github/skills/upstream-sync/references/03-conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -9,13 +9,13 @@ Some files have a fixed correct resolution that never changes. Examples: - `.github/workflows/spelling2.yml` — always take upstream (verified on sister repo `agentic-terminal`). -The list of these paths lives in [`known-conflicts.md`](./known-conflicts.md). +The list of these paths lives in [`03-known-conflicts.md`](./03-known-conflicts.md). **Algorithm:** ```pwsh $conflictingPaths = git diff --name-only --diff-filter=U -$tier0List = Get-KnownConflicts # parses known-conflicts.md +$tier0List = Get-KnownConflicts # parses 03-known-conflicts.md foreach ($p in $conflictingPaths) { $entry = $tier0List | Where-Object { $_.Path -eq $p } if (-not $entry) { return $false } # Tier 0 doesn't cover this commit diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index 7486d269c..09d7e7185 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -42,9 +42,9 @@ follow-up PR. ### Worktree setup -Stay on the [primary-worktree-clean-main -convention](./conflict-triage.md). Open a sibling worktree for the -follow-up so the main worktree stays clean: +Keep the primary worktree at `Q:\official\intelligent-terminal` on `main` +and open a sibling worktree for the follow-up so the main worktree stays +clean: ```pwsh $syncPr = 220 @@ -148,4 +148,4 @@ gh pr edit --repo microsoft/intelligent-terminal --base main - [SKILL.md § After-PR review handling](../SKILL.md#after-pr-review-handling--fix-in-pr-vs-follow-up-pr) — short version of this doc. - [`copilot-pr-review-loop`](../../copilot-pr-review-loop/SKILL.md) — the skill that drives the follow-up PR through its own review rounds. -- [Branch + worktree convention](./conflict-triage.md) — keep `Q:\official\intelligent-terminal` on `main`; use sibling worktrees for PR work. +- [`03-conflict-triage.md`](./03-conflict-triage.md) — Tier 0/1/2/3/4 conflict resolution rubric (referenced when deciding whether a build failure belongs on the sync PR or a follow-up). diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 7e7dcfc63..e750d7581 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -4,7 +4,7 @@ .DESCRIPTION Runs `git cherry-pick -x `. On conflict, attempts Tier-0 (known - take-{upstream,ours} files from known-conflicts.md), then Tier-1 (empty + take-{upstream,ours} files from 03-known-conflicts.md), then Tier-1 (empty after staging → skip). Anything else returns 'stuck' and leaves the cherry-pick aborted. @@ -43,7 +43,7 @@ function Get-KnownConflicts { function Get-ConflictPaths { # core.quotepath=off keeps non-ASCII paths in raw UTF-8 so Tier-0 - # path matching against known-conflicts.md works without C-quoting. + # path matching against 03-known-conflicts.md works without C-quoting. $u = git -c core.quotepath=off diff --name-only --diff-filter=U if (-not $u) { return @() } return @($u -split "`n" | Where-Object { $_ }) diff --git a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 index f733c5660..ed4674910 100644 --- a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 @@ -45,9 +45,11 @@ param( git push -u origin $Branch 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } -$shortSha = $StuckSha.Substring(0,9) -$subj = (git log -1 --format='%s' $StuckSha).Trim() -$title = "Upstream sync stuck at ${shortSha}: $subj" +$shortSha = $StuckSha.Substring(0,9) +$subj = (git log -1 --format='%s' $StuckSha).Trim() +$author = (git log -1 --format='%an <%ae>' $StuckSha).Trim() +$upstreamUrl = "https://github.com/microsoft/terminal/commit/$StuckSha" +$title = "Upstream sync stuck at ${shortSha}: $subj" $yamlBlock = Format-StuckYamlBlock @{ tier = '3' @@ -74,6 +76,8 @@ $body = @" > Closing the issue IS the lock-clear signal — no separate script needed. **Stuck on:** ``$StuckSha`` — $subj +**Upstream commit:** [$shortSha]($upstreamUrl) +**Author:** $author **Sync branch:** ``$Branch`` (push attempted — run ``git ls-remote --heads origin $Branch`` to verify it landed) **Conflicting paths:** From efdb6b6d906b5cf73c37b3fa778c59618e8eefd4 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 16:24:58 +0800 Subject: [PATCH 55/82] fix: address 6 more Copilot findings (round 5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Common.ps1: docstring now correctly says newlines fold to a single space (not preserved). Implementation was already correct; comment was wrong. * 06-open-stuck-issue.ps1: capture and validate \0 after each git log call so a bogus \ fails fast with a clear error instead of filing an issue with empty subject/author. * 01-fetch-upstream.ps1: normalize remote URLs before comparing so the same logical repo configured under common variants (https/ssh, with/without trailing .git or /) is accepted as a match. * allow.txt: drop orwardable, hashtables, oolsets — confirmed unused via repo-wide grep (only matches were in allow.txt itself). Keeps the allowlist minimal so future misspellings don't get masked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/actions/spelling/allow/allow.txt | 3 --- .../scripts/01-fetch-upstream.ps1 | 21 +++++++++++++++++-- .../scripts/06-open-stuck-issue.ps1 | 15 +++++++++++-- .../skills/upstream-sync/scripts/Common.ps1 | 8 ++++--- 4 files changed, 37 insertions(+), 10 deletions(-) diff --git a/.github/actions/spelling/allow/allow.txt b/.github/actions/spelling/allow/allow.txt index 5c01c78fd..99c6b8e94 100644 --- a/.github/actions/spelling/allow/allow.txt +++ b/.github/actions/spelling/allow/allow.txt @@ -69,7 +69,6 @@ foob fooba footgun formedness -forwardable FTCS gantt gfm @@ -86,7 +85,6 @@ greppable haikus handover hashtable -hashtables historicals hstrings https @@ -236,7 +234,6 @@ tokio tombstoned toolpath toolset -toolsets trn TUs txs diff --git a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 index 4bde1869d..a78a681f3 100644 --- a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +++ b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 @@ -12,12 +12,29 @@ param( . "$PSScriptRoot/Common.ps1" # Ensure-UpstreamRemote (inlined — single-use). Adds the `upstream` remote -# if missing; bails if it points somewhere unexpected. +# if missing; bails if it points somewhere unexpected. Normalizes common URL +# variants (https vs ssh, with/without trailing `.git`, with/without trailing +# slash) so the same logical repo configured under any of those forms is +# accepted. +function _NormalizeRemoteUrl { + param([string] $Url) + $u = $Url.Trim().TrimEnd('/').ToLowerInvariant() + # git@github.com:owner/repo(.git)? -> github.com/owner/repo + if ($u -match '^git@([^:]+):(.+?)(\.git)?$') { + $u = "$($Matches[1])/$($Matches[2])" + } + # https://host/owner/repo(.git)? -> host/owner/repo + elseif ($u -match '^https?://([^/]+)/(.+?)(\.git)?$') { + $u = "$($Matches[1])/$($Matches[2])" + } + return $u +} + $existing = git remote get-url upstream 2>$null if ($LASTEXITCODE -ne 0) { git remote add upstream $UpstreamUrl | Out-Null if ($LASTEXITCODE -ne 0) { throw "Failed to add 'upstream' remote." } -} elseif ($existing.Trim() -ne $UpstreamUrl) { +} elseif ((_NormalizeRemoteUrl $existing) -ne (_NormalizeRemoteUrl $UpstreamUrl)) { throw "Remote 'upstream' points at '$($existing.Trim())' (expected '$UpstreamUrl'). Fix the remote before running upstream-sync." } diff --git a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 index ed4674910..ef1657503 100644 --- a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 @@ -46,8 +46,19 @@ git push -u origin $Branch 2>&1 | Out-Host if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } $shortSha = $StuckSha.Substring(0,9) -$subj = (git log -1 --format='%s' $StuckSha).Trim() -$author = (git log -1 --format='%an <%ae>' $StuckSha).Trim() + +$subj = (git log -1 --format='%s' $StuckSha 2>$null) +if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($subj)) { + throw "git log failed for stuck SHA '$StuckSha' (exit $LASTEXITCODE). Refusing to file an issue with a missing subject." +} +$subj = $subj.Trim() + +$author = (git log -1 --format='%an <%ae>' $StuckSha 2>$null) +if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($author)) { + throw "git log failed to resolve author for stuck SHA '$StuckSha' (exit $LASTEXITCODE)." +} +$author = $author.Trim() + $upstreamUrl = "https://github.com/microsoft/terminal/commit/$StuckSha" $title = "Upstream sync stuck at ${shortSha}: $subj" diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 index 449b0dda3..8eb99a17f 100644 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ b/.github/skills/upstream-sync/scripts/Common.ps1 @@ -16,9 +16,11 @@ $script:WtaStateFence = '# wta-state' function Format-StuckYamlBlock { # Fenced ```yaml ... # wta-state ... ``` block embedded in stuck-issue - # bodies. Values are single-quoted with `'` -> `''` escaping so colons, - # newlines, leading dashes round-trip through the issue body. Used by - # 06-open-stuck-issue.ps1 and 06b-open-build-stuck-issue.ps1. + # bodies. Values are single-quoted with `'` -> `''` escaping so colons + # and leading dashes round-trip through the issue body. Newlines in + # values are folded to a single space (multi-line values are not + # preserved). Used by 06-open-stuck-issue.ps1 and + # 06b-open-build-stuck-issue.ps1. param([Parameter(Mandatory)] [hashtable] $Fields) $lines = @('```yaml', $script:WtaStateFence) foreach ($k in $Fields.Keys) { From fe2279596c3b8cf789899939bd8ee6456e4e4eb7 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 16:36:06 +0800 Subject: [PATCH 56/82] fix: pipe diagnostics to stderr so scripts honor stdout contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot round 6 flagged that 'git fetch/push/cherry-pick | Out-Host' contaminates stdout for the JSON / single-URL contracts every script documents. PowerShell's Out-Host writes through the host UI which IS stdout for the pwsh process, so '\ = pwsh -File ...' capture picks up that text and parsing breaks. Replace each ' 2>&1 | Out-Host' with ' 2>&1 | ForEach-Object { [Console]::Error.WriteLine(\) }' so combined stderr+stdout from git/gh lands on the parent process's stderr fd while the script's success stream stays clean (JSON for 03/04, PR URL for 05, issue URL for 06/06b, SHA for 01). Also moves the 'Auto-merge armed' Write-Host in 05 to Console.Error for the same reason. 7 sites fixed (01, 03x2, 05x2, 06, 06b) plus the Write-Host in 05. Write-Warning calls are kept as-is — they emit to the warning stream prefixed with 'WARNING:' and aren't captured as success output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 | 2 +- .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 | 4 ++-- .github/skills/upstream-sync/scripts/05-finalize-pr.ps1 | 6 +++--- .../skills/upstream-sync/scripts/06-open-stuck-issue.ps1 | 2 +- .../upstream-sync/scripts/06b-open-build-stuck-issue.ps1 | 2 +- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 index a78a681f3..8e7b4583b 100644 --- a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +++ b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 @@ -38,7 +38,7 @@ if ($LASTEXITCODE -ne 0) { throw "Remote 'upstream' points at '$($existing.Trim())' (expected '$UpstreamUrl'). Fix the remote before running upstream-sync." } -git fetch upstream main --no-tags 2>&1 | Out-Host +git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } $sha = git rev-parse upstream/main diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index e750d7581..ce550f57e 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -96,7 +96,7 @@ $env:GIT_COMMITTER_DATE = $info[5] try { # Attempt the pick. -git cherry-pick --keep-redundant-commits -x $fullSha 2>&1 | Out-Host +git cherry-pick --keep-redundant-commits -x $fullSha 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } $pickCode = $LASTEXITCODE if ($pickCode -eq 0) { @@ -181,7 +181,7 @@ if ($unhandled.Count -gt 0) { } # All conflicts handled by Tier-0; continue the pick (preserve original message). -git cherry-pick --continue --no-edit 2>&1 | Out-Host +git cherry-pick --continue --no-edit 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { # Could still be empty after Tier-0. $staged = git diff --cached --name-only diff --git a/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 index 7d2fa2538..f602f7871 100644 --- a/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 +++ b/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 @@ -65,7 +65,7 @@ $bodyContent = $banner + $PrBody $shortTo = $UpstreamHeadSha.Substring(0,9) $title = "chore(upstream): sync microsoft/terminal up to $shortTo" -git push -u origin $Branch | Out-Host +git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue throw "git push failed for $Branch." @@ -100,11 +100,11 @@ $prUrl = $prUrl.Trim() if ($AutoMergeStrategy -ne 'none') { $strategyFlag = "--$AutoMergeStrategy" - gh pr merge -R microsoft/intelligent-terminal $prUrl $strategyFlag --auto --delete-branch | Out-Host + gh pr merge -R microsoft/intelligent-terminal $prUrl $strategyFlag --auto --delete-branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { Write-Warning "gh pr merge --auto failed. PR is open at $prUrl; merge manually with '$AutoMergeStrategy' strategy (NOT squash)." } else { - Write-Host "Auto-merge armed with strategy: $AutoMergeStrategy" -ForegroundColor Green + [Console]::Error.WriteLine("Auto-merge armed with strategy: $AutoMergeStrategy") } } diff --git a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 index ef1657503..06f1170d6 100644 --- a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 @@ -42,7 +42,7 @@ param( . "$PSScriptRoot/Common.ps1" # Push the stuck branch so the human can resume on it. -git push -u origin $Branch 2>&1 | Out-Host +git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } $shortSha = $StuckSha.Substring(0,9) diff --git a/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 index 444f472ac..67522c3bb 100644 --- a/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 +++ b/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 @@ -64,7 +64,7 @@ $findingsForHash = if ($Kind -eq 'build-failed') { $findingsHash = Get-FindingsHash $findingsForHash # Push the sync branch so the human can resume on it. -git push -u origin $Branch 2>&1 | Out-Host +git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push sync branch — issue still being filed for visibility." } $titleKindLabel = if ($Kind -eq 'build-failed') { 'build failure' } else { 'build inconclusive (timeout)' } From bc151b816311367e4d60fdb06db798496663d45e Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 16:46:52 +0800 Subject: [PATCH 57/82] remove 05-finalize-pr.ps1; agent calls gh pr create directly (round 7) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit User feedback on the PR: 'Do you see any reason we need this ps1? I just for create PR, and I never see copilot has any problem to create a PR. Even you want to ensure the PR created is using rebase one, you can put the instruction in md.' Agreed. Deleted scripts/05-finalize-pr.ps1 and inlined the recipe (banner composition + push + gh pr create + 3x retry + optional gh pr merge --auto --rebase) into SKILL.md step 8. The agent now composes the banner and body, retries gh pr create itself, and decides per-run whether to arm auto-merge — same behaviour, no wrapper. Three other Copilot findings in the same round: * SKILL.md:56 + 03-conflict-triage.md:96 — triple-backtick fences embedded inside table cells / inside another triple-backtick fence break Markdown rendering. Reworded the table cell to plain text and switched the outer pwsh fence to tildes (~~~pwsh ... ~~~) so the inner literal backticks are no longer a fence. * 03-cherry-pick-one.ps1:75 — author/committer parsing split on tab, but tabs are legal in git ident strings. Switched the delimiter to ASCII unit separator (\x1F) which is forbidden in idents, and added a defensive field-count check (must be exactly 6) before binding to GIT_AUTHOR_* / GIT_COMMITTER_* env vars. Updated SKILL.md gotchas + troubleshooting + References list to drop all mentions of 05-finalize-pr.ps1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 90 ++++++++++---- .../references/03-conflict-triage.md | 6 +- .../scripts/03-cherry-pick-one.ps1 | 9 +- .../upstream-sync/scripts/05-finalize-pr.ps1 | 111 ------------------ 4 files changed, 78 insertions(+), 138 deletions(-) delete mode 100644 .github/skills/upstream-sync/scripts/05-finalize-pr.ps1 diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 4ea7bdae5..dd5c41557 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -53,7 +53,7 @@ Every persistent fact lives in the source that owns it: | What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived inline by [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | | What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, then drop SHAs older than (or equal to) the watermark above. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | | Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | -| What does the lock mean? | A fenced ```yaml # wta-state``` block in the issue body carries `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. | +| What does the lock mean? | A fenced YAML block carrying `# wta-state` in the issue body holds `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. | | Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | ### Why cherry-pick (and not rebase or merge) @@ -207,11 +207,34 @@ and EXIT. ### 8. Finalize the PR -Build a concise PR body (just YOU, the agent, composing markdown — there -is no report file to inline): +The agent (you) push the branch and open the PR directly with `git` and +`gh`. There is no wrapper script — there's nothing here that needs more +than `gh pr create` plus a body string you compose from the data you +already have in `$picked` / `$build` / `$pending`. + +Compose the body: ```pwsh -$body = @" +$banner = @" +> [!WARNING] +> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked +> upstream commit into one, destroying per-commit attribution, original +> author dates, the ``(cherry picked from commit )`` trailers that the +> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. +> Merge with **"Rebase and merge"** (preferred — flat history, all +> $($picked.Count) commit(s) land individually) or **"Create a merge commit"**. + +> [!NOTE] +> **Review-fix policy.** Only build-blocking fixes belong on this branch +> as **one** focused extra commit. All other Copilot / human review +> feedback goes into a **follow-up PR** based on this PR's head. See +> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). + +--- + +"@ + +$summary = @" Syncs **$($picked.Count)** commit(s) from microsoft/terminal up to ``$($upstreamSha.Substring(0,9))``. ## Picked @@ -231,17 +254,38 @@ $( ($skippedEmpty | ForEach-Object { "- ``$($_.Substring(0,9))``" }) -join "`n" ``04-try-build.ps1`` → ``$($build.kind)`` (exit $($build.exit_code), $($build.duration_ms) ms). Log: ``$($build.log_path)`` (gitignored). "@ -$prUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/05-finalize-pr.ps1 ` - -Branch $branch ` - -UpstreamHeadSha $upstreamSha ` - -PickedCount $picked.Count ` - -PrBody $body ` - -AutoMergeStrategy 'none' # or 'rebase' / 'merge' for hands-off; NEVER squash +$bodyFile = New-TemporaryFile +[System.IO.File]::WriteAllText($bodyFile, ($banner + $summary), (New-Object System.Text.UTF8Encoding($false))) +``` + +Push and create the PR. `gh pr create` on Windows can occasionally fail +with "Head sha can't be blank" right after a push — retry up to 3× with +a short delay: + +```pwsh +git push -u origin $branch + +$short = $upstreamSha.Substring(0,9) +$title = "chore(upstream): sync microsoft/terminal up to $short" + +$prUrl = $null +for ($attempt = 1; $attempt -le 3; $attempt++) { + $prUrl = gh pr create -R microsoft/intelligent-terminal ` + --base main --head $branch --title $title --body-file $bodyFile 2>&1 | + Select-Object -Last 1 + if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } + Write-Warning "gh pr create attempt $attempt failed: $prUrl" + Start-Sleep -Seconds 5 +} +Remove-Item -LiteralPath $bodyFile -Force +if ($prUrl -notmatch '^https://github.com/') { throw "gh pr create did not return a URL after 3 attempts." } + +# Optional: arm GitHub auto-merge with rebase (NEVER squash). +# Skip this if the operator wants to merge by hand. +gh pr merge -R microsoft/intelligent-terminal $prUrl --rebase --auto --delete-branch ``` -Pass `-AutoMergeStrategy rebase` if the operator wants GitHub to merge the -PR automatically once CI + approvals pass. **Never** pass `'squash'` — -the script doesn't accept it and the PR body's banner shouts about it. +Surface `$prUrl` to the operator. Done. Surface `$prUrl` to the operator. Done. @@ -335,16 +379,17 @@ attribution the cherry-pick approach was chosen to preserve. - If the sync PR merges first, rebase the follow-up onto `main` before it merges. -The orchestrator's PR banner ([scripts/05-finalize-pr.ps1](./scripts/05-finalize-pr.ps1)) -spells this policy out to the first reviewer so they don't push back on -deferred fixes. +The PR body's banner (see [step 8](#8-finalize-the-pr)) spells this +policy out to the first reviewer so they don't push back on deferred +fixes. ## Gotchas - **Never squash-merge the sync PR.** Use **"Rebase and merge"** (preferred) or **"Create a merge commit"**. The PR body opens with a - banner reminding the reviewer; `-AutoMergeStrategy rebase` arms GitHub - auto-merge with the right strategy so a tired reviewer can't get it wrong. + banner reminding the reviewer; the step-8 recipe shows the + `gh pr merge --rebase --auto` invocation so a tired reviewer can't get + it wrong. - **Don't amend substantive review fixes into the sync PR.** Only build-blocking fixes get **one** extra commit on the sync branch. See [references/follow-up-pr.md](./references/follow-up-pr.md). @@ -356,9 +401,9 @@ deferred fixes. handles this automatically — extend the list when you discover the next file with the same pattern. - **`gh pr create` on Windows can fail with "Head sha can't be blank"** if the - branch is freshly pushed and not yet visible. `05-finalize-pr.ps1` - retries 3× — do not "fix" the script to use `--head :` - (which would point `gh` at a fork). + branch is freshly pushed and not yet visible. The step-8 recipe wraps + the call in a 3× retry loop — do not "fix" the recipe to use + `--head :` (which would point `gh` at a fork). - **Do not run the orchestration twice while a stuck issue is open.** The step-1 preflight catches it, but a human bypassing that gate manually would overwrite the stuck branch and lose their in-progress resolution. @@ -392,7 +437,7 @@ deferred fixes. | Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | | Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"skipped-empty"` and the agent's loop skips it. No action needed. | | Same file conflicts every run | Add it to the Tier-0 list in [references/03-known-conflicts.md](./references/03-known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | -| `gh pr create` returns "Head sha can't be blank" | `05-finalize-pr.ps1` retries 3× automatically. On slow networks may need a manual second run. | +| `gh pr create` returns "Head sha can't be blank" | The step-8 recipe retries 3× automatically. On slow networks, the operator may need a manual second run of the whole sync. | ## References @@ -404,7 +449,6 @@ deferred fixes. - [scripts/02-compute-pending.ps1](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). - [scripts/03-cherry-pick-one.ps1](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. - [scripts/04-try-build.ps1](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. -- [scripts/05-finalize-pr.ps1](./scripts/05-finalize-pr.ps1) — push branch + open PR with squash-warning banner. - [scripts/06-open-stuck-issue.ps1](./scripts/06-open-stuck-issue.ps1) — Tier-3 stuck issue (mid-pick conflict). - [scripts/06b-open-build-stuck-issue.ps1](./scripts/06b-open-build-stuck-issue.ps1) — Tier-4 stuck issue (build failed after clean batch). - [scripts/Common.ps1](./scripts/Common.ps1) — the only two helpers shared by 2+ scripts. diff --git a/.github/skills/upstream-sync/references/03-conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md index 01a4043f6..4a6d68b3e 100644 --- a/.github/skills/upstream-sync/references/03-conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -89,12 +89,12 @@ Stage only if both agents agree `high`/`OK`. Otherwise, route to Tier 3. Anything not resolved by Tier 0–2: -```pwsh +~~~pwsh git cherry-pick --abort # Open the labeled stuck issue (06-open-stuck-issue.ps1) — issue body -# carries the ```yaml # wta-state``` block with stuck_on_sha + branch. +# carries the fenced YAML "# wta-state" block with stuck_on_sha + branch. # Surface the issue URL + branch to the operator and exit. -``` +~~~ The issue body (built by [`scripts/06-open-stuck-issue.ps1`](../scripts/06-open-stuck-issue.ps1)) **must** include: diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index ce550f57e..67cb50dbb 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -72,7 +72,14 @@ if ($LASTEXITCODE -ne 0) { throw "Could not resolve upstream commit $Sha." } $prePickHead = (git rev-parse HEAD).Trim() if ($LASTEXITCODE -ne 0) { throw "Could not record pre-pick HEAD before cherry-picking $Sha." } -$info = (git log -1 --format='%an%x09%ae%x09%aI%x09%cn%x09%ce%x09%cI' $fullSha) -split "`t" +# Use ASCII unit separator (\x1F) as the delimiter — that byte is forbidden +# in git ident strings, so it cannot collide with a tab inside an author +# or committer name (git allows tabs in idents). Validate field count +# defensively before binding to GIT_* env vars. +$us = [char]0x1F +$info = (git log -1 --format="%an${us}%ae${us}%aI${us}%cn${us}%ce${us}%cI" $fullSha) -split $us +if ($LASTEXITCODE -ne 0) { throw "git log failed resolving author/committer for $fullSha (exit $LASTEXITCODE)." } +if ($info.Count -ne 6) { throw "Unexpected field count parsing author/committer for ${fullSha}: got $($info.Count) (expected 6)." } # Capture caller's existing GIT_AUTHOR_* / GIT_COMMITTER_* env so the finally # block can restore them. Higher-level automation may have set these diff --git a/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 b/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 deleted file mode 100644 index f602f7871..000000000 --- a/.github/skills/upstream-sync/scripts/05-finalize-pr.ps1 +++ /dev/null @@ -1,111 +0,0 @@ -<# -.SYNOPSIS - Push the sync branch and open a PR. No state file, no extra commits. - -.DESCRIPTION - The branch already carries the cherry-picked commits (each with its - `(cherry picked from commit )` trailer — that IS the watermark - the next run reads). We just push it and open the PR. - - Called by the agent after a clean cherry-pick batch (and build pass). - -.PARAMETER Branch - Sync branch name (must already exist locally and be checked out / pushable). - -.PARAMETER UpstreamHeadSha - Upstream/main SHA at fetch time. Used only in the PR title. - -.PARAMETER PickedCount - Number of commits cherry-picked in this batch. Used only in the banner. - -.PARAMETER PrBody - Full markdown body for the PR. The banner (squash-warning + review-fix - policy) is prepended automatically. - -.PARAMETER AutoMergeStrategy - rebase | merge | none. Passed to `gh pr merge --auto`. - -.OUTPUTS - PR URL on stdout. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] [string] $Branch, - [Parameter(Mandatory)] [string] $UpstreamHeadSha, - [Parameter(Mandatory)] [int] $PickedCount, - [Parameter(Mandatory)] [string] $PrBody, - [ValidateSet('rebase','merge','none')] [string] $AutoMergeStrategy = 'none' -) - -. "$PSScriptRoot/Common.ps1" - -$banner = @" -> [!WARNING] -> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked -> upstream commit into one, destroying per-commit attribution, original -> author dates, the ``(cherry picked from commit )`` trailers that the -> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. -> Merge with **"Rebase and merge"** (preferred — flat history, all -> $PickedCount commit(s) land individually) or **"Create a merge commit"**. - -> [!NOTE] -> **Review-fix policy.** Only build-blocking fixes belong on this branch -> as **one** focused extra commit. All other Copilot / human review -> feedback goes into a **follow-up PR** based on this PR's head. See -> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). - ---- - -"@ - -$bodyPath = New-TemporaryFile -$bodyContent = $banner + $PrBody -[System.IO.File]::WriteAllText($bodyPath, $bodyContent, (New-Object System.Text.UTF8Encoding($false))) - -$shortTo = $UpstreamHeadSha.Substring(0,9) -$title = "chore(upstream): sync microsoft/terminal up to $shortTo" - -git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { - Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue - throw "git push failed for $Branch." -} - -# `gh pr create` on Windows occasionally fails with "Head sha can't be blank" -# right after a push — retry up to 3x with a short delay. stderr goes to a -# separate temp file so a `gh` version-update notice can't displace the -# URL as the "last line" of merged output. -$prUrl = $null -$errFile = [System.IO.Path]::GetTempFileName() -try { - for ($attempt = 1; $attempt -le 3; $attempt++) { - Set-Content -LiteralPath $errFile -Value '' -NoNewline - $prUrl = gh pr create -R microsoft/intelligent-terminal --base main --head $Branch --title $title --body-file $bodyPath 2>$errFile | Select-Object -Last 1 - if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } - $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } - Write-Warning "gh pr create attempt $attempt failed (exit $LASTEXITCODE): stdout='$prUrl' stderr='$errText'" - Start-Sleep -Seconds 5 - } - if ($LASTEXITCODE -ne 0 -or $prUrl -notmatch '^https://github.com/') { - $errText = if (Test-Path -LiteralPath $errFile) { (Get-Content -Raw -LiteralPath $errFile) } else { '' } - throw "gh pr create did not return a PR URL after 3 attempts. Last stdout: '$prUrl'. Last stderr: '$errText'." - } -} -finally { - Remove-Item -LiteralPath $bodyPath -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} - -$prUrl = $prUrl.Trim() - -if ($AutoMergeStrategy -ne 'none') { - $strategyFlag = "--$AutoMergeStrategy" - gh pr merge -R microsoft/intelligent-terminal $prUrl $strategyFlag --auto --delete-branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } - if ($LASTEXITCODE -ne 0) { - Write-Warning "gh pr merge --auto failed. PR is open at $prUrl; merge manually with '$AutoMergeStrategy' strategy (NOT squash)." - } else { - [Console]::Error.WriteLine("Auto-merge armed with strategy: $AutoMergeStrategy") - } -} - -return $prUrl From 55f0026af1191cc5abecb377395d2ee179ba6e99 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:01:33 +0800 Subject: [PATCH 58/82] Round 8: prune the orchestration to its essential algorithms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per operator review: any script that is mostly a thin wrapper around git / gh should not exist as a script — inline the recipe into SKILL.md so the agent reads what it executes. Only scripts whose value is a non-trivial algorithm survive. Deleted: - scripts/01-fetch-upstream.ps1 (3 git calls; inlined into step 3) - scripts/06-open-stuck-issue.ps1 (1 git push + 1 gh issue create + body string; inlined into step 5a as plain-markdown body — no fenced YAML metadata, since closing the labeled issue IS the lock-clear signal and nothing parses the body back) - scripts/06b-open-build-stuck-issue.ps1 (build failures no longer file an issue; agent surfaces the failure + log path/tail to the operator and exits per step 7a) - scripts/Common.ps1 (Format-Iso8601 was a wrapper around .ToString('o'); Format-StuckYamlBlock served the deleted YAML metadata block; the remaining 2-line PowerShell prelude was inlined into each script) Remaining scripts (real algorithms only): - 02-compute-pending.ps1 — patch-id-aware pending list + revert-pair detect - 03-cherry-pick-one.ps1 — cherry-pick with author/date pinning + Tier-0/1 - 04-try-build.ps1 — bz no_clean with timeout + log capture Copilot round-8 findings addressed in the same commit: - SKILL.md: removed duplicate 'Surface \' line at step 8. - 02-compute-pending.ps1: fixed misleading comment that mentioned -Since on the cherry-pick walk; the code actually filters by git merge-base --is-ancestor against the watermark. - 03-cherry-pick-one.ps1: documented the 'error' field in the JSON output schema and initialized it to '' so the field is always present regardless of status. References updated to match: - references/03-conflict-triage.md Tier 3: plain-markdown body; no YAML. Tier 4: surface + exit, no issue. - references/04-build-verification.md: log tail goes to operator, not to an issue. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 91 +++++++---- .../references/03-conflict-triage.md | 36 ++--- .../references/04-build-verification.md | 27 ++-- .../scripts/01-fetch-upstream.ps1 | 46 ------ .../scripts/02-compute-pending.ps1 | 10 +- .../scripts/03-cherry-pick-one.ps1 | 12 +- .../upstream-sync/scripts/04-try-build.ps1 | 5 +- .../scripts/06-open-stuck-issue.ps1 | 131 ---------------- .../scripts/06b-open-build-stuck-issue.ps1 | 141 ------------------ .../skills/upstream-sync/scripts/Common.ps1 | 40 ----- 10 files changed, 114 insertions(+), 425 deletions(-) delete mode 100644 .github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 delete mode 100644 .github/skills/upstream-sync/scripts/Common.ps1 diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index dd5c41557..f51b9b416 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -53,7 +53,7 @@ Every persistent fact lives in the source that owns it: | What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived inline by [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | | What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, then drop SHAs older than (or equal to) the watermark above. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | | Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | -| What does the lock mean? | A fenced YAML block carrying `# wta-state` in the issue body holds `tier`, `kind`, `stuck_on_sha`/`findings_hash`, etc. | +| What does the lock mean? | The issue body (plain markdown — no machine-parseable block) names the stuck commit, the branch, and the conflicting paths. Closing the issue clears the lock. | | Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | ### Why cherry-pick (and not rebase or merge) @@ -102,10 +102,17 @@ The fresh suffix means re-runs never collide with stale local branches. ### 3. Fetch upstream ```pwsh -$upstreamSha = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +if (-not (git remote get-url upstream 2>$null)) { + git remote add upstream https://github.com/microsoft/terminal.git +} +git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } +$upstreamSha = (git rev-parse upstream/main).Trim() ``` -Returns the `upstream/main` SHA. Creates the `upstream` remote if missing. +If `upstream` already exists with a different URL (someone configured it +to a private mirror, etc.), surface that to the operator and exit — don't +silently rewrite it. ### 4. Compute pending @@ -139,12 +146,49 @@ Branch on `$pick.status`: #### 5a. On stuck — open the Tier-3 issue and exit +Push the branch, ensure the lock label exists, and file the issue with a +plain-markdown body. Closing the issue is the lock-clear signal — no +machine-readable metadata is needed. + ```pwsh -$issueUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 ` - -Branch $branch ` - -StuckSha $sha ` - -ConflictPaths $pick.conflict_paths ` - -StuckError $pick.error +git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } + +# Idempotent — ignores "already exists" exit code. +gh label create 'upstream-sync-stuck' ` + --color B60205 ` + --description 'Upstream sync paused — close to clear the lock' ` + -R microsoft/intelligent-terminal 2>$null + +$author = git log -1 --format='%an <%ae>' $sha +$subject = git log -1 --format='%s' $sha +$shortSha = $sha.Substring(0,9) +$pathLines = ($pick.conflict_paths | ForEach-Object { "- ``$_``" }) -join "`n" +$body = @" +> [!CAUTION] +> Upstream sync stopped at a conflict that needs human judgment. +> **Close this issue when resolved — that IS the lock-clear signal.** + +**Stuck on:** ``$sha`` — $subject +**Upstream commit:** https://github.com/microsoft/terminal/commit/$sha +**Author:** $author +**Sync branch:** ``$branch`` + +**Conflicting paths:** +$pathLines + +$(if ($pick.error) { "**Error:** ``$($pick.error)``" }) + +See [references/03-conflict-triage.md](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/03-conflict-triage.md) for the resolution rubric. +"@ + +$body | Set-Content -Encoding utf8 .issuebody +$issueUrl = gh issue create -R microsoft/intelligent-terminal ` + --title "Upstream sync stuck at $shortSha" ` + --label upstream-sync-stuck ` + --body-file .issuebody +if ($LASTEXITCODE -ne 0) { throw "gh issue create failed." } +Remove-Item .issuebody ``` Surface `$issueUrl` and `$branch` to the operator. The human is expected to @@ -190,21 +234,24 @@ Branch on `$build.kind`: - `"build-inconclusive"` — go to step 7a (timeout, treated as a real failure unless the operator explicitly opts out). -#### 7a. On build failure — open the Tier-4 issue and exit +#### 7a. On build failure — surface and exit + +The fix attempt didn't land or the failure is too big for a one-commit +fix. Don't open an issue. Surface the failure to the operator and exit — +they'll either fix the underlying defect on `main` and re-run, or push a +manual fix commit on top of `$branch` and continue to step 8 by hand. ```pwsh -$issueUrl = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 ` - -Branch $branch ` - -Kind $build.kind ` - -PickedCount $picked.Count ` - -BuildExitCode $build.exit_code ` - -BuildLogTail $build.log_tail ` - -BuildLogPath $build.log_path +[Console]::Error.WriteLine("Build failed after upstream sync:") +[Console]::Error.WriteLine(" branch: $branch") +[Console]::Error.WriteLine(" kind: $($build.kind)") +[Console]::Error.WriteLine(" exit_code: $($build.exit_code)") +[Console]::Error.WriteLine(" log_path: $($build.log_path)") +[Console]::Error.WriteLine("--- log tail ---") +[Console]::Error.WriteLine($build.log_tail) +exit 1 ``` -The branch is pushed by the script. Surface `$issueUrl` to the operator -and EXIT. - ### 8. Finalize the PR The agent (you) push the branch and open the PR directly with `git` and @@ -287,8 +334,6 @@ gh pr merge -R microsoft/intelligent-terminal $prUrl --rebase --auto --delete-br Surface `$prUrl` to the operator. Done. -Surface `$prUrl` to the operator. Done. - ### Direct-to-main (admin-only escape hatch) If the operator explicitly says "skip the PR, push straight to main": after @@ -445,10 +490,6 @@ fixes. - [references/03-known-conflicts.md](./references/03-known-conflicts.md) — files that always need a fixed resolution. - [references/04-build-verification.md](./references/04-build-verification.md) — try-build pipeline expectations. - [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. -- [scripts/01-fetch-upstream.ps1](./scripts/01-fetch-upstream.ps1) — fetch microsoft/terminal main; return SHA. - [scripts/02-compute-pending.ps1](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). - [scripts/03-cherry-pick-one.ps1](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. - [scripts/04-try-build.ps1](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. -- [scripts/06-open-stuck-issue.ps1](./scripts/06-open-stuck-issue.ps1) — Tier-3 stuck issue (mid-pick conflict). -- [scripts/06b-open-build-stuck-issue.ps1](./scripts/06b-open-build-stuck-issue.ps1) — Tier-4 stuck issue (build failed after clean batch). -- [scripts/Common.ps1](./scripts/Common.ps1) — the only two helpers shared by 2+ scripts. diff --git a/.github/skills/upstream-sync/references/03-conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md index 4a6d68b3e..107132554 100644 --- a/.github/skills/upstream-sync/references/03-conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -91,21 +91,24 @@ Anything not resolved by Tier 0–2: ~~~pwsh git cherry-pick --abort -# Open the labeled stuck issue (06-open-stuck-issue.ps1) — issue body -# carries the fenced YAML "# wta-state" block with stuck_on_sha + branch. -# Surface the issue URL + branch to the operator and exit. +# Push the branch, ensure the 'upstream-sync-stuck' label exists, and +# file a plain-markdown issue with the stuck SHA, upstream URL, author, +# branch name, and conflicting paths. See SKILL.md step 5a for the +# exact recipe. Surface the issue URL + branch to the operator and exit. ~~~ -The issue body (built by [`scripts/06-open-stuck-issue.ps1`](../scripts/06-open-stuck-issue.ps1)) **must** include: +The issue body **must** include: - The conflicting commit SHA, subject, author, and upstream URL. -- The list of conflicting paths with a one-line classification each - (`semantic-overlap`, `deleted-by-us`, `binary-merge`, etc.). +- The list of conflicting paths. - The exact local branch name where the human picks up. - The exact resume action: resolve on the stuck branch, merge a PR that keeps the `(cherry picked from commit )` trailer, then CLOSE the stuck issue (that's the lock-clear signal — no script). +No fenced YAML metadata block is needed. Closing the labeled issue IS +the lock-clear signal; nothing parses the body back. + ## Tier 4 — Post-pick build failed The cherry-picks all applied cleanly, but [`scripts/04-try-build.ps1`](../scripts/04-try-build.ps1) @@ -115,20 +118,13 @@ build-then-finalize ordering and the one-focused-fix-commit rule. | Sub-kind | Trigger | Action | |---|---|---| -| **build-failed** | `bz no_clean` exited non-zero within timeout | Try ONE focused build-fix commit per [SKILL.md step 7](../SKILL.md#7-build). If that fails or scope is too large → open [Tier-4 stuck issue](../scripts/06b-open-build-stuck-issue.ps1) and exit. | -| **build-inconclusive** | Wall-clock cap (default 45 min) hit | Open Tier-4 stuck issue immediately (don't guess at fixing a hang). | - -Tier-4 state lives in the body of an open `upstream-sync-stuck` labeled -issue (separate per kind by `findings_hash`); any such open issue causes -the scheduler to skip. Clear by **closing the issue** after the human -resolves the validation failure — either by merging a fix PR (whose -trailers will advance the watermark) or by fixing the underlying defect -on `main` directly (the next run re-attempts the same range and -re-validates). No script needed. - -The Tier-4 report includes a `findings_hash` (16-hex prefix). Re-runs -that produce the same hash mean the underlying defect is unchanged; -a changed hash means validation has moved to a new failure mode. +| **build-failed** | `bz no_clean` exited non-zero within timeout | Try ONE focused build-fix commit per [SKILL.md step 7](../SKILL.md#7-build). If that fails or scope is too large → surface the failure to the operator and exit (no issue is filed). | +| **build-inconclusive** | Wall-clock cap (default 45 min) hit | Surface the timeout to the operator and exit (don't guess at fixing a hang). | + +No stuck issue is filed for build failures. The operator either fixes +the underlying defect on `main` and re-runs (the next sync re-attempts +the same range and re-validates), or pushes a manual fix commit on top +of the sync branch and finishes step 8 by hand. ## Line endings diff --git a/.github/skills/upstream-sync/references/04-build-verification.md b/.github/skills/upstream-sync/references/04-build-verification.md index fa30c8905..a3f3d6081 100644 --- a/.github/skills/upstream-sync/references/04-build-verification.md +++ b/.github/skills/upstream-sync/references/04-build-verification.md @@ -3,7 +3,8 @@ Post-pick hard gate. Runs **after** the cherry-pick loop and **before** the PR is opened. If the build fails, the agent either lands one focused build-fix commit on the same branch (so it ships in the same PR) or — if -the fix is too large / scope creep — opens a Tier-4 stuck issue and exits. +the fix is too large / scope creep — surfaces the failure to the operator +and exits without filing an issue. The full orchestration around try-build lives in [`SKILL.md` → Run a sync → step 7](../SKILL.md#7-build); this file is @@ -20,8 +21,8 @@ catches that with zero false positives — `git` cannot. Toolchain provisioning (e.g. `PlatformToolset` v143/v145) is treated as the operator's problem, not the scheduler's: an under-provisioned host -just keeps tripping the build gate and the human notices via the open -stuck issue. We intentionally do **not** auto-bump toolset versions in +just keeps tripping the build gate and the human notices on the next +re-run. We intentionally do **not** auto-bump toolset versions in the repo on behalf of a single host. ## Try-build (`scripts/04-try-build.ps1`) @@ -47,7 +48,7 @@ Output (returned as JSON on stdout): | `duration_ms` | Wall-clock ms | | `command` | The build command that was run | | `log_path` | Repo-relative path to the full log (under `Generated Files/upstream-sync//build-logs/`, gitignored) | -| `log_tail` | Last ~200 lines for inline display in the stuck issue | +| `log_tail` | Last ~200 lines for inline display to the operator | Timeout: @@ -58,23 +59,23 @@ Timeout: The agent's decision tree is in [`SKILL.md` step 7](../SKILL.md#7-build). In short: try ONE focused fix commit when the cause is mechanical and -clearly caused by the pick batch; otherwise open the Tier-4 issue and -exit. Do **not** pile up multiple fix commits — the one-fix-per-PR rule -exists so the cherry-pick PR stays auditable as "upstream batch + at -most one mechanical fix". +clearly caused by the pick batch; otherwise surface the failure to the +operator and exit. Do **not** pile up multiple fix commits — the +one-fix-per-PR rule exists so the cherry-pick PR stays auditable as +"upstream batch + at most one mechanical fix". ## When the build fails for fork-unrelated reasons If a flaky build (transient toolchain glitch, env issue, missing PlatformToolset, ...) trips the gate: -1. The Tier-4 stuck issue gives a clear log tail. -2. A human can re-run the build locally, confirm it's transient or fix - the host, then **close the stuck issue** to clear the lock. -3. The next scheduler tick re-attempts the same pick range from scratch. +1. The operator sees the log tail surfaced from step 7a. +2. They re-run the build locally, confirm it's transient or fix + the host, then re-run the sync. The next attempt re-picks the same + range from scratch and re-validates. Distinguishing transient-build from real-pick-broke-build is left to -the human reviewing the issue — too noisy to automate, and the cost +the human reviewing the failure — too noisy to automate, and the cost of a manual cross-check is small (~once per N runs). ## Build artifacts diff --git a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 b/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 deleted file mode 100644 index 8e7b4583b..000000000 --- a/.github/skills/upstream-sync/scripts/01-fetch-upstream.ps1 +++ /dev/null @@ -1,46 +0,0 @@ -<# -.SYNOPSIS - Ensure upstream remote exists and fetch upstream/main. -.OUTPUTS - Writes the current upstream/main SHA to stdout. -#> -[CmdletBinding()] -param( - [string] $UpstreamUrl = 'https://github.com/microsoft/terminal.git' -) - -. "$PSScriptRoot/Common.ps1" - -# Ensure-UpstreamRemote (inlined — single-use). Adds the `upstream` remote -# if missing; bails if it points somewhere unexpected. Normalizes common URL -# variants (https vs ssh, with/without trailing `.git`, with/without trailing -# slash) so the same logical repo configured under any of those forms is -# accepted. -function _NormalizeRemoteUrl { - param([string] $Url) - $u = $Url.Trim().TrimEnd('/').ToLowerInvariant() - # git@github.com:owner/repo(.git)? -> github.com/owner/repo - if ($u -match '^git@([^:]+):(.+?)(\.git)?$') { - $u = "$($Matches[1])/$($Matches[2])" - } - # https://host/owner/repo(.git)? -> host/owner/repo - elseif ($u -match '^https?://([^/]+)/(.+?)(\.git)?$') { - $u = "$($Matches[1])/$($Matches[2])" - } - return $u -} - -$existing = git remote get-url upstream 2>$null -if ($LASTEXITCODE -ne 0) { - git remote add upstream $UpstreamUrl | Out-Null - if ($LASTEXITCODE -ne 0) { throw "Failed to add 'upstream' remote." } -} elseif ((_NormalizeRemoteUrl $existing) -ne (_NormalizeRemoteUrl $UpstreamUrl)) { - throw "Remote 'upstream' points at '$($existing.Trim())' (expected '$UpstreamUrl'). Fix the remote before running upstream-sync." -} - -git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } - -$sha = git rev-parse upstream/main -if ($LASTEXITCODE -ne 0) { throw "git rev-parse upstream/main failed." } -return $sha.Trim() diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index c7460dcea..63954e40a 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -9,7 +9,7 @@ within the range and drops them, detects upstream-empty commits and drops them, and emits the final pending list as JSON. - `01-fetch-upstream.ps1` must have been run first (we need upstream/main + Step 3's inline recipe must have been run first (we need upstream/main in this clone). .OUTPUTS @@ -25,9 +25,10 @@ [CmdletBinding()] param() -. "$PSScriptRoot/Common.ps1" +$ErrorActionPreference = 'Stop' +Set-StrictMode -Version Latest -# --- Inlined helpers (single-use; see Common.ps1 comment for why) ---------- +# --- Inlined helpers (single-use) ---------- function Resolve-FullCommitSha { param([Parameter(Mandatory)] [string] $Sha) @@ -60,7 +61,8 @@ function Get-LastSyncedUpstreamSha { function Get-PendingUpstreamShas { # Patch-id-aware oldest-first list of upstream/main commits not yet on # origin/main. `--cherry-pick` drops picked-then-reverted pairs by patch - # ID; -Since further trims by ancestry to keep the walk fast. + # ID. The optional ancestry filter drops any SHA already reachable from + # the watermark, so the walk stays bounded even on a long-untouched fork. param([string] $Since) $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 67cb50dbb..803a2210c 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -13,15 +13,20 @@ .OUTPUTS JSON status object on stdout: - { "sha": "...", "status": "picked|skipped-empty|stuck", - "tier0_paths": ["..."], "conflict_paths": ["..."] } + { "sha": "...", + "status": "picked" | "skipped-empty" | "stuck", + "tier0_paths": ["..."], // files Tier-0 auto-resolved (may be empty) + "conflict_paths": ["..."], // unmerged files when status = 'stuck' + "error": "..." // human-readable explanation when status = 'stuck'; empty otherwise + } #> [CmdletBinding()] param( [Parameter(Mandatory)] [string] $Sha ) -. "$PSScriptRoot/Common.ps1" +$ErrorActionPreference = 'Stop' +Set-StrictMode -Version Latest function Get-KnownConflicts { $md = Join-Path (Split-Path $PSScriptRoot -Parent) 'references/03-known-conflicts.md' @@ -54,6 +59,7 @@ $result = [ordered] @{ status = 'unknown' tier0_paths = @() conflict_paths = @() + error = '' } # Capture upstream's author AND committer identity + dates so the diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index d9bb9f914..84a640437 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -39,9 +39,10 @@ param( [string] $LogDir ) -. "$PSScriptRoot/Common.ps1" +$ErrorActionPreference = 'Stop' +Set-StrictMode -Version Latest -# --- Inlined helpers (single-use; see Common.ps1 comment for why) ---------- +# --- Inlined helpers (single-use) ---------- function Get-RepoRoot { $r = git rev-parse --show-toplevel 2>$null diff --git a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 deleted file mode 100644 index 06f1170d6..000000000 --- a/.github/skills/upstream-sync/scripts/06-open-stuck-issue.ps1 +++ /dev/null @@ -1,131 +0,0 @@ -<# -.SYNOPSIS - Open a Tier-3 stuck issue (cherry-pick stopped at a real merge conflict). - -.DESCRIPTION - The "lock" IS this open issue — the next sync run queries - `gh issue list --label upstream-sync-stuck --state open` and bails when - one exists. The issue body carries a fenced ```yaml # wta-state``` block - with enough metadata to recognize the same failure on a re-run. - - Cleared by: a human closes the issue. That's it. - -.PARAMETER Branch - Stuck sync branch name. - -.PARAMETER StuckSha - The upstream SHA the pick got stuck on. - -.PARAMETER ConflictPaths - Files that 03-cherry-pick-one.ps1 left unmerged. - -.PARAMETER StuckError - Optional error string captured from 03 (e.g. "git cherry-pick exited 1 - with no conflict paths" for a non-conflict failure). - -.PARAMETER ExtraBody - Optional markdown to append after the YAML block (resume instructions, - log excerpts, etc.). - -.OUTPUTS - Issue URL on stdout. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] [string] $Branch, - [Parameter(Mandatory)] [string] $StuckSha, - [string[]] $ConflictPaths = @(), - [string] $StuckError = '', - [string] $ExtraBody = '' -) - -. "$PSScriptRoot/Common.ps1" - -# Push the stuck branch so the human can resume on it. -git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push stuck branch — issue still being filed for visibility." } - -$shortSha = $StuckSha.Substring(0,9) - -$subj = (git log -1 --format='%s' $StuckSha 2>$null) -if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($subj)) { - throw "git log failed for stuck SHA '$StuckSha' (exit $LASTEXITCODE). Refusing to file an issue with a missing subject." -} -$subj = $subj.Trim() - -$author = (git log -1 --format='%an <%ae>' $StuckSha 2>$null) -if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($author)) { - throw "git log failed to resolve author for stuck SHA '$StuckSha' (exit $LASTEXITCODE)." -} -$author = $author.Trim() - -$upstreamUrl = "https://github.com/microsoft/terminal/commit/$StuckSha" -$title = "Upstream sync stuck at ${shortSha}: $subj" - -$yamlBlock = Format-StuckYamlBlock @{ - tier = '3' - kind = 'cherry-pick-conflict' - stuck_on_sha = $StuckSha - branch = $Branch - at = Format-Iso8601 - host = $env:COMPUTERNAME - conflict_count = $ConflictPaths.Count - error = $StuckError -} - -$conflictList = if ($ConflictPaths.Count -gt 0) { - ($ConflictPaths | ForEach-Object { "- ``$_``" }) -join "`n" -} else { - '_(no conflict paths reported — see error)_' -} - -$body = @" -> [!CAUTION] -> **Upstream sync stopped at a conflict that needs human judgment.** -> -> The scheduler will keep skipping its runs until this issue is **closed**. -> Closing the issue IS the lock-clear signal — no separate script needed. - -**Stuck on:** ``$StuckSha`` — $subj -**Upstream commit:** [$shortSha]($upstreamUrl) -**Author:** $author -**Sync branch:** ``$Branch`` (push attempted — run ``git ls-remote --heads origin $Branch`` to verify it landed) - -**Conflicting paths:** -$conflictList - -$yamlBlock - ---- - -$ExtraBody -"@ - -$tmp = New-TemporaryFile -[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - -# Ensure label exists (best-effort). -R pinned because an `upstream` remote -# can make `gh` default to microsoft/terminal where this account has no -# label-create permission. -gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual conflict' -R microsoft/intelligent-terminal 2>$null | Out-Null - -# Capture stderr to a separate temp file so a `gh` warning on stderr can't -# displace the URL as the last line of merged output. -$errFile = [System.IO.Path]::GetTempFileName() -$errText = '' -$issueUrl = $null -$ghExit = 0 -try { - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } -} -finally { - Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} -if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" -} - -return $issueUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 b/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 deleted file mode 100644 index 67522c3bb..000000000 --- a/.github/skills/upstream-sync/scripts/06b-open-build-stuck-issue.ps1 +++ /dev/null @@ -1,141 +0,0 @@ -<# -.SYNOPSIS - Open a Tier-4 stuck issue (build failed after a clean cherry-pick batch). - -.DESCRIPTION - Counterpart to 06-open-stuck-issue.ps1 (Tier-3 = mid-pick conflict). - Tier-4 means all picks completed cleanly but `04-try-build.ps1` said NO - and the agent couldn't auto-fix. - - Same lock model: the open issue IS the lock; closing it clears. - -.PARAMETER Branch - Sync branch name. - -.PARAMETER Kind - 'build-failed' or 'build-inconclusive'. - -.PARAMETER PickedCount - How many commits landed cleanly before the build failed. - -.PARAMETER BuildExitCode - Exit code from 04-try-build.ps1. - -.PARAMETER BuildLogTail - Tail of the build log to embed in the issue body (last ~200 lines). - -.PARAMETER BuildLogPath - Repo-relative path to the full build log (gitignored — for operator - reference, not committed). - -.OUTPUTS - Issue URL on stdout. -#> -[CmdletBinding()] -param( - [Parameter(Mandatory)] [string] $Branch, - [Parameter(Mandatory)] [ValidateSet('build-failed','build-inconclusive')] [string] $Kind, - [Parameter(Mandatory)] [int] $PickedCount, - [Parameter(Mandatory)] [int] $BuildExitCode, - [string] $BuildLogTail = '', - [string] $BuildLogPath = '' -) - -. "$PSScriptRoot/Common.ps1" - -# Findings hash — stable across runs of the same broken batch so a future -# scheduler tick can recognize "same failure as last time" and skip -# re-opening. Inlined (single use). -function Get-FindingsHash { - param([Parameter(Mandatory)] $Findings) - $norm = ($Findings | ConvertTo-Json -Depth 8 -Compress) - $sha = [System.Security.Cryptography.SHA256]::Create() - try { - $hash = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($norm)) - return ([System.BitConverter]::ToString($hash) -replace '-','').ToLowerInvariant().Substring(0,16) - } finally { $sha.Dispose() } -} - -$findingsForHash = if ($Kind -eq 'build-failed') { - @([ordered] @{ exit_code = $BuildExitCode; tail_excerpt = ($BuildLogTail -split "`n" | Select-Object -Last 20) -join "`n" }) -} else { - @([ordered] @{ kind = 'inconclusive'; exit_code = $BuildExitCode }) -} -$findingsHash = Get-FindingsHash $findingsForHash - -# Push the sync branch so the human can resume on it. -git push -u origin $Branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { Write-Warning "Could not push sync branch — issue still being filed for visibility." } - -$titleKindLabel = if ($Kind -eq 'build-failed') { 'build failure' } else { 'build inconclusive (timeout)' } -$title = "Upstream sync stuck after $PickedCount clean picks: $titleKindLabel ($findingsHash)" - -$yamlBlock = Format-StuckYamlBlock @{ - tier = '4' - kind = $Kind - branch = $Branch - findings_hash = $findingsHash - picked_count = $PickedCount - at = Format-Iso8601 - host = $env:COMPUTERNAME -} - -$logSection = if ($BuildLogTail) { - @" -
Build log tail (last ~200 lines) - -`````` -$BuildLogTail -`````` - -
-"@ -} else { '' } - -$logPathLine = if ($BuildLogPath) { - "**Full log:** ``$BuildLogPath`` (gitignored; on the host that ran the build)" -} else { '' } - -$body = @" -> [!CAUTION] -> **Upstream sync stopped after build failed.** -> -> All $PickedCount cherry-pick(s) applied cleanly, but ``04-try-build.ps1`` -> said NO before the PR could be finalized. Stop reason: **$Kind** (exit $BuildExitCode). -> -> The scheduler will keep skipping its runs until this issue is **closed**. - -**Sync branch:** ``$Branch`` (push attempted — run ``git ls-remote --heads origin $Branch`` to verify it landed) -**Findings hash:** ``$findingsHash`` (re-runs of the same broken batch will match) -$logPathLine - -$yamlBlock - ---- - -$logSection -"@ - -$tmp = New-TemporaryFile -[System.IO.File]::WriteAllText($tmp, $body, (New-Object System.Text.UTF8Encoding($false))) - -gh label create 'upstream-sync-stuck' --color 'B60205' --description 'Upstream sync blocked on a manual issue' -R microsoft/intelligent-terminal 2>$null | Out-Null - -$errFile = [System.IO.Path]::GetTempFileName() -$errText = '' -$issueUrl = $null -$ghExit = 0 -try { - $issueUrl = gh issue create -R microsoft/intelligent-terminal --title $title --label 'upstream-sync-stuck' --body-file $tmp 2>$errFile | Select-Object -Last 1 - $ghExit = $LASTEXITCODE - if (Test-Path -LiteralPath $errFile) { $errText = (Get-Content -Raw -LiteralPath $errFile) } -} -finally { - Remove-Item -LiteralPath $tmp -Force -ErrorAction SilentlyContinue - Remove-Item -LiteralPath $errFile -Force -ErrorAction SilentlyContinue -} -if ($ghExit -ne 0 -or $issueUrl -notmatch '^https://github.com/') { - throw "gh issue create failed (exit $ghExit): stdout='$issueUrl' stderr='$errText'" -} - -return $issueUrl.Trim() diff --git a/.github/skills/upstream-sync/scripts/Common.ps1 b/.github/skills/upstream-sync/scripts/Common.ps1 deleted file mode 100644 index 8eb99a17f..000000000 --- a/.github/skills/upstream-sync/scripts/Common.ps1 +++ /dev/null @@ -1,40 +0,0 @@ -# Common.ps1 — shared helpers for upstream-sync scripts. -# Dot-source from each script: . "$PSScriptRoot/Common.ps1" -# -# Only contains helpers used by 2+ scripts. Single-use helpers live inline -# in the script that uses them. -# -# The skill keeps no `state.json`. Watermark is the most recent -# `(cherry picked from commit )` trailer on origin/main (read by -# 02-compute-pending.ps1). Stuck-lock is any OPEN gh issue with the -# `upstream-sync-stuck` label (agent queries `gh issue list` directly). - -$ErrorActionPreference = 'Stop' -Set-StrictMode -Version Latest - -$script:WtaStateFence = '# wta-state' - -function Format-StuckYamlBlock { - # Fenced ```yaml ... # wta-state ... ``` block embedded in stuck-issue - # bodies. Values are single-quoted with `'` -> `''` escaping so colons - # and leading dashes round-trip through the issue body. Newlines in - # values are folded to a single space (multi-line values are not - # preserved). Used by 06-open-stuck-issue.ps1 and - # 06b-open-build-stuck-issue.ps1. - param([Parameter(Mandatory)] [hashtable] $Fields) - $lines = @('```yaml', $script:WtaStateFence) - foreach ($k in $Fields.Keys) { - $raw = "$($Fields[$k])" - $folded = $raw -replace '\r?\n', ' ' - $escaped = $folded -replace "'", "''" - $lines += ("{0}: '{1}'" -f $k, $escaped) - } - $lines += '```' - return ($lines -join "`n") -} - -function Format-Iso8601 { - # Used by 06 + 06b for the `at` field of the stuck-issue YAML block. - param([DateTime] $When = (Get-Date)) - return $When.ToString('yyyy-MM-ddTHH:mm:sszzz') -} From 10c576cf119022d47bb595206cbf4c3347a7da61 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:13:46 +0800 Subject: [PATCH 59/82] Round 9: address Copilot post-prune findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 03-cherry-pick-one.ps1: - Populate \rror\ on the unhandled-conflict stuck path (line 187 block). - Populate \rror\ on the Tier-0-continue-failed stuck path; also capture remaining unmerged files so the Tier-3 issue lists something concrete instead of empty conflict_paths. 04-try-build.ps1: - Help text used to say "before finalizing the PR (05)" — there is no step 05 in this skill. Now says "before finalizing the PR (SKILL.md step 8)". SKILL.md step 5a: - \gh label create\ no longer silently swallows every failure. Now inspects stderr for "already exists" and throws on any other failure (auth, repo typo, etc.) so the subsequent gh issue create doesn't fail with a confusing downstream error. SKILL.md step 8: - \git push -u origin \\ now checks \\0\ and throws on failure before attempting \gh pr create\ (matches the gh-CLI scripting convention used elsewhere in the skill). references/03-conflict-triage.md Tier-0: - Tier-0 algorithm snippet no longer shows a fake \git merge-file --union\ body for the 'union' strategy. Added an explicit callout that 'union' is recognized by the parser but escalates to Tier-3 (matches the current 03-cherry-pick-one.ps1 behavior). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 14 ++++++++++---- .../upstream-sync/references/03-conflict-triage.md | 4 +++- .../upstream-sync/scripts/03-cherry-pick-one.ps1 | 6 ++++++ .../skills/upstream-sync/scripts/04-try-build.ps1 | 7 ++++--- 4 files changed, 23 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index f51b9b416..df0c1fa86 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -154,11 +154,16 @@ machine-readable metadata is needed. git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } -# Idempotent — ignores "already exists" exit code. -gh label create 'upstream-sync-stuck' ` +# Idempotent — `gh label create` returns non-zero when the label already +# exists. We swallow that specific case but surface everything else so an +# auth/repo-typo failure doesn't get silently absorbed. +$labelOut = gh label create 'upstream-sync-stuck' ` --color B60205 ` --description 'Upstream sync paused — close to clear the lock' ` - -R microsoft/intelligent-terminal 2>$null + -R microsoft/intelligent-terminal 2>&1 +if ($LASTEXITCODE -ne 0 -and ($labelOut -notmatch 'already exists')) { + throw "gh label create failed: $labelOut" +} $author = git log -1 --format='%an <%ae>' $sha $subject = git log -1 --format='%s' $sha @@ -310,7 +315,8 @@ with "Head sha can't be blank" right after a push — retry up to 3× with a short delay: ```pwsh -git push -u origin $branch +git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } $short = $upstreamSha.Substring(0,9) $title = "chore(upstream): sync microsoft/terminal up to $short" diff --git a/.github/skills/upstream-sync/references/03-conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md index 107132554..1866dbd4b 100644 --- a/.github/skills/upstream-sync/references/03-conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -22,7 +22,7 @@ foreach ($p in $conflictingPaths) { switch ($entry.Strategy) { 'take-upstream' { git checkout --theirs -- $p; git add -- $p } 'take-ours' { git checkout --ours -- $p; git add -- $p } - 'union' { git merge-file --union ... } + 'union' { <# escalates — Tier-3 #> } } } git cherry-pick --continue --no-edit @@ -30,6 +30,8 @@ git cherry-pick --continue --no-edit If `git status` is now clean and the cherry-pick continued, **Tier 0 fully resolved** — record the file(s) auto-resolved and move on. +> **Note:** `union` is recognized by [`03-known-conflicts.md`](./03-known-conflicts.md) parsing for forward compatibility but is **not implemented** in [`scripts/03-cherry-pick-one.ps1`](../scripts/03-cherry-pick-one.ps1) — a `union` entry currently falls through to Tier-3 with the path listed under `conflict_paths`. Author Tier-0 entries with `take-upstream` or `take-ours` only. + ## Tier 1 — Empty after staging After Tier 0 (or with no conflicts to begin with), if the staged diff is diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 803a2210c..41b912021 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -189,6 +189,7 @@ if ($unhandled.Count -gt 0) { if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after unhandled conflicts; repository may still be mid-cherry-pick." } $result.status = 'stuck' $result.conflict_paths = $unhandled + $result.error = "Cherry-pick of $Sha hit conflicts not covered by Tier-0 known-conflicts rules in $($unhandled.Count) path(s)." $result | ConvertTo-Json -Compress return } @@ -208,6 +209,11 @@ if ($LASTEXITCODE -ne 0) { git cherry-pick --abort | Out-Null if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after Tier-0 continuation failed; repository may still be mid-cherry-pick." } $result.status = 'stuck' + # Capture any unmerged files left after the failed --continue so the + # Tier-3 issue body lists *something* concrete. + $stillUnmerged = @(git diff --name-only --diff-filter=U 2>$null) + if ($stillUnmerged.Count -gt 0) { $result.conflict_paths = $stillUnmerged } + $result.error = "git cherry-pick --continue failed after Tier-0 auto-resolution for $Sha; the residual conflict is not mechanically resolvable." $result | ConvertTo-Json -Compress return } diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 84a640437..07451710c 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -4,9 +4,10 @@ captures the result. Default: `cmd /c "tools\razzle.cmd && bz no_clean"`. .DESCRIPTION - Run AFTER cherry-picking (03) and BEFORE finalizing the PR (05). If the - build fails, the agent commits a fix on the same sync branch so it lands - in the same PR — that is why try-build is step 04, not step 99. + Run AFTER cherry-picking (03) and BEFORE finalizing the PR (SKILL.md + step 8). If the build fails, the agent commits a fix on the same sync + branch so it lands in the same PR — that is why try-build is step 04, + not the last step. .PARAMETER BuildCommand Override the default build command. Must be a string the cmd.exe shell From e5e42f86c5243e5ddd2b19fdcb3cba997d65d6e6 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:23:52 +0800 Subject: [PATCH 60/82] Round 10: address Copilot post-restage findings SKILL.md step 3: - Don't fetch from a misconfigured upstream remote. The recipe used to say (in prose) "if upstream exists with a wrong URL, bail" but the code only checked for existence. Now compares git remote get-url upstream against the expected microsoft/terminal URL (trailing-slash tolerant) and throws if they differ, so we can't silently sync from a private mirror or stale fork. SKILL.md step 5a: - Issue-body recipe used a fixed .issuebody file in the repo root. If gh issue create failed (auth/rate-limit/network) that file would be left behind and break the next run's "working tree clean" precondition. Switched to New-TemporaryFile + try/finally so the temp file is guaranteed to be cleaned up regardless of outcome. references/follow-up-pr.md: - `"..\it-$syncPr ix"` contained an accidental backtick-f escape that PowerShell expands to a literal form-feed (\f), producing an invalid worktree path. Replaced with a normal hyphen suffix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 30 +++++++++++-------- .../upstream-sync/references/follow-up-pr.md | 2 +- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index df0c1fa86..d3b2f07b7 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -102,18 +102,18 @@ The fresh suffix means re-runs never collide with stale local branches. ### 3. Fetch upstream ```pwsh -if (-not (git remote get-url upstream 2>$null)) { - git remote add upstream https://github.com/microsoft/terminal.git +$expectedUpstream = 'https://github.com/microsoft/terminal.git' +$existing = git remote get-url upstream 2>$null +if (-not $existing) { + git remote add upstream $expectedUpstream +} elseif ($existing.Trim().TrimEnd('/') -ne $expectedUpstream.TrimEnd('/')) { + throw "Remote 'upstream' points at '$existing' but this skill requires '$expectedUpstream'. Aborting so we don't silently sync from the wrong fork." } git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } $upstreamSha = (git rev-parse upstream/main).Trim() ``` -If `upstream` already exists with a different URL (someone configured it -to a private mirror, etc.), surface that to the operator and exit — don't -silently rewrite it. - ### 4. Compute pending ```pwsh @@ -187,13 +187,17 @@ $(if ($pick.error) { "**Error:** ``$($pick.error)``" }) See [references/03-conflict-triage.md](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/03-conflict-triage.md) for the resolution rubric. "@ -$body | Set-Content -Encoding utf8 .issuebody -$issueUrl = gh issue create -R microsoft/intelligent-terminal ` - --title "Upstream sync stuck at $shortSha" ` - --label upstream-sync-stuck ` - --body-file .issuebody -if ($LASTEXITCODE -ne 0) { throw "gh issue create failed." } -Remove-Item .issuebody +$bodyFile = New-TemporaryFile +try { + Set-Content -Encoding utf8 -Path $bodyFile -Value $body + $issueUrl = gh issue create -R microsoft/intelligent-terminal ` + --title "Upstream sync stuck at $shortSha" ` + --label upstream-sync-stuck ` + --body-file $bodyFile + if ($LASTEXITCODE -ne 0) { throw "gh issue create failed." } +} finally { + Remove-Item -Force -ErrorAction SilentlyContinue $bodyFile +} ``` Surface `$issueUrl` and `$branch` to the operator. The human is expected to diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index 09d7e7185..ff3204ab2 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -50,7 +50,7 @@ clean: $syncPr = 220 $syncBranch = 'upstream-sync/2026-06-04-091512-a3f1' # the sync PR's head — copy the exact name from the PR, branches are date+UTC-HHmmss+random-hex per run $fixBranch = "dev/$env:USERNAME/sync-$syncPr-review-fixes" -$fixWorktree = "..\it-$syncPr`fix" +$fixWorktree = "..\it-$syncPr-fix" # Sync branch tip must be local for the worktree to land at the right base git fetch origin $syncBranch From 8e21b479eb196f6eaeee41f39228e065bf2bb61e Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:33:13 +0800 Subject: [PATCH 61/82] Round 11: scan all cherry-pick trailers per commit, newest-first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 02-compute-pending.ps1 / Get-LastSyncedUpstreamSha: - A single commit body can carry multiple `(cherry picked from commit )` trailers when several upstream picks are squash-merged into one fork commit. The previous implementation grabbed only the FIRST match via `-match`, which could select an older trailer even when a newer one existed in the same body — moving the watermark backward and causing the next run to re-pick already-synced commits. - Switched to `[regex]::Matches` + bottom-up iteration. Git appends new trailers, so the last match in the body is the most-recent upstream commit. We walk trailers newest-first within each commit, returning the first one whose target is reachable from upstream/main. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/02-compute-pending.ps1 | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 63954e40a..0e2218daf 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -44,13 +44,22 @@ function Get-LastSyncedUpstreamSha { # weekly-sync deployment. Resolves the trailer SHA to its full 40-char # form before the ancestry check so an abbreviated/ambiguous trailer # doesn't silently skip an otherwise-valid watermark candidate. + # + # A single commit body can carry multiple `(cherry picked from + # commit )` trailers (e.g. a squash-merge of several upstream + # picks). Git appends new trailers, so the LAST match in the body is + # the most-recent upstream commit — scan all matches and check them + # newest-first to avoid moving the watermark backward. $commits = @(git log origin/main --max-count=5000 --grep='cherry picked from commit' --format='%H' 2>$null) if ($LASTEXITCODE -ne 0) { throw "git log on origin/main failed while deriving last-synced SHA." } foreach ($c in $commits) { $body = git log -1 --format='%B' $c 2>$null - if ($body -match '\(cherry picked from commit ([0-9a-f]{7,40})\)') { + $allMatches = [regex]::Matches($body, '\(cherry picked from commit ([0-9a-f]{7,40})\)') + if ($allMatches.Count -eq 0) { continue } + # Walk trailers bottom-up (newest-first within the same commit). + for ($i = $allMatches.Count - 1; $i -ge 0; $i--) { $fullSha = $null - try { $fullSha = Resolve-FullCommitSha $matches[1] } catch { continue } + try { $fullSha = Resolve-FullCommitSha $allMatches[$i].Groups[1].Value } catch { continue } $null = git merge-base --is-ancestor $fullSha upstream/main 2>$null if ($LASTEXITCODE -eq 0) { return $fullSha } } From 4d884454300b6fde308f58a0fded81d6e23f0498 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:42:41 +0800 Subject: [PATCH 62/82] Round 12: derive branch date and time from one UTC timestamp SKILL.md step 2: - Branch name used `Get-Date` for the date and `ToUniversalTime()` for the time. Around midnight UTC the two halves disagree (e.g. a branch named with yesterday's local date but today's UTC HHmmss). - Take both halves from a single `(Get-Date).ToUniversalTime()` so the branch name is consistent with the date+UTC-HHmmss+random scheme documented elsewhere. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index d3b2f07b7..efad28a1b 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -90,8 +90,9 @@ gh issue list -R microsoft/intelligent-terminal --label upstream-sync-stuck --st ### 2. Build a branch name ```pwsh -$date = (Get-Date).ToString('yyyy-MM-dd') -$tstamp = (Get-Date).ToUniversalTime().ToString('HHmmss') +$utc = (Get-Date).ToUniversalTime() +$date = $utc.ToString('yyyy-MM-dd') +$tstamp = $utc.ToString('HHmmss') $rand = [guid]::NewGuid().ToString('N').Substring(0,4) $branch = "upstream-sync/$date-$tstamp-$rand" git switch -c $branch From 840bc8b2d70bcebf2f0a1bc8943f507bac8ea800 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 17:52:15 +0800 Subject: [PATCH 63/82] Round 13: exit-code checks + robust label/log/cmd handling SKILL.md: - Step 4 / 5 / 7: each pwsh-script invocation now checks `\0` before piping to `ConvertFrom-Json`, so a script that throws/terminates surfaces as a real error instead of a confusing JSON parse failure. - Step 5a: `gh label create` output is now joined into a single string via `Out-String` before the `already exists` match. PowerShell's `-notmatch` returns the non-matching elements of an array, so the previous form could throw spuriously when the gh stderr came back as multiple lines and only one contained the magic phrase. 04-try-build.ps1: - Log filename used second-level precision (`yyyy-MM-ddTHHmmss`); two invocations within the same second (e.g. quick rerun after a small fix) overwrote each other's log. Switched to UTC ms-precision plus a 4-char random suffix so paths are always unique. - `\C:\Windows\system32\cmd.exe` is not guaranteed in every host (some CI/scheduled task environments); fall back to `cmd.exe` when unset so `ProcessStartInfo.FileName` is never null. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 8 ++++++-- .github/skills/upstream-sync/scripts/04-try-build.ps1 | 8 +++++--- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index efad28a1b..a6c22d41c 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -119,6 +119,7 @@ $upstreamSha = (git rev-parse upstream/main).Trim() ```pwsh $pendingJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/02-compute-pending.ps1 +if ($LASTEXITCODE -ne 0) { throw "02-compute-pending.ps1 exited $LASTEXITCODE." } $pending = $pendingJson | ConvertFrom-Json # $pending.from = last-synced watermark SHA # $pending.to = upstream/main SHA @@ -136,6 +137,7 @@ For each SHA in `$pending.pending`: ```pwsh $pickJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 -Sha $sha +if ($LASTEXITCODE -ne 0) { throw "03-cherry-pick-one.ps1 exited $LASTEXITCODE for $sha." } $pick = $pickJson | ConvertFrom-Json ``` @@ -162,8 +164,9 @@ $labelOut = gh label create 'upstream-sync-stuck' ` --color B60205 ` --description 'Upstream sync paused — close to clear the lock' ` -R microsoft/intelligent-terminal 2>&1 -if ($LASTEXITCODE -ne 0 -and ($labelOut -notmatch 'already exists')) { - throw "gh label create failed: $labelOut" +$labelOutText = ($labelOut | Out-String) +if ($LASTEXITCODE -ne 0 -and $labelOutText -notmatch 'already exists') { + throw "gh label create failed: $labelOutText" } $author = git log -1 --format='%an <%ae>' $sha @@ -219,6 +222,7 @@ a no-op) there is nothing to build or finalize. Delete the branch, report ```pwsh $buildJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/04-try-build.ps1 +if ($LASTEXITCODE -ne 0) { throw "04-try-build.ps1 exited $LASTEXITCODE before producing a JSON result." } $build = $buildJson | ConvertFrom-Json ``` diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 07451710c..11f32a7f4 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -90,14 +90,16 @@ try { $root = Get-RepoRoot if (-not (Test-Path -LiteralPath $LogDir)) { New-Item -ItemType Directory -Path $LogDir -Force | Out-Null } - $stamp = (Get-Date).ToString('yyyy-MM-ddTHHmmss') - $logPath = Join-Path $LogDir "$stamp.log" + $stamp = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHHmmss.fff') + $suffix = [guid]::NewGuid().ToString('N').Substring(0,4) + $logPath = Join-Path $LogDir "$stamp-$suffix.log" $cmdLine = "/c `"cd /d `"$root`" && $BuildCommand`"" $started = Get-Date $psi = New-Object System.Diagnostics.ProcessStartInfo - $psi.FileName = $env:ComSpec + $shell = if ($env:ComSpec) { $env:ComSpec } else { 'cmd.exe' } + $psi.FileName = $shell $psi.Arguments = $cmdLine $psi.WorkingDirectory = $root $psi.RedirectStandardOutput = $true From 823f2d4f4970fba175711eacb2eacbb4dba61469 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:02:04 +0800 Subject: [PATCH 64/82] Round 14: fail fast on git errors; drop hardcoded path 02-compute-pending.ps1: - The subject/body fetch loop and the upstream-empty detection loop silently ignored `git log` / `git diff-tree` failures, so a missing object or incomplete fetch could produce empty subject/body/files strings and feed them into revert-pair detection or upstream-empty classification. Both loops now capture stderr via `2>&1` and throw on non-zero exit so the script fails fast. references/follow-up-pr.md: - Worktree-setup section hardcoded `Q:\official\intelligent-terminal`, which is the maintainer's specific dev path. Reworded to describe the requirement (keep the primary worktree on `main` and open a sibling) without prescribing any specific drive. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/references/follow-up-pr.md | 5 ++--- .../skills/upstream-sync/scripts/02-compute-pending.ps1 | 9 ++++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/references/follow-up-pr.md b/.github/skills/upstream-sync/references/follow-up-pr.md index ff3204ab2..707b037b5 100644 --- a/.github/skills/upstream-sync/references/follow-up-pr.md +++ b/.github/skills/upstream-sync/references/follow-up-pr.md @@ -42,9 +42,8 @@ follow-up PR. ### Worktree setup -Keep the primary worktree at `Q:\official\intelligent-terminal` on `main` -and open a sibling worktree for the follow-up so the main worktree stays -clean: +Keep the primary worktree on `main` and open a sibling worktree for the +follow-up so the primary stays clean: ```pwsh $syncPr = 220 diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 0e2218daf..09ab6b816 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -104,8 +104,10 @@ $all = @(Get-PendingUpstreamShas -Since $from) # fine at typical batch sizes). $info = @{} foreach ($sha in $all) { - $subj = git log -1 --format='%s' $sha - $body = git log -1 --format='%B' $sha + $subj = git log -1 --format='%s' $sha 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git log --format=%s failed for $sha : $subj" } + $body = git log -1 --format='%B' $sha 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git log --format=%B failed for $sha : $body" } $info[$sha] = @{ subject = $subj; body = $body } } @@ -150,7 +152,8 @@ foreach ($sha in $all) { $empty = @() foreach ($sha in $all) { if ($dropped.Contains($sha)) { continue } - $files = git diff-tree --no-commit-id --name-only -r $sha + $files = git diff-tree --no-commit-id --name-only -r $sha 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git diff-tree failed for $sha : $files" } if (-not $files) { $empty += $sha [void] $dropped.Add($sha) From 28ba860f73e954684f4d4c458079d2688da53e83 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:11:47 +0800 Subject: [PATCH 65/82] Round 15: flatten multi-line %B; UTC-date artifacts; permissive upstream URL MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 02-compute-pending.ps1 (CRITICAL): - `git log -1 --format='%B' ` returns `string[]` in PowerShell (one element per body line), not a single string. Two places relied on it being a string: 1. `Get-LastSyncedUpstreamSha` passed it to `[regex]::Matches`, which coerces array operands to the literal `\"System.String[]\"` and silently lost every trailer — preventing watermark resolution on any commit with a multi-line body. 2. The revert-detection loop later did `\ -match '...'`, which on an array returns the filtered subset of matching elements and does NOT populate `\`, so revert pairs went undetected. - Both call sites now flatten with `-join \"\ \"` immediately after capture so downstream regex/match operates on a real string. 04-try-build.ps1: - `Get-GeneratedDir` used local time for the per-day folder while the log filename used UTC. Around local midnight this split a single run's artifacts across two date folders and disagreed with the UTC-based sync-branch names from SKILL.md. Switched to UTC throughout. SKILL.md step 3: - Upstream URL check was an exact-string compare on `https://github.com/microsoft/terminal.git`, which rejected the `.git`-less HTTPS form and the SSH `git@github.com:...` form even though both point at the same repo. Switched to an owner/repo-identity extractor (regex over both URL shapes); only the `microsoft/terminal` identity matters, not the URL shape. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 16 ++++++++++++---- .../upstream-sync/scripts/02-compute-pending.ps1 | 13 ++++++++++--- .../upstream-sync/scripts/04-try-build.ps1 | 2 +- 3 files changed, 23 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index a6c22d41c..c15ead16a 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -103,12 +103,20 @@ The fresh suffix means re-runs never collide with stale local branches. ### 3. Fetch upstream ```pwsh -$expectedUpstream = 'https://github.com/microsoft/terminal.git' +# Accept any equivalent form pointing at microsoft/terminal — HTTPS with +# or without .git, SSH form, etc. We only care about owner/repo identity. +$expectedOwnerRepo = 'microsoft/terminal' $existing = git remote get-url upstream 2>$null +function Get-OwnerRepo([string] $url) { + if (-not $url) { return $null } + $u = $url.Trim().TrimEnd('/') + if ($u -match '^(?:https?://[^/]+/|git@[^:]+:)([^/]+/[^/]+?)(?:\.git)?$') { return $Matches[1].ToLowerInvariant() } + return $null +} if (-not $existing) { - git remote add upstream $expectedUpstream -} elseif ($existing.Trim().TrimEnd('/') -ne $expectedUpstream.TrimEnd('/')) { - throw "Remote 'upstream' points at '$existing' but this skill requires '$expectedUpstream'. Aborting so we don't silently sync from the wrong fork." + git remote add upstream "https://github.com/$expectedOwnerRepo.git" +} elseif ((Get-OwnerRepo $existing) -ne $expectedOwnerRepo) { + throw "Remote 'upstream' points at '$existing' but this skill requires $expectedOwnerRepo. Aborting so we don't silently sync from the wrong fork." } git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 09ab6b816..b4d5177ea 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -53,7 +53,10 @@ function Get-LastSyncedUpstreamSha { $commits = @(git log origin/main --max-count=5000 --grep='cherry picked from commit' --format='%H' 2>$null) if ($LASTEXITCODE -ne 0) { throw "git log on origin/main failed while deriving last-synced SHA." } foreach ($c in $commits) { - $body = git log -1 --format='%B' $c 2>$null + # %B comes back as string[] (one element per line) — join to a + # single string so [regex]::Matches doesn't coerce the array to + # the literal "System.String[]". + $body = (git log -1 --format='%B' $c 2>$null) -join "`n" $allMatches = [regex]::Matches($body, '\(cherry picked from commit ([0-9a-f]{7,40})\)') if ($allMatches.Count -eq 0) { continue } # Walk trailers bottom-up (newest-first within the same commit). @@ -106,8 +109,12 @@ $info = @{} foreach ($sha in $all) { $subj = git log -1 --format='%s' $sha 2>&1 if ($LASTEXITCODE -ne 0) { throw "git log --format=%s failed for $sha : $subj" } - $body = git log -1 --format='%B' $sha 2>&1 - if ($LASTEXITCODE -ne 0) { throw "git log --format=%B failed for $sha : $body" } + # %B is multi-line — flatten to a single string so `-match` later + # operates on the body rather than an array of lines (where -match + # filters elements instead of returning a single bool + $Matches). + $bodyArr = git log -1 --format='%B' $sha 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git log --format=%B failed for $sha : $bodyArr" } + $body = $bodyArr -join "`n" $info[$sha] = @{ subject = $subj; body = $body } } diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 11f32a7f4..76a16d05d 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -57,7 +57,7 @@ function Get-GeneratedDir { # top-level .gitignore has `**/Generated Files/`). param([string] $Sub) $root = Get-RepoRoot - $date = (Get-Date).ToString('yyyy-MM-dd') + $date = (Get-Date).ToUniversalTime().ToString('yyyy-MM-dd') $path = Join-Path $root "Generated Files/upstream-sync/$date" if ($Sub) { $path = Join-Path $path $Sub } if (-not (Test-Path -LiteralPath $path)) { From 34020411b4ba9d8abf28bccc838dcc5cc7ddcc02 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:21:59 +0800 Subject: [PATCH 66/82] Round 16: fail-fast on repo-relative log path; spell out cherry-pick resume MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 04-try-build.ps1: - `log_path` is documented as a repo-relative string and ships in PR/operator-visible output. The fallback `try { … } catch { \ }` silently leaked the absolute path (e.g. `Q:\official\…`) when `ConvertTo-RepoRelativePath` failed. Removed the fallback so a log directory outside the repo aborts the script instead of breaking the output contract. SKILL.md step 5a + references/03-conflict-triage.md Tier-3: - Resume instructions used to say "check out the branch, resolve the conflict". But `03-cherry-pick-one.ps1` runs `git cherry-pick --abort` before returning `stuck`, so a fresh checkout has nothing to resolve. Updated both surfaces to spell out the missing step: `git cherry-pick -x ` first, to reproduce the conflict with the `-x` trailer preserved, *then* resolve / `--continue` / push / merge without squashing / close issue. - The issue body itself now embeds the resume recipe so the human sees it without having to follow a link. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 34 ++++++++++++++++--- .../references/03-conflict-triage.md | 12 +++++-- .../upstream-sync/scripts/04-try-build.ps1 | 2 +- 3 files changed, 40 insertions(+), 8 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index c15ead16a..1df712e32 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -196,6 +196,21 @@ $pathLines $(if ($pick.error) { "**Error:** ``$($pick.error)``" }) +**To resume** (the script ran ``git cherry-pick --abort`` — the branch +has no in-progress conflict state to resolve directly): + +``````pwsh +git fetch origin +git switch $branch +git cherry-pick -x $sha # re-run to reproduce the conflict; -x keeps the trailer +# ... resolve conflicts ... +git add -A +git cherry-pick --continue +git push +# Open a PR against main, merge without squashing (the trailer must survive), +# then close this issue to clear the lock. +`````` + See [references/03-conflict-triage.md](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/03-conflict-triage.md) for the resolution rubric. "@ @@ -212,10 +227,21 @@ try { } ``` -Surface `$issueUrl` and `$branch` to the operator. The human is expected to -check out the branch, resolve the conflict, push it, open a PR, merge it -(keeping the `(cherry picked from commit )` trailer!), then close -the issue. EXIT — do not attempt the build. +Surface `$issueUrl` and `$branch` to the operator. The human is expected to: + +1. `git fetch origin && git switch $branch` to pick up the sync branch. +2. **Re-run the cherry-pick to reproduce the conflict** — `03-cherry-pick-one.ps1` + calls `git cherry-pick --abort` before returning `stuck`, so the branch + itself has no conflict markers / `MERGE_MSG`. Use the stuck SHA from + the issue body: `git cherry-pick -x ` (the `-x` preserves + the `(cherry picked from commit )` trailer — critical for the + watermark). +3. Resolve the conflicts, `git add` the resolutions, `git cherry-pick --continue`. +4. Push, open a PR against `main`, merge it (the `-x` trailer must + survive the merge — DO NOT squash). +5. Close the stuck issue. + +EXIT — do not attempt the build. See [references/03-conflict-triage.md](./references/03-conflict-triage.md) for what "Tier-3" means and the resolution rubric. diff --git a/.github/skills/upstream-sync/references/03-conflict-triage.md b/.github/skills/upstream-sync/references/03-conflict-triage.md index 1866dbd4b..d1a851612 100644 --- a/.github/skills/upstream-sync/references/03-conflict-triage.md +++ b/.github/skills/upstream-sync/references/03-conflict-triage.md @@ -104,9 +104,15 @@ The issue body **must** include: - The conflicting commit SHA, subject, author, and upstream URL. - The list of conflicting paths. - The exact local branch name where the human picks up. -- The exact resume action: resolve on the stuck branch, merge a PR - that keeps the `(cherry picked from commit )` trailer, then - CLOSE the stuck issue (that's the lock-clear signal — no script). +- The exact resume action: (1) check out the stuck branch, + (2) **re-run the cherry-pick to reproduce the conflict** — the script + calls `git cherry-pick --abort` before returning, so a fresh checkout + has no `MERGE_MSG` / conflict markers to resolve. Use + `git cherry-pick -x ` (the `-x` preserves the + `(cherry picked from commit )` trailer — critical for the + watermark). (3) Resolve, `git add`, `git cherry-pick --continue`. + (4) Push, open a PR, merge keeping the trailer (no squash). + (5) CLOSE the stuck issue (that's the lock-clear signal — no script). No fenced YAML metadata block is needed. Closing the labeled issue IS the lock-clear signal; nothing parses the body back. diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 76a16d05d..cd97debb0 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -153,7 +153,7 @@ try { @(Get-Content -LiteralPath $logPath -Tail 200) -join "`n" } else { '' } - $logPathForReport = try { ConvertTo-RepoRelativePath $logPath } catch { $logPath } + $logPathForReport = ConvertTo-RepoRelativePath $logPath # fails fast if outside repo — log_path is a public contract field [ordered] @{ kind = $kind From 6033ca580fc5156ef9fb77e7c7ad9993e1663afb Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:31:29 +0800 Subject: [PATCH 67/82] Round 17: normalize CR/whitespace; resolve -LogDir relative paths 02-compute-pending.ps1: - `git log --format='%H'` lines could carry trailing CR or whitespace on some PS hosts/terminals; the SHA regex would then drop them and silently truncate the pending list. Each line is now `.Trim()`'d before the `^[0-9a-f]{40}\$` check. 03-cherry-pick-one.ps1: - `Get-ConflictPaths` split `git diff --name-only` on LF but didn't strip a trailing CR. A CRLF-normalized stream would yield paths ending in `\r`, missing the exact-string lookup in 03-known-conflicts.md and unnecessarily escalating Tier-0-eligible commits to Tier-3. Added `.TrimEnd(\"\ \")` per line. 04-try-build.ps1: - `-LogDir` overrides now get rooted to the repo when supplied as a relative path. Previously the path stayed relative through Join-Path, then `ConvertTo-RepoRelativePath` threw because the input wasn't rooted under the repo (since round 16 removed the silent absolute-path fallback). Absolute paths outside the repo still abort, as intended. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../skills/upstream-sync/scripts/02-compute-pending.ps1 | 2 +- .../skills/upstream-sync/scripts/03-cherry-pick-one.ps1 | 2 +- .github/skills/upstream-sync/scripts/04-try-build.ps1 | 8 +++++++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index b4d5177ea..7ddb735d7 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -78,7 +78,7 @@ function Get-PendingUpstreamShas { param([string] $Since) $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } - $shas = @($out | Where-Object { $_ -match '^[0-9a-f]{40}$' }) + $shas = @($out | ForEach-Object { $_.Trim() } | Where-Object { $_ -match '^[0-9a-f]{40}$' }) if ($Since) { $filtered = New-Object 'System.Collections.Generic.List[string]' foreach ($sha in $shas) { diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 41b912021..23dc46737 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -51,7 +51,7 @@ function Get-ConflictPaths { # path matching against 03-known-conflicts.md works without C-quoting. $u = git -c core.quotepath=off diff --name-only --diff-filter=U if (-not $u) { return @() } - return @($u -split "`n" | Where-Object { $_ }) + return @($u -split "`n" | ForEach-Object { $_.TrimEnd("`r") } | Where-Object { $_ }) } $result = [ordered] @{ diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index cd97debb0..f2b537e17 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -86,8 +86,14 @@ function ConvertTo-RepoRelativePath { # --- Main logic ------------------------------------------------------------ try { - if (-not $LogDir) { $LogDir = Get-GeneratedDir -Sub 'build-logs' } $root = Get-RepoRoot + if (-not $LogDir) { + $LogDir = Get-GeneratedDir -Sub 'build-logs' + } elseif (-not [System.IO.Path]::IsPathRooted($LogDir)) { + # Treat caller-supplied relative paths as repo-relative so the + # later ConvertTo-RepoRelativePath call succeeds. + $LogDir = Join-Path $root $LogDir + } if (-not (Test-Path -LiteralPath $LogDir)) { New-Item -ItemType Directory -Path $LogDir -Force | Out-Null } $stamp = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHHmmss.fff') From b14aa7bec576e401e1c2383eb49b2e600150a15d Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:40:42 +0800 Subject: [PATCH 68/82] Round 18: full-history prereq; full gh-pr-create error context MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SKILL.md prerequisites: - Added explicit prereq that `origin/main` must have full git history. Watermark discovery scans up to 5000 commits on origin/main for cherry-pick trailers and the pending walk uses `merge-base --is-ancestor`. A shallow clone (e.g. GitHub Actions default `fetch-depth=1`) will produce empty/wrong results. CI should use `actions/checkout@v4` with `fetch-depth: 0` or `git fetch --unshallow`. SKILL.md step 8 (gh pr create retry): - The retry loop used `Select-Object -Last 1` to capture gh's output, which drops every line before the last — usually losing the actual error context (auth failure, missing head ref, etc.). Now captures full output via `Out-String`, surfaces it in the `Write-Warning`, and extracts the URL line by filtering for `^https://github.com/` so the success path still works. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 1df712e32..b885bf880 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -39,6 +39,13 @@ the operator can audit in your transcript. - PowerShell 7+ (`pwsh`) on PATH. - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment (build is a hard gate before finalize — see [step 7](#7-build)). - Remote named `upstream` — the scripts create it if missing. +- **Full git history on `origin/main`** (no shallow clone). Watermark + discovery scans up to 5000 commits on `origin/main` for `cherry picked + from commit ` trailers and the pending walk does `merge-base + --is-ancestor` checks. A shallow clone (e.g. GitHub Actions default + `fetch-depth=1`) will produce wrong/empty results. If running in CI, + use `actions/checkout@v4` with `fetch-depth: 0` (or run `git fetch + --unshallow` before invoking the skill). - **No `state.json` to bootstrap.** Watermark comes from the `(cherry picked from commit )` trailers on `origin/main`. If the fork has never used `cherry-pick -x` (or trailers were stripped), @@ -366,15 +373,19 @@ $title = "chore(upstream): sync microsoft/terminal up to $short" $prUrl = $null for ($attempt = 1; $attempt -le 3; $attempt++) { - $prUrl = gh pr create -R microsoft/intelligent-terminal ` - --base main --head $branch --title $title --body-file $bodyFile 2>&1 | - Select-Object -Last 1 - if ($LASTEXITCODE -eq 0 -and $prUrl -match '^https://github.com/') { break } - Write-Warning "gh pr create attempt $attempt failed: $prUrl" + $prOut = gh pr create -R microsoft/intelligent-terminal ` + --base main --head $branch --title $title --body-file $bodyFile 2>&1 + $prExit = $LASTEXITCODE + $prText = ($prOut | Out-String).Trim() + if ($prExit -eq 0) { + $urlLine = ($prOut | Where-Object { "$_" -match '^https://github.com/' } | Select-Object -Last 1) + if ($urlLine) { $prUrl = "$urlLine".Trim(); break } + } + Write-Warning "gh pr create attempt $attempt failed (exit $prExit). Full output:`n$prText" Start-Sleep -Seconds 5 } Remove-Item -LiteralPath $bodyFile -Force -if ($prUrl -notmatch '^https://github.com/') { throw "gh pr create did not return a URL after 3 attempts." } +if (-not $prUrl) { throw "gh pr create did not return a URL after 3 attempts." } # Optional: arm GitHub auto-merge with rebase (NEVER squash). # Skip this if the operator wants to merge by hand. From 12c8c19465d5dedb1ffde3c36ee29a5a59a96836 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:49:38 +0800 Subject: [PATCH 69/82] Round 19: drop fragile cmd /c nested quoting in try-build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 04-try-build.ps1: - `\` used nested `cmd /c \"cd /d \"\\" && …\"` to ensure the build ran in the repo root, but `ProcessStartInfo.WorkingDirectory` is already set to `\` two lines below — the `cd /d` prefix was redundant *and* the double-nested quoting breaks when the repo path contains spaces or quotes (cmd.exe quote parsing is famously fragile). Dropped to a plain `/c \`; the working directory is enforced by the ProcessStartInfo, not by an in-line `cd`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/04-try-build.ps1 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index f2b537e17..3199a33e1 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -100,7 +100,10 @@ try { $suffix = [guid]::NewGuid().ToString('N').Substring(0,4) $logPath = Join-Path $LogDir "$stamp-$suffix.log" - $cmdLine = "/c `"cd /d `"$root`" && $BuildCommand`"" + # WorkingDirectory is already $root via ProcessStartInfo below, so + # we don't need a `cd /d "" &&` prefix — that nested quoting + # is brittle under cmd.exe (paths with spaces/quotes break it). + $cmdLine = "/c $BuildCommand" $started = Get-Date $psi = New-Object System.Diagnostics.ProcessStartInfo From 71b8e085a776929e94df87b96c4655801227519a Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 18:58:41 +0800 Subject: [PATCH 70/82] Round 20: hex case, merge-base error class, diff exit check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 02-compute-pending.ps1: - Trailer regex and SHA validator now accept uppercase A–F as well as lowercase a–f. Hand-edited or seed-commit trailers using uppercase hex no longer slip past watermark discovery. - `git merge-base --is-ancestor` previously treated *any* non-zero exit as `not an ancestor`. That conflated the documented exit 1 (semantic `not an ancestor`, keep the SHA) with exit 128 (bad object / missing ref / shallow-clone error). Now switches on the exit code: 0 = ancestor (drop), 1 = keep, anything else throws so a broken repo state can't silently produce a wrong pending list. 03-cherry-pick-one.ps1: - `Get-ConflictPaths` now captures stderr via `2>&1` and throws on non-zero `git diff --diff-filter=U` exit, so a real repo error can't be misreported as `no conflict paths`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/02-compute-pending.ps1 | 14 +++++++++++--- .../upstream-sync/scripts/03-cherry-pick-one.ps1 | 3 ++- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 7ddb735d7..8313e9746 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -57,7 +57,7 @@ function Get-LastSyncedUpstreamSha { # single string so [regex]::Matches doesn't coerce the array to # the literal "System.String[]". $body = (git log -1 --format='%B' $c 2>$null) -join "`n" - $allMatches = [regex]::Matches($body, '\(cherry picked from commit ([0-9a-f]{7,40})\)') + $allMatches = [regex]::Matches($body, '\(cherry picked from commit ([0-9a-fA-F]{7,40})\)') if ($allMatches.Count -eq 0) { continue } # Walk trailers bottom-up (newest-first within the same commit). for ($i = $allMatches.Count - 1; $i -ge 0; $i--) { @@ -78,12 +78,20 @@ function Get-PendingUpstreamShas { param([string] $Since) $out = @(git log --cherry-pick --right-only --no-merges --format='%H' --reverse 'origin/main...upstream/main' 2>$null) if ($LASTEXITCODE -ne 0) { throw "git log --cherry-pick failed while computing pending list." } - $shas = @($out | ForEach-Object { $_.Trim() } | Where-Object { $_ -match '^[0-9a-f]{40}$' }) + $shas = @($out | ForEach-Object { $_.Trim() } | Where-Object { $_ -match '^[0-9a-fA-F]{40}$' }) if ($Since) { $filtered = New-Object 'System.Collections.Generic.List[string]' foreach ($sha in $shas) { + # git merge-base --is-ancestor: + # exit 0 = ancestor (already on origin/main, drop it) + # exit 1 = not an ancestor (keep) + # exit >1 = real error (bad object, missing ref, shallow clone) $null = git merge-base --is-ancestor $sha $Since 2>$null - if ($LASTEXITCODE -ne 0) { [void] $filtered.Add($sha) } + switch ($LASTEXITCODE) { + 0 { } # ancestor, drop + 1 { [void] $filtered.Add($sha) } + default { throw "git merge-base --is-ancestor failed (exit $LASTEXITCODE) on $sha vs $Since; pending list cannot be trusted." } + } } $shas = @($filtered) } diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 23dc46737..348ca4a04 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -49,7 +49,8 @@ function Get-KnownConflicts { function Get-ConflictPaths { # core.quotepath=off keeps non-ASCII paths in raw UTF-8 so Tier-0 # path matching against 03-known-conflicts.md works without C-quoting. - $u = git -c core.quotepath=off diff --name-only --diff-filter=U + $u = git -c core.quotepath=off diff --name-only --diff-filter=U 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git diff --name-only --diff-filter=U failed: $u" } if (-not $u) { return @() } return @($u -split "`n" | ForEach-Object { $_.TrimEnd("`r") } | Where-Object { $_ }) } From a967a3563713d574fa69b3394cb699ee76e1e823 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 19:08:42 +0800 Subject: [PATCH 71/82] Round 21: capture cherry-pick output; fix array unwrap in Get-PendingUpstreamShas MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 03-cherry-pick-one.ps1: - `git cherry-pick` output was streamed straight to stderr, so when the exit was non-zero AND there were no unmerged paths (e.g. `cherry- pick` rejected a merge commit without `-m`, or a git version rejected `--keep-redundant-commits`), the stuck result said only `exited N with no conflict paths` and the actionable git message was nowhere in the JSON / issue body. - Now captures the stream into `\` first, forwards it to stderr as before for live visibility, AND grabs the last 20 lines as `\`. The Write-Warning and the `result.error` field both include that tail so the operator can diagnose the failure without rerunning to reproduce. 02-compute-pending.ps1 (real bug — Copilot was right): - `Get-PendingUpstreamShas` ended with `return ,\`. The comma wraps the string[] as a single pipeline object, so the caller's `\ = @(Get-PendingUpstreamShas …)` ended up with one element that was itself the SHA[], and the subsequent `foreach (\ in \)` iterated once with `\` set to a System.Object[]. Verified the bug interactively (`count=1, type[0]=Object[]`) and the fix (`return \`, `count=N, type[0]=String`). PowerShell's `@(...)` already flattens the stream — the comma was actively harmful here. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/02-compute-pending.ps1 | 7 ++++++- .../upstream-sync/scripts/03-cherry-pick-one.ps1 | 15 ++++++++++----- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 8313e9746..57288a0a3 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -95,7 +95,12 @@ function Get-PendingUpstreamShas { } $shas = @($filtered) } - return ,$shas + # PowerShell streams arrays. Use `,$shas` would wrap into a single + # pipeline object; the caller's `@(...)` then receives a 1-element + # array containing the nested array, breaking the subsequent + # foreach. Just emit the array — `@(Get-PendingUpstreamShas …)` + # rebuilds a flat string[] from the stream. + return $shas } # --- Main logic ------------------------------------------------------------ diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 348ca4a04..10f297ae7 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -109,9 +109,14 @@ $env:GIT_COMMITTER_DATE = $info[5] try { -# Attempt the pick. -git cherry-pick --keep-redundant-commits -x $fullSha 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -$pickCode = $LASTEXITCODE +# Attempt the pick. Capture the stream so we can both forward it to our +# own stderr (for live operator visibility) AND surface the trailing +# context in the stuck result for non-conflict failures (e.g. merge +# commit without -m, hook failures, unsupported git version). +$pickOutput = git cherry-pick --keep-redundant-commits -x $fullSha 2>&1 +$pickCode = $LASTEXITCODE +$pickOutput | ForEach-Object { [Console]::Error.WriteLine($_) } +$pickTail = (@($pickOutput) | Select-Object -Last 20 | Out-String).Trim() if ($pickCode -eq 0) { # Tier-1 check: did we just create an empty commit (allowed by --keep-redundant-commits)? @@ -140,12 +145,12 @@ if ($pickCode -eq 0) { # mid-cherry-pick. Abort and surface a controlled stuck result. $conflicts = Get-ConflictPaths if (-not $conflicts -or $conflicts.Count -eq 0) { - Write-Warning "git cherry-pick exited $pickCode with no conflict paths — likely a non-conflict failure (e.g. merge commit without -m). Aborting and marking stuck." + Write-Warning "git cherry-pick exited $pickCode with no conflict paths — likely a non-conflict failure (e.g. merge commit without -m). Aborting and marking stuck.`nLast git output:`n$pickTail" git cherry-pick --abort 2>&1 | Out-Null if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after a non-conflict failure; repository may still be mid-cherry-pick." } $result.status = 'stuck' $result.conflict_paths = @() - $result.error = "git cherry-pick exited $pickCode with no conflict paths" + $result.error = "git cherry-pick exited $pickCode with no conflict paths. Last output: $pickTail" $result | ConvertTo-Json -Compress return } From 1f13d1a3a478efd707c480e2516c76f463f58df5 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 19:18:16 +0800 Subject: [PATCH 72/82] Round 22: tolerate cherry-pick --abort failure; stuck-issue wording reflects failure class MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 03-cherry-pick-one.ps1: - When git cherry-pick fails *before* starting (bad args, unsupported option on older git, hook reject), the in-progress state is empty and the subsequent `git cherry-pick --abort` itself fails with `no cherry-pick or revert in progress`. Previously we threw on that, which discarded the carefully-built stuck JSON and aborted the orchestration with a misleading error. - Now capture --abort's exit and output, classify the "nothing to abort" case as benign, and fold any abort diagnostic into `result.error` so the operator still gets the JSON-stuck payload with full context. SKILL.md step 5a (stuck-issue body): - Body said "stopped at a conflict" unconditionally, which misled operators when 03 stuck on a non-conflict failure (empty conflict list, error like "merge commit without -m"). Body now branches on whether conflict_paths is non-empty: - conflict path > 0 → "stopped at a conflict that needs human judgment" - empty → "non-conflict cherry-pick failure (e.g. merge commit without `-m`, unsupported git option, hook failure)" and the "Conflicting paths" section renders `_(none …)_` instead of an empty list when there are none. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 12 ++++++++-- .../scripts/03-cherry-pick-one.ps1 | 22 +++++++++++++++---- 2 files changed, 28 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index b885bf880..2ac1e81e5 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -187,10 +187,18 @@ if ($LASTEXITCODE -ne 0 -and $labelOutText -notmatch 'already exists') { $author = git log -1 --format='%an <%ae>' $sha $subject = git log -1 --format='%s' $sha $shortSha = $sha.Substring(0,9) -$pathLines = ($pick.conflict_paths | ForEach-Object { "- ``$_``" }) -join "`n" +$hasConflicts = $pick.conflict_paths -and $pick.conflict_paths.Count -gt 0 +$pathLines = if ($hasConflicts) { + ($pick.conflict_paths | ForEach-Object { "- ``$_``" }) -join "`n" +} else { '_(none — cherry-pick failed without producing unmerged paths)_' } +$headline = if ($hasConflicts) { + 'Upstream sync stopped at a conflict that needs human judgment.' +} else { + 'Upstream sync stopped on a non-conflict cherry-pick failure (e.g. merge commit without `-m`, unsupported git option, hook failure).' +} $body = @" > [!CAUTION] -> Upstream sync stopped at a conflict that needs human judgment. +> $headline > **Close this issue when resolved — that IS the lock-clear signal.** **Stuck on:** ``$sha`` — $subject diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 10f297ae7..78aad928f 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -145,12 +145,26 @@ if ($pickCode -eq 0) { # mid-cherry-pick. Abort and surface a controlled stuck result. $conflicts = Get-ConflictPaths if (-not $conflicts -or $conflicts.Count -eq 0) { - Write-Warning "git cherry-pick exited $pickCode with no conflict paths — likely a non-conflict failure (e.g. merge commit without -m). Aborting and marking stuck.`nLast git output:`n$pickTail" - git cherry-pick --abort 2>&1 | Out-Null - if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --abort failed after a non-conflict failure; repository may still be mid-cherry-pick." } + Write-Warning "git cherry-pick exited $pickCode with no conflict paths — likely a non-conflict failure (e.g. merge commit without -m, unsupported git option). Aborting and marking stuck.`nLast git output:`n$pickTail" + # If cherry-pick failed *before* starting (bad args / unsupported + # option / old git), --abort itself fails with "no cherry-pick or + # revert in progress". That's benign here — there is nothing to + # clean up. Capture and surface the abort outcome in the stuck + # result instead of throwing, so the operator still gets the JSON. + $abortOut = git cherry-pick --abort 2>&1 + $abortCode = $LASTEXITCODE + $abortNote = '' + if ($abortCode -ne 0) { + $abortText = ($abortOut | Out-String).Trim() + if ($abortText -match 'no cherry-pick or revert in progress') { + $abortNote = ' (no in-progress cherry-pick to abort — failed before starting)' + } else { + $abortNote = " (cherry-pick --abort also failed: exit $abortCode; $abortText)" + } + } $result.status = 'stuck' $result.conflict_paths = @() - $result.error = "git cherry-pick exited $pickCode with no conflict paths. Last output: $pickTail" + $result.error = "git cherry-pick exited $pickCode with no conflict paths$abortNote. Last output: $pickTail" $result | ConvertTo-Json -Compress return } From cd6ac7a32674717ff995aadc41cc7c210ede8a1c Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 19:27:47 +0800 Subject: [PATCH 73/82] Round 23: SSH-URL parsing covers ssh://; exit-check git diff-tree + git log MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SKILL.md Get-OwnerRepo: - Earlier rewrite (round 15) split out HTTPS + scp-style only. A remote like `ssh://git@github.com/microsoft/terminal.git` (the fully-qualified SSH-protocol form, including `ssh://…:22/…`) matched neither branch and was treated as "wrong fork" — the identity guard then aborted the sync. - Now three explicit branches: https, ssh://[user@]host[:port]/, and scp-style `user@host:owner/repo`. Verified against eight URL variants (with/without `.git`, with/without `user@`, with trailing slash, with port) — all resolve to `microsoft/terminal`. 03-cherry-pick-one.ps1: - Tier-1 empty-commit detection ran `git diff-tree` and `git log` without checking `\0`. If either failed (corrupt ref, plumbing change, hook issue), an empty stdout would `-not \` true → silently reset --hard \; non-empty stdout from a failure would proceed as if the pick had real changes. - Now captures stderr, fails fast on non-zero exit with the actual git diagnostic, and only then evaluates the change list / cherry-pick footer match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 9 +++++++-- .../skills/upstream-sync/scripts/03-cherry-pick-one.ps1 | 8 ++++++-- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 2ac1e81e5..05e0e80d7 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -116,8 +116,13 @@ $expectedOwnerRepo = 'microsoft/terminal' $existing = git remote get-url upstream 2>$null function Get-OwnerRepo([string] $url) { if (-not $url) { return $null } - $u = $url.Trim().TrimEnd('/') - if ($u -match '^(?:https?://[^/]+/|git@[^:]+:)([^/]+/[^/]+?)(?:\.git)?$') { return $Matches[1].ToLowerInvariant() } + $u = $url.Trim() + # https://host/owner/repo[.git] + if ($u -match '^https?://[^/]+/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } + # ssh://[user@]host[:port]/owner/repo[.git] + if ($u -match '^ssh://(?:[^@/]+@)?[^/:]+(?::\d+)?/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } + # scp-style: [user@]host:owner/repo[.git] + if ($u -match '^(?:[^@:/]+@)?[^:/]+:([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } return $null } if (-not $existing) { diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 78aad928f..cb8085a55 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -120,9 +120,13 @@ $pickTail = (@($pickOutput) | Select-Object -Last 20 | Out-String).Trim() if ($pickCode -eq 0) { # Tier-1 check: did we just create an empty commit (allowed by --keep-redundant-commits)? - $changed = git diff-tree --no-commit-id --name-only -r HEAD + $changedOut = git diff-tree --no-commit-id --name-only -r HEAD 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git diff-tree failed after cherry-pick: $(($changedOut | Out-String).Trim())" } + $changed = @($changedOut | Where-Object { $_ -and $_.Trim() }) if (-not $changed) { - $commitMessage = @(git log -1 --format='%B' HEAD) -join "`n" + $msgOut = git log -1 --format='%B' HEAD 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git log --format='%B' HEAD failed after cherry-pick: $(($msgOut | Out-String).Trim())" } + $commitMessage = @($msgOut) -join "`n" $expectedFooter = "\(cherry picked from commit $([regex]::Escape($fullSha))\)" if ($commitMessage -notmatch $expectedFooter) { throw "Refusing to reset --hard ${prePickHead}: HEAD does not contain the cherry-pick footer for $fullSha. Investigate before retrying." From 5d9d280f4b8b26a4810749bea9bcaa88968da514 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 19:37:18 +0800 Subject: [PATCH 74/82] Round 24: register Process handlers before Start; don't Write-Error in catch under Stop; exit-check git diff --cached MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 04-try-build.ps1: - Was `Process::Start(psi)` then attached OutputDataReceived / ErrorDataReceived afterwards. Anything the cmd shell, razzle.cmd preamble, or MSBuild banner emitted before .NET wired up the handlers was dropped, so `log_tail` could miss the only line that explained the failure (e.g. "razzle: bad arg"). - Now construct a `New-Object Process`, assign StartInfo, attach both handlers, `[void]\.Start()`, then BeginOutputReadLine / BeginErrorReadLine. No window between Start and the handler subscription. - Outer `catch` did `Write-Error \.Exception.Message` while `\Continue='Stop'` was set at the top of the script. That turns each Write-Error into its own terminating error, so the `throw` after it never sees the original exception — operators got the second-order stop record instead of the actual git/msbuild failure. Switched to `[Console]::Error.WriteLine` for the diagnostic and kept the bare `throw` to rethrow `\` unmodified. 03-cherry-pick-one.ps1: - `git diff --cached --name-only` after a failed `cherry-pick --continue` ran without an exit-code check. If `git diff` itself failed (corrupted index, missing object) the stderr text would be assigned to `\`, become truthy, and the script would proceed to `git cherry-pick --abort` instead of taking the `--skip` branch — or worse, masked the underlying repo problem with a misleading stuck issue. - Now captures stderr (`2>&1`), throws with the actual git message on non-zero exit, and filters whitespace-only lines before deciding the staged set is empty. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../scripts/03-cherry-pick-one.ps1 | 4 +++- .../upstream-sync/scripts/04-try-build.ps1 | 18 +++++++++++++----- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index cb8085a55..398ff82ac 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -222,7 +222,9 @@ if ($unhandled.Count -gt 0) { git cherry-pick --continue --no-edit 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } if ($LASTEXITCODE -ne 0) { # Could still be empty after Tier-0. - $staged = git diff --cached --name-only + $stagedOut = git diff --cached --name-only 2>&1 + if ($LASTEXITCODE -ne 0) { throw "git diff --cached --name-only failed after cherry-pick --continue: $(($stagedOut | Out-String).Trim())" } + $staged = @($stagedOut | Where-Object { $_ -and $_.Trim() }) if (-not $staged) { git cherry-pick --skip | Out-Null if ($LASTEXITCODE -ne 0) { throw "git cherry-pick --skip failed after an empty Tier-0 continuation." } diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 3199a33e1..2df697f2c 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -116,17 +116,21 @@ try { $psi.UseShellExecute = $false $psi.CreateNoWindow = $true - $proc = [System.Diagnostics.Process]::Start($psi) - $baseWriter = $null - $writer = $null + $proc = New-Object System.Diagnostics.Process + $proc.StartInfo = $psi + $baseWriter = $null + $writer = $null try { # Tee stdout/stderr to the log file as the build runs. The synchronized # wrapper serializes concurrent stdout/stderr DataReceived callbacks. + # Open the log + register handlers *before* Start() so the earliest + # build banner (razzle preamble, MSBuild startup) cannot be lost. $baseWriter = [System.IO.StreamWriter]::new($logPath, $false, [System.Text.UTF8Encoding]::new($false)) $writer = [System.IO.TextWriter]::Synchronized($baseWriter) $proc.add_OutputDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) $proc.add_ErrorDataReceived({ param($s,$e) if ($null -ne $e.Data) { $writer.WriteLine($e.Data) } }) + [void]$proc.Start() $proc.BeginOutputReadLine() $proc.BeginErrorReadLine() @@ -174,7 +178,11 @@ try { } | ConvertTo-Json -Depth 4 } catch { - Write-Error $_.Exception.Message - Write-Error $_.ScriptStackTrace + # $ErrorActionPreference='Stop' turns Write-Error into a terminating + # error, which would shadow the original exception. Emit diagnostics + # straight to stderr instead so we preserve the original record on + # the rethrow below. + [Console]::Error.WriteLine($_.Exception.Message) + [Console]::Error.WriteLine($_.ScriptStackTrace) throw } From 853d11d2f4edb1a84820f827e07fafad225e7f8f Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Fri, 5 Jun 2026 19:46:14 +0800 Subject: [PATCH 75/82] Round 25: clarify merge-base --is-ancestor comment in 02 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comment said "ancestor (already on origin/main, drop it)" but the ancestor check is `\` vs `\` (the watermark from `Get-LastSyncedUpstreamSha`), not `origin/main` directly. The distinction matters: the watermark is the *newest* upstream SHA we've already picked, recorded in the trailer; the filter drops any SHA at or before the watermark on upstream/main's history. Rewrote to call that out — "reachable from \ (the watermark)" — and explained exit-1 as "newer than the watermark, keep it". No behavior change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../upstream-sync/scripts/02-compute-pending.ps1 | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 57288a0a3..649dacd7b 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -82,13 +82,17 @@ function Get-PendingUpstreamShas { if ($Since) { $filtered = New-Object 'System.Collections.Generic.List[string]' foreach ($sha in $shas) { - # git merge-base --is-ancestor: - # exit 0 = ancestor (already on origin/main, drop it) - # exit 1 = not an ancestor (keep) - # exit >1 = real error (bad object, missing ref, shallow clone) + # git merge-base --is-ancestor $sha $Since: + # exit 0 = $sha is reachable from $Since (the watermark) — + # i.e. it was already cherry-picked at or before + # the trailer we read; drop it. + # exit 1 = $sha is NOT reachable from $Since — it is newer + # than the watermark; keep it for picking. + # exit >1 = real error (bad object, missing ref, shallow + # clone — pending list cannot be trusted). $null = git merge-base --is-ancestor $sha $Since 2>$null switch ($LASTEXITCODE) { - 0 { } # ancestor, drop + 0 { } # at/before watermark, drop 1 { [void] $filtered.Add($sha) } default { throw "git merge-base --is-ancestor failed (exit $LASTEXITCODE) on $sha vs $Since; pending list cannot be trusted." } } From 2724b5bd77d227a52b4838988edff139590ac519 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 14:26:42 +0800 Subject: [PATCH 76/82] Round 27 (post-merge polish): shrink SKILL.md under 500-line cap; delete duplicated reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After convergence on Copilot review, a fresh structural review caught 3 concrete issues against `.github/instructions/agent-skills.instructions.md`: 1. SKILL.md was 568 lines, exceeding the hard 500-line cap (`agent-skills.instructions.md:364` — "500 is the hard maximum"). 2. `references/04-build-verification.md` duplicated SKILL.md step 7 without adding unique content. 3. (Considered: absolute URLs in SKILL.md gh-issue/PR-body heredocs. Kept absolute — GitHub-rendered issue/PR bodies do not resolve relative repo paths, so absolute is correct in those contexts.) Changes: - Extracted ``Direct-to-main``, ``First-time sync``, and ``Squash-merge recovery`` (55 lines) from SKILL.md into a new `references/recovery-procedures.md`. Replaced with a 6-line pointer section in SKILL.md. - Compacted the ``After-PR review handling`` section (40 lines) into a 10-line summary plus a pointer to the existing `references/follow-up-pr.md` (which already carries the full rubric, so the in-SKILL.md version was duplicate prose). - Deleted `references/04-build-verification.md`. Its JSON output contract lives in `04-try-build.ps1`'s docstring; its Generated-Files artifact note is already in SKILL.md step 7. - Updated SKILL.md References list to drop 04-build-verification and add recovery-procedures. - Fixed two stale ``#first-time-sync`` anchor links (in the State Model bullet at line 52 and in Troubleshooting) to point at the extracted section in `references/recovery-procedures.md`. Result: SKILL.md is now 495 lines (down from 568), under the 500-line cap. No information is lost — all extracted content is preserved in the references, and the SKILL.md body stays focused on the runbook the agent actually executes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 109 +++--------------- .../references/04-build-verification.md | 86 -------------- .../references/recovery-procedures.md | 60 ++++++++++ 3 files changed, 78 insertions(+), 177 deletions(-) delete mode 100644 .github/skills/upstream-sync/references/04-build-verification.md create mode 100644 .github/skills/upstream-sync/references/recovery-procedures.md diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 05e0e80d7..5e46efaa1 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -49,7 +49,7 @@ the operator can audit in your transcript. - **No `state.json` to bootstrap.** Watermark comes from the `(cherry picked from commit )` trailers on `origin/main`. If the fork has never used `cherry-pick -x` (or trailers were stripped), - see [First-time sync](#first-time-sync) below for the one-time operator step. + see [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark) for the one-time operator step. ## State Model (no state file) @@ -407,99 +407,26 @@ gh pr merge -R microsoft/intelligent-terminal $prUrl --rebase --auto --delete-br Surface `$prUrl` to the operator. Done. -### Direct-to-main (admin-only escape hatch) +### Direct-to-main, first-time seed, squash-recovery -If the operator explicitly says "skip the PR, push straight to main": after -step 7 succeeds, skip step 8 and instead: - -```pwsh -git switch main -git merge --ff-only $branch -git push origin main -git branch -D $branch -``` - -This requires bypass-branch-protection rights. No PR, no review checkpoint — -use only for explicit admin runs. - -### First-time sync - -If the fork has no `(cherry picked from commit )` trailer on -`origin/main` yet, `02-compute-pending.ps1` will throw. To seed, -the operator commits an **empty seed commit** carrying the trailer -in its message — the same format `cherry-pick -x` would emit, written -by hand exactly once: - -```pwsh -git remote add upstream https://github.com/microsoft/terminal.git -git fetch upstream main --no-tags -git switch main -# Pick an upstream SHA you consider "already in this fork" (typically the -# commit the fork was originally branched from). Write the trailer EXACTLY -# as cherry-pick -x would so the next sync picks it up. -$seedSha = '<40-char-upstream-sha-already-in-fork>' -git commit --allow-empty -m "chore(upstream): seed upstream-sync watermark" -m "(cherry picked from commit $seedSha)" -git push origin main -``` - -That one merged trailer becomes the watermark. From then on, every -run extends it. No script needed — the bootstrap is just one -`commit --allow-empty` ever. - -### Squash-merge recovery (don't do this, but if you did) - -The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` -locks in the safe path. If a reviewer squash-merges anyway: - -- The squash commit's body usually concatenates every cherry-picked - message, so it contains MANY `(cherry picked from commit )` - trailers. The watermark resolver matches the FIRST one it sees, - which is the oldest cherry-pick in the squashed batch. That means - the watermark moves backward, and the next sync run re-picks - everything between the oldest and newest commits of the squashed batch. -- The recovery is the same as a first-time seed: commit an empty - watermark commit on `main` carrying the trailer for the upstream - HEAD that was actually merged, and push. +Off-the-happy-path procedures (admin direct-push, seeding the +watermark on a fresh fork, recovering from an accidental squash-merge) +live in [`references/recovery-procedures.md`](./references/recovery-procedures.md). +The agent does not drive them — each requires an explicit operator +decision. ## After-PR review handling — fix-in-PR vs. follow-up PR -Once the sync PR is open, Copilot and human reviewers will leave -comments. The cherry-pick PR's commits must stay reviewable as -"per-commit, faithful to upstream + minimal must-merge delta" so the -reviewer can spot-check each upstream commit was applied correctly. -That constraint shapes the response policy: +Once the sync PR is open, reviewers (Copilot + humans) will comment. +**Only build-blocking fixes** belong on the sync branch as **one** +focused extra commit. Everything else — code-quality, logic-bug +suggestions, translation corrections, spelling-allowlist migrations, +doc nits, design feedback — goes to a **follow-up PR** that targets +the sync branch (not `main`). The cherry-pick PR must stay reviewable +as "per-commit, faithful to upstream + minimum must-merge delta". -| Comment type | Where to fix | -|---|---| -| **Build-blocking** (compile error, dedup of resw/manifest collisions exposed only at build time, CI gate failure on the sync PR itself) | **One** focused extra commit on the sync branch. The cherry-pick PR is what's broken; the cherry-pick PR is what gets the fix. The build-then-finalize order in this file already lands this commit in the same PR as the picks. | -| **Everything else** — code-quality findings, logic-bug suggestions, translation corrections, spelling-allowlist migrations, typo fixes, doc nits, design feedback | **Follow-up PR** on top of the cherry-pick PR (see [references/follow-up-pr.md](./references/follow-up-pr.md)) | - -**Why split.** A reviewer scanning the cherry-pick PR is auditing -"did the sync engine faithfully apply the upstream batch?" Mixing -substantive review feedback into the same PR forces them to mentally -subtract those commits from every upstream-comparison check. Worse, -amending or squashing in review fixes destroys the per-commit -attribution the cherry-pick approach was chosen to preserve. - -**Follow-up PR shape** (template in -[references/follow-up-pr.md](./references/follow-up-pr.md)): - -- New worktree + branch `dev//sync--review-fixes` - off the sync PR's HEAD. -- Base = the sync branch (e.g. `upstream-sync/2026-06-04-091512-a3f1` — - copy the exact name from the sync PR), **not** `main`. The follow-up - rides along with the sync PR. -- One focused commit per concern (code-bugs / translations / - spelling-cleanup / etc.) — same "audit trail per finding" rule as the - Copilot PR review loop skill. -- Reply + resolve every original thread on the sync PR pointing to the - follow-up PR number. -- If the sync PR merges first, rebase the follow-up onto `main` before - it merges. - -The PR body's banner (see [step 8](#8-finalize-the-pr)) spells this -policy out to the first reviewer so they don't push back on deferred -fixes. +Full rubric, worktree mechanics, and PR-body template: +[`references/follow-up-pr.md`](./references/follow-up-pr.md). ## Gotchas @@ -551,7 +478,7 @@ fixes. | Issue | Solution | |---|---| -| `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [First-time sync](#first-time-sync). | +| `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | | Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | | Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"skipped-empty"` and the agent's loop skips it. No action needed. | | Same file conflicts every run | Add it to the Tier-0 list in [references/03-known-conflicts.md](./references/03-known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | @@ -561,8 +488,8 @@ fixes. - [references/03-conflict-triage.md](./references/03-conflict-triage.md) — Tier 0/1/2/3 resolution rubric with examples. - [references/03-known-conflicts.md](./references/03-known-conflicts.md) — files that always need a fixed resolution. -- [references/04-build-verification.md](./references/04-build-verification.md) — try-build pipeline expectations. - [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. +- [references/recovery-procedures.md](./references/recovery-procedures.md) — direct-to-main, first-time seed, squash-merge recovery. - [scripts/02-compute-pending.ps1](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). - [scripts/03-cherry-pick-one.ps1](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. - [scripts/04-try-build.ps1](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. diff --git a/.github/skills/upstream-sync/references/04-build-verification.md b/.github/skills/upstream-sync/references/04-build-verification.md deleted file mode 100644 index a3f3d6081..000000000 --- a/.github/skills/upstream-sync/references/04-build-verification.md +++ /dev/null @@ -1,86 +0,0 @@ -# Build verification - -Post-pick hard gate. Runs **after** the cherry-pick loop and **before** -the PR is opened. If the build fails, the agent either lands one focused -build-fix commit on the same branch (so it ships in the same PR) or — if -the fix is too large / scope creep — surfaces the failure to the operator -and exits without filing an issue. - -The full orchestration around try-build lives in -[`SKILL.md` → Run a sync → step 7](../SKILL.md#7-build); this file is -just the contract for the `04-try-build.ps1` script and its diagnostics. - -## Why this exists - -A scheduler that opens PRs without proof the codebase still builds is -opening broken PRs. PR #220 was the motivating real-world failure: -every cherry-pick applied cleanly, every file looked right under -`git diff`, and the build broke because two unrelated upstream renames -landed `.resw` keys that collided with a fork-local commit. The compiler -catches that with zero false positives — `git` cannot. - -Toolchain provisioning (e.g. `PlatformToolset` v143/v145) is treated as -the operator's problem, not the scheduler's: an under-provisioned host -just keeps tripping the build gate and the human notices on the next -re-run. We intentionally do **not** auto-bump toolset versions in -the repo on behalf of a single host. - -## Try-build (`scripts/04-try-build.ps1`) - -Default invocation: - -```cmd -cmd.exe /c "tools\razzle.cmd && bz no_clean" -``` - -(`bz no_clean` = incremental Debug build of the full solution.) - -Configurable via the script's `-BuildCommand` parameter. The default is -verified on the maintainer host; if the build fails, the diagnostics -include the log path and tail. - -Output (returned as JSON on stdout): - -| Field | Meaning | -|---|---| -| `kind` | `build-ok` / `build-failed` / `build-inconclusive` | -| `exit_code` | Process exit code (`-1` for `build-inconclusive`) | -| `duration_ms` | Wall-clock ms | -| `command` | The build command that was run | -| `log_path` | Repo-relative path to the full log (under `Generated Files/upstream-sync//build-logs/`, gitignored) | -| `log_tail` | Last ~200 lines for inline display to the operator | - -Timeout: - -- Default 45 minutes (`-TimeoutMinutes`). -- On timeout the build is killed and classified as `build-inconclusive`. - -## When the build fails - -The agent's decision tree is in [`SKILL.md` step 7](../SKILL.md#7-build). -In short: try ONE focused fix commit when the cause is mechanical and -clearly caused by the pick batch; otherwise surface the failure to the -operator and exit. Do **not** pile up multiple fix commits — the -one-fix-per-PR rule exists so the cherry-pick PR stays auditable as -"upstream batch + at most one mechanical fix". - -## When the build fails for fork-unrelated reasons - -If a flaky build (transient toolchain glitch, env issue, missing -PlatformToolset, ...) trips the gate: - -1. The operator sees the log tail surfaced from step 7a. -2. They re-run the build locally, confirm it's transient or fix - the host, then re-run the sync. The next attempt re-picks the same - range from scratch and re-validates. - -Distinguishing transient-build from real-pick-broke-build is left to -the human reviewing the failure — too noisy to automate, and the cost -of a manual cross-check is small (~once per N runs). - -## Build artifacts - -`Generated Files/upstream-sync//build-logs/` is **not** -committed — the repo root's `**/Generated Files/` gitignore rule -covers it. Build outputs under `bin/`, `obj/`, etc. follow the repo's -existing `.gitignore`. diff --git a/.github/skills/upstream-sync/references/recovery-procedures.md b/.github/skills/upstream-sync/references/recovery-procedures.md new file mode 100644 index 000000000..635597635 --- /dev/null +++ b/.github/skills/upstream-sync/references/recovery-procedures.md @@ -0,0 +1,60 @@ +# Recovery procedures (rarely needed) + +Three off-the-happy-path procedures that the agent does **not** drive +itself — they require explicit operator decisions. Linked from +[`SKILL.md`](../SKILL.md) so the main runbook stays focused on the +common case. + +## Direct-to-main (admin-only escape hatch) + +If the operator explicitly says "skip the PR, push straight to main": +after step 7 succeeds, skip step 8 and instead: + +```pwsh +git switch main +git merge --ff-only $branch +git push origin main +git branch -D $branch +``` + +Requires bypass-branch-protection rights. No PR, no review checkpoint — +use only for explicit admin runs. + +## First-time sync (seeding the watermark) + +If the fork has no `(cherry picked from commit )` trailer on +`origin/main` yet, `02-compute-pending.ps1` will throw. To seed, +commit an **empty seed commit** carrying the trailer in its message — +the same format `cherry-pick -x` would emit, written by hand exactly +once: + +```pwsh +git remote add upstream https://github.com/microsoft/terminal.git +git fetch upstream main --no-tags +git switch main +# Pick an upstream SHA you consider "already in this fork" (typically +# the commit the fork was originally branched from). Write the trailer +# EXACTLY as cherry-pick -x would so the next sync picks it up. +$seedSha = '<40-char-upstream-sha-already-in-fork>' +git commit --allow-empty -m "chore(upstream): seed upstream-sync watermark" -m "(cherry picked from commit $seedSha)" +git push origin main +``` + +That one merged trailer becomes the watermark. From then on, every +run extends it. No script needed — the bootstrap is one +`commit --allow-empty` ever. + +## Squash-merge recovery (don't do this, but if you did) + +The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` +locks in the safe path. If a reviewer squash-merges anyway: + +- The squash commit's body usually concatenates every cherry-picked + message, so it contains MANY `(cherry picked from commit )` + trailers. The watermark resolver matches the FIRST one it sees, + which is the oldest cherry-pick in the squashed batch. That means + the watermark moves backward, and the next sync re-picks everything + between the oldest and newest commits of the squashed batch. +- The recovery is the same as a first-time seed: commit an empty + watermark commit on `main` carrying the trailer for the upstream + HEAD that was actually merged, and push. From 2755d170a6a58595f902fdb14cbe8ad363719775 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 15:14:56 +0800 Subject: [PATCH 77/82] Round 28 (concision): extract Run-a-sync runbook into references/ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SKILL.md: 495 → 156 lines (well under the 200-line soft target from agent-skills.instructions.md L312; matches sibling skill copilot-pr-review-loop/SKILL.md which is 117 lines). The full 8-step procedure with commands, JSON contracts, and failure-handling moved verbatim into references/run-a-sync.md (336 lines). SKILL.md now keeps only the ASCII flow diagram + four key invariants and points at the runbook — same progressive-disclosure pattern the sibling skill uses. Gotchas compressed from 13 to 9 entries; kept only items Copilot would not know from training (the trailer-as-watermark trick, squash-merge hazard, gh pr create retry pattern, take-upstream tier-0 file, single-host scheduler caveat, CRLF/LF on manifests). Generic git advice dropped. No information lost — everything is reachable via the References list. The 8-step contract, all scripts, and all reference docs are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 535 ++++-------------- .../upstream-sync/references/run-a-sync.md | 336 +++++++++++ 2 files changed, 434 insertions(+), 437 deletions(-) create mode 100644 .github/skills/upstream-sync/references/run-a-sync.md diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 5e46efaa1..15df541aa 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -7,49 +7,34 @@ license: MIT # Upstream Sync (microsoft/terminal → intelligent-terminal) Cherry-pick commit-by-commit from `https://github.com/microsoft/terminal` -into this fork, preserving per-commit attribution, skipping commits that -cancel each other out, and stopping cleanly the moment a human-judgment -conflict appears. +into this fork, preserving per-commit attribution, auto-skipping picks +that cancel each other out, and stopping cleanly the moment a +human-judgment conflict appears. **You — the agent reading this file — are the orchestrator.** Each step -below is a single atomic call into one of the `scripts/*.ps1` files. Run -them in order, parse their JSON output, and decide where to branch based -on the result. There is intentionally no PowerShell driver that calls -them for you, because every interesting decision (build-fix vs. open -stuck issue, retry vs. bail, finalize vs. dry-run) wants LLM judgment -the operator can audit in your transcript. +in the run is a single atomic call into one of the `scripts/*.ps1` files. +There is intentionally no PowerShell driver, because every interesting +decision (build-fix vs. open stuck issue, retry vs. bail, finalize vs. +dry-run) wants LLM judgment the operator can audit in your transcript. ## When to Use This Skill - User asks to "sync upstream", "pull from microsoft/terminal", "catch up to upstream", or "run upstream sync". -- A scheduler invokes the agent on a weekly/daily cadence to walk this file's [Run a sync](#run-a-sync) procedure. -- The previous run left an `upstream-sync-stuck` labeled issue open and the - human has finished resolving the conflict — **close the issue** (that IS - the lock-clear signal) and re-run. +- A scheduler invokes the agent on a weekly/daily cadence. +- The previous run left an `upstream-sync-stuck` labeled issue open and the human has finished resolving the conflict — **close the issue** (that IS the lock-clear signal) and re-run. ## When NOT to Use This Skill -- The user wants a **one-shot rebase** of a single feature branch onto upstream — that's a normal `git rebase`, not this skill. +- User wants a **one-shot rebase** of a single feature branch onto upstream — that's a normal `git rebase`, not this skill. - An `upstream-sync-stuck` labeled issue is open on `microsoft/intelligent-terminal` — do not re-run; resolve the conflict on the stuck branch first, then close the issue. ## Prerequisites -- `git` 2.38+ (needed for `git cherry-pick --keep-redundant-commits`) and `gh` CLI authenticated against `microsoft/intelligent-terminal`. The credential needs **push to topic branches matching `upstream-sync/*`** and **issue + label create** on the same repo. -- **`origin` MUST point at `microsoft/intelligent-terminal`.** All scripts push to `origin` and the PR / stuck-issue creation passes `-R microsoft/intelligent-terminal` explicitly. If `origin` is a personal fork, the push will land on the fork and the PR will fail to find its head. Use `git remote -v` to verify before running. -- PowerShell 7+ (`pwsh`) on PATH. -- Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and the repo's `tools\razzle.cmd`/`bz` build environment (build is a hard gate before finalize — see [step 7](#7-build)). -- Remote named `upstream` — the scripts create it if missing. -- **Full git history on `origin/main`** (no shallow clone). Watermark - discovery scans up to 5000 commits on `origin/main` for `cherry picked - from commit ` trailers and the pending walk does `merge-base - --is-ancestor` checks. A shallow clone (e.g. GitHub Actions default - `fetch-depth=1`) will produce wrong/empty results. If running in CI, - use `actions/checkout@v4` with `fetch-depth: 0` (or run `git fetch - --unshallow` before invoking the skill). -- **No `state.json` to bootstrap.** Watermark comes from the - `(cherry picked from commit )` trailers on `origin/main`. If - the fork has never used `cherry-pick -x` (or trailers were stripped), - see [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark) for the one-time operator step. +- `git` 2.38+ (for `cherry-pick --keep-redundant-commits`), `gh` CLI authenticated against `microsoft/intelligent-terminal` (needs push to `upstream-sync/*` topic branches + issue/label create), PowerShell 7+ (`pwsh`) on PATH. +- **`origin` MUST point at `microsoft/intelligent-terminal`.** All scripts push to `origin` and pass `-R microsoft/intelligent-terminal` explicitly. Verify with `git remote -v` before running. +- **Full git history on `origin/main`** (no shallow clone). Watermark discovery walks up to 5000 commits scanning for `cherry picked from commit ` trailers. In CI, use `actions/checkout@v4` with `fetch-depth: 0`. +- Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and `tools\razzle.cmd` / `bz`. Build is a hard gate before finalize. +- **No `state.json` to bootstrap.** Watermark = newest `(cherry picked from commit )` trailer on `origin/main`. If the fork has never used `cherry-pick -x`, run the one-time [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). ## State Model (no state file) @@ -57,439 +42,115 @@ Every persistent fact lives in the source that owns it: | Question | Source of truth | |---|---| -| What's the last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived inline by [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | -| What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, then drop SHAs older than (or equal to) the watermark above. Patch-id-based, so a picked-then-reverted commit correctly re-appears. | -| Is the scheduler locked? | Any OPEN issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing the issue IS the lock-clear signal. | -| What does the lock mean? | The issue body (plain markdown — no machine-parseable block) names the stuck commit, the branch, and the conflicting paths. Closing the issue clears the lock. | -| Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/` rule. Never committed. | +| Last-synced upstream commit? | Newest `(cherry picked from commit )` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived by [`02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | +| What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, drop SHAs at/before the watermark. Patch-id based, so a picked-then-reverted commit correctly re-appears. | +| Is the scheduler locked? | Any open issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing it IS the lock-clear signal. | +| Where do build logs go? | `Generated Files/upstream-sync//` — gitignored by the repo root's `**/Generated Files/`. Never committed. | -### Why cherry-pick (and not rebase or merge) +### Why cherry-pick (not rebase or merge) -| Approach | Why rejected / chosen | -|---|---| -| **Rebase** `upstream/main` | ❌ Fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts. Verified failure on sister repo `agentic-terminal`. | -| **Merge** `upstream/main` | ⚠️ Works, but collapses the whole sync into one blob commit — kills per-commit review, kills `git bisect`. | -| **Cherry-pick commit-by-commit** | ✅ Preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR with N small commits. | +- **Rebase upstream/main** — ❌ fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts (verified failure on sister repo). +- **Merge upstream/main** — ⚠️ works, but collapses the whole sync into one blob commit, killing per-commit review and `git bisect`. +- **Cherry-pick commit-by-commit** — ✅ preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR. ## Run a sync -This is the orchestration you (the agent) execute. **Do not skip steps.** -Each step's "On failure" path is mandatory. - -### 1. Preconditions (bail fast) - -```pwsh -# (a) working tree clean -git status --porcelain -# → empty? continue. nonempty? bail and tell the operator. - -# (b) on main, fast-forward -git switch main -git pull --ff-only - -# (c) no open stuck-lock -gh issue list -R microsoft/intelligent-terminal --label upstream-sync-stuck --state open --json number,title,url -# → []? continue. -# → nonempty? STOP. Surface the issue URL(s) to the operator and exit. -# The lock-clear signal is the human closing the issue, not a script. -``` - -### 2. Build a branch name - -```pwsh -$utc = (Get-Date).ToUniversalTime() -$date = $utc.ToString('yyyy-MM-dd') -$tstamp = $utc.ToString('HHmmss') -$rand = [guid]::NewGuid().ToString('N').Substring(0,4) -$branch = "upstream-sync/$date-$tstamp-$rand" -git switch -c $branch -``` - -The fresh suffix means re-runs never collide with stale local branches. - -### 3. Fetch upstream - -```pwsh -# Accept any equivalent form pointing at microsoft/terminal — HTTPS with -# or without .git, SSH form, etc. We only care about owner/repo identity. -$expectedOwnerRepo = 'microsoft/terminal' -$existing = git remote get-url upstream 2>$null -function Get-OwnerRepo([string] $url) { - if (-not $url) { return $null } - $u = $url.Trim() - # https://host/owner/repo[.git] - if ($u -match '^https?://[^/]+/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } - # ssh://[user@]host[:port]/owner/repo[.git] - if ($u -match '^ssh://(?:[^@/]+@)?[^/:]+(?::\d+)?/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } - # scp-style: [user@]host:owner/repo[.git] - if ($u -match '^(?:[^@:/]+@)?[^:/]+:([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } - return $null -} -if (-not $existing) { - git remote add upstream "https://github.com/$expectedOwnerRepo.git" -} elseif ((Get-OwnerRepo $existing) -ne $expectedOwnerRepo) { - throw "Remote 'upstream' points at '$existing' but this skill requires $expectedOwnerRepo. Aborting so we don't silently sync from the wrong fork." -} -git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } -$upstreamSha = (git rev-parse upstream/main).Trim() -``` - -### 4. Compute pending - -```pwsh -$pendingJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/02-compute-pending.ps1 -if ($LASTEXITCODE -ne 0) { throw "02-compute-pending.ps1 exited $LASTEXITCODE." } -$pending = $pendingJson | ConvertFrom-Json -# $pending.from = last-synced watermark SHA -# $pending.to = upstream/main SHA -# $pending.pending = SHAs to pick, oldest-first -# $pending.dropped_pairs = [[picked, reverted], ...] — auto-cancelled -# $pending.skipped_empty = SHAs the script can statically prove are no-ops -``` - -If `$pending.pending.Count -eq 0`: nothing to do. Delete the branch, tell -the operator "fork is up to date with ", exit. - -### 5. Cherry-pick loop - -For each SHA in `$pending.pending`: - -```pwsh -$pickJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 -Sha $sha -if ($LASTEXITCODE -ne 0) { throw "03-cherry-pick-one.ps1 exited $LASTEXITCODE for $sha." } -$pick = $pickJson | ConvertFrom-Json -``` - -Branch on `$pick.status`: - -- `"picked"` — record `$sha` in `$picked`, continue. -- `"skipped-empty"` — record `$sha` in `$skippedEmpty`, continue. -- `"stuck"` — **stop the loop** and go to step 5a. - -#### 5a. On stuck — open the Tier-3 issue and exit - -Push the branch, ensure the lock label exists, and file the issue with a -plain-markdown body. Closing the issue is the lock-clear signal — no -machine-readable metadata is needed. - -```pwsh -git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } - -# Idempotent — `gh label create` returns non-zero when the label already -# exists. We swallow that specific case but surface everything else so an -# auth/repo-typo failure doesn't get silently absorbed. -$labelOut = gh label create 'upstream-sync-stuck' ` - --color B60205 ` - --description 'Upstream sync paused — close to clear the lock' ` - -R microsoft/intelligent-terminal 2>&1 -$labelOutText = ($labelOut | Out-String) -if ($LASTEXITCODE -ne 0 -and $labelOutText -notmatch 'already exists') { - throw "gh label create failed: $labelOutText" -} - -$author = git log -1 --format='%an <%ae>' $sha -$subject = git log -1 --format='%s' $sha -$shortSha = $sha.Substring(0,9) -$hasConflicts = $pick.conflict_paths -and $pick.conflict_paths.Count -gt 0 -$pathLines = if ($hasConflicts) { - ($pick.conflict_paths | ForEach-Object { "- ``$_``" }) -join "`n" -} else { '_(none — cherry-pick failed without producing unmerged paths)_' } -$headline = if ($hasConflicts) { - 'Upstream sync stopped at a conflict that needs human judgment.' -} else { - 'Upstream sync stopped on a non-conflict cherry-pick failure (e.g. merge commit without `-m`, unsupported git option, hook failure).' -} -$body = @" -> [!CAUTION] -> $headline -> **Close this issue when resolved — that IS the lock-clear signal.** - -**Stuck on:** ``$sha`` — $subject -**Upstream commit:** https://github.com/microsoft/terminal/commit/$sha -**Author:** $author -**Sync branch:** ``$branch`` - -**Conflicting paths:** -$pathLines - -$(if ($pick.error) { "**Error:** ``$($pick.error)``" }) - -**To resume** (the script ran ``git cherry-pick --abort`` — the branch -has no in-progress conflict state to resolve directly): - -``````pwsh -git fetch origin -git switch $branch -git cherry-pick -x $sha # re-run to reproduce the conflict; -x keeps the trailer -# ... resolve conflicts ... -git add -A -git cherry-pick --continue -git push -# Open a PR against main, merge without squashing (the trailer must survive), -# then close this issue to clear the lock. -`````` - -See [references/03-conflict-triage.md](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/03-conflict-triage.md) for the resolution rubric. -"@ - -$bodyFile = New-TemporaryFile -try { - Set-Content -Encoding utf8 -Path $bodyFile -Value $body - $issueUrl = gh issue create -R microsoft/intelligent-terminal ` - --title "Upstream sync stuck at $shortSha" ` - --label upstream-sync-stuck ` - --body-file $bodyFile - if ($LASTEXITCODE -ne 0) { throw "gh issue create failed." } -} finally { - Remove-Item -Force -ErrorAction SilentlyContinue $bodyFile -} -``` - -Surface `$issueUrl` and `$branch` to the operator. The human is expected to: - -1. `git fetch origin && git switch $branch` to pick up the sync branch. -2. **Re-run the cherry-pick to reproduce the conflict** — `03-cherry-pick-one.ps1` - calls `git cherry-pick --abort` before returning `stuck`, so the branch - itself has no conflict markers / `MERGE_MSG`. Use the stuck SHA from - the issue body: `git cherry-pick -x ` (the `-x` preserves - the `(cherry picked from commit )` trailer — critical for the - watermark). -3. Resolve the conflicts, `git add` the resolutions, `git cherry-pick --continue`. -4. Push, open a PR against `main`, merge it (the `-x` trailer must - survive the merge — DO NOT squash). -5. Close the stuck issue. - -EXIT — do not attempt the build. - -See [references/03-conflict-triage.md](./references/03-conflict-triage.md) -for what "Tier-3" means and the resolution rubric. - -### 6. (No commits picked? exit clean.) - -If the loop emitted only `empty` results (rare — every pending commit was -a no-op) there is nothing to build or finalize. Delete the branch, report -"sync completed with 0 commits picked (all empty)", exit. - -### 7. Build - -```pwsh -$buildJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/04-try-build.ps1 -if ($LASTEXITCODE -ne 0) { throw "04-try-build.ps1 exited $LASTEXITCODE before producing a JSON result." } -$build = $buildJson | ConvertFrom-Json -``` - -Build runs BEFORE finalize on purpose — if it fails, the fix lands as ONE -extra commit on the same branch so it ends up in the same PR. - -Branch on `$build.kind`: - -- `"build-ok"` — continue to step 8. -- `"build-failed"` — try ONE focused build-fix: - 1. Read `$build.log_tail`. Decide if the failure is small, mechanical, and - clearly caused by the cherry-pick batch (e.g. duplicate `.resw` key - from a fork-local commit colliding with an upstream rename, missing - `#include` resolved by the upstream batch, etc.). - 2. If yes — fix it, `git add` only the affected files, commit with - subject `chore(upstream): fix build after upstream sync` (carry the - `Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>` - trailer if you authored the fix), re-run `04-try-build.ps1`, go back - to start of step 7. - 3. If no (large change, scope creep, requires design decisions) — go to step 7a. - 4. If the fix attempt itself fails to compile — go to step 7a. Do NOT - pile up multiple fix commits; the one-fix-per-PR rule is policy. -- `"build-inconclusive"` — go to step 7a (timeout, treated as a real failure - unless the operator explicitly opts out). - -#### 7a. On build failure — surface and exit - -The fix attempt didn't land or the failure is too big for a one-commit -fix. Don't open an issue. Surface the failure to the operator and exit — -they'll either fix the underlying defect on `main` and re-run, or push a -manual fix commit on top of `$branch` and continue to step 8 by hand. +Eight-step orchestration. Full commands, JSON contracts, and +failure-handling for every step are in +[`references/run-a-sync.md`](./references/run-a-sync.md). The flow: -```pwsh -[Console]::Error.WriteLine("Build failed after upstream sync:") -[Console]::Error.WriteLine(" branch: $branch") -[Console]::Error.WriteLine(" kind: $($build.kind)") -[Console]::Error.WriteLine(" exit_code: $($build.exit_code)") -[Console]::Error.WriteLine(" log_path: $($build.log_path)") -[Console]::Error.WriteLine("--- log tail ---") -[Console]::Error.WriteLine($build.log_tail) -exit 1 ``` - -### 8. Finalize the PR - -The agent (you) push the branch and open the PR directly with `git` and -`gh`. There is no wrapper script — there's nothing here that needs more -than `gh pr create` plus a body string you compose from the data you -already have in `$picked` / `$build` / `$pending`. - -Compose the body: - -```pwsh -$banner = @" -> [!WARNING] -> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked -> upstream commit into one, destroying per-commit attribution, original -> author dates, the ``(cherry picked from commit )`` trailers that the -> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. -> Merge with **"Rebase and merge"** (preferred — flat history, all -> $($picked.Count) commit(s) land individually) or **"Create a merge commit"**. - -> [!NOTE] -> **Review-fix policy.** Only build-blocking fixes belong on this branch -> as **one** focused extra commit. All other Copilot / human review -> feedback goes into a **follow-up PR** based on this PR's head. See -> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). - ---- - -"@ - -$summary = @" -Syncs **$($picked.Count)** commit(s) from microsoft/terminal up to ``$($upstreamSha.Substring(0,9))``. - -## Picked -$( ($picked | ForEach-Object { "- ``$($_.Substring(0,9))`` $(git log -1 --format='%s' $_)" }) -join "`n" ) - -$(if ($pending.dropped_pairs.Count) { @" -## Auto-dropped revert pairs -$( ($pending.dropped_pairs | ForEach-Object { "- ``$($_[0].Substring(0,9))`` ↔ ``$($_[1].Substring(0,9))`` (cancel out within range)" }) -join "`n" ) -"@ }) - -$(if ($skippedEmpty.Count) { @" -## Skipped (empty / already applied) -$( ($skippedEmpty | ForEach-Object { "- ``$($_.Substring(0,9))``" }) -join "`n" ) -"@ }) - -## Build -``04-try-build.ps1`` → ``$($build.kind)`` (exit $($build.exit_code), $($build.duration_ms) ms). Log: ``$($build.log_path)`` (gitignored). -"@ - -$bodyFile = New-TemporaryFile -[System.IO.File]::WriteAllText($bodyFile, ($banner + $summary), (New-Object System.Text.UTF8Encoding($false))) +1. Preconditions (clean tree, on main FF, no open stuck issue) +2. Create branch upstream-sync/ +3. Fetch upstream (verify owner/repo identity of the `upstream` remote) +4. Compute pending ← scripts/02-compute-pending.ps1 (→ JSON) +5. Cherry-pick loop ← scripts/03-cherry-pick-one.ps1 per SHA (→ JSON) + picked / skipped-empty → continue + stuck → 5a (push branch, file upstream-sync-stuck issue, EXIT) +6. (No commits picked? exit clean.) +7. Build ← scripts/04-try-build.ps1 (→ JSON) + build-ok → step 8 + build-failed → ONE focused fix commit, re-build; else 7a + build-inconclusive → 7a + 7a. Surface failure to operator, EXIT (no issue filed for build failures) +8. Finalize: push, `gh pr create`, optional `gh pr merge --rebase --auto` ``` -Push and create the PR. `gh pr create` on Windows can occasionally fail -with "Head sha can't be blank" right after a push — retry up to 3× with -a short delay: - -```pwsh -git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } -if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } - -$short = $upstreamSha.Substring(0,9) -$title = "chore(upstream): sync microsoft/terminal up to $short" +**Key invariants** (the agent MUST hold these — full rationale in the runbook): -$prUrl = $null -for ($attempt = 1; $attempt -le 3; $attempt++) { - $prOut = gh pr create -R microsoft/intelligent-terminal ` - --base main --head $branch --title $title --body-file $bodyFile 2>&1 - $prExit = $LASTEXITCODE - $prText = ($prOut | Out-String).Trim() - if ($prExit -eq 0) { - $urlLine = ($prOut | Where-Object { "$_" -match '^https://github.com/' } | Select-Object -Last 1) - if ($urlLine) { $prUrl = "$urlLine".Trim(); break } - } - Write-Warning "gh pr create attempt $attempt failed (exit $prExit). Full output:`n$prText" - Start-Sleep -Seconds 5 -} -Remove-Item -LiteralPath $bodyFile -Force -if (-not $prUrl) { throw "gh pr create did not return a URL after 3 attempts." } - -# Optional: arm GitHub auto-merge with rebase (NEVER squash). -# Skip this if the operator wants to merge by hand. -gh pr merge -R microsoft/intelligent-terminal $prUrl --rebase --auto --delete-branch -``` +- **Atomic per-commit.** Each pick is one commit on the sync branch carrying the upstream author/date and the `(cherry picked from commit )` trailer. Never amend across picks. +- **Stuck → exit clean.** On a Tier-3 conflict (script returns `status: "stuck"`), do NOT continue past it — push the branch, file the labeled issue, exit. The human resolves on the stuck branch, merges (no squash), closes the issue. +- **One build-fix commit, max.** Build-blocking fixes land as exactly one extra commit on the sync branch. Anything bigger → exit; the operator pushes a manual fix or runs a follow-up PR. +- **Never squash-merge the sync PR.** Squashing destroys per-commit attribution AND the trailer the next sync uses as its watermark. The PR body banner warns reviewers; the recipe arms `--rebase --auto`. -Surface `$prUrl` to the operator. Done. +## Recovery procedures (rare) -### Direct-to-main, first-time seed, squash-recovery - -Off-the-happy-path procedures (admin direct-push, seeding the -watermark on a fresh fork, recovering from an accidental squash-merge) -live in [`references/recovery-procedures.md`](./references/recovery-procedures.md). -The agent does not drive them — each requires an explicit operator -decision. +Direct-to-main escape hatch, first-time watermark seeding, squash-merge +recovery: [`references/recovery-procedures.md`](./references/recovery-procedures.md). +Each requires explicit operator action. ## After-PR review handling — fix-in-PR vs. follow-up PR Once the sync PR is open, reviewers (Copilot + humans) will comment. -**Only build-blocking fixes** belong on the sync branch as **one** -focused extra commit. Everything else — code-quality, logic-bug -suggestions, translation corrections, spelling-allowlist migrations, -doc nits, design feedback — goes to a **follow-up PR** that targets -the sync branch (not `main`). The cherry-pick PR must stay reviewable -as "per-commit, faithful to upstream + minimum must-merge delta". - -Full rubric, worktree mechanics, and PR-body template: +**Only build-blocking fixes** belong on the sync branch as one focused +extra commit. Everything else — code-quality, logic-bug suggestions, +translation corrections, spelling-allowlist migrations, doc nits, design +feedback — goes into a **follow-up PR** that targets the sync branch +(not `main`). The cherry-pick PR must stay reviewable as "per-commit, +faithful to upstream + minimum must-merge delta". + +Full rubric, worktree mechanics, PR-body template: [`references/follow-up-pr.md`](./references/follow-up-pr.md). ## Gotchas -- **Never squash-merge the sync PR.** Use **"Rebase and merge"** - (preferred) or **"Create a merge commit"**. The PR body opens with a - banner reminding the reviewer; the step-8 recipe shows the - `gh pr merge --rebase --auto` invocation so a tired reviewer can't get - it wrong. -- **Don't amend substantive review fixes into the sync PR.** Only - build-blocking fixes get **one** extra commit on the sync branch. See - [references/follow-up-pr.md](./references/follow-up-pr.md). -- **Never rebase `upstream/main` onto this fork.** Use cherry-pick. - Verified failure mode on the sister repo `agentic-terminal`. -- **`.github/workflows/spelling2.yml` always conflicts** and the correct - resolution is always "take upstream wholesale". The Tier-0 list in - [references/03-known-conflicts.md](./references/03-known-conflicts.md) - handles this automatically — extend the list when you discover the - next file with the same pattern. -- **`gh pr create` on Windows can fail with "Head sha can't be blank"** if the - branch is freshly pushed and not yet visible. The step-8 recipe wraps - the call in a 3× retry loop — do not "fix" the recipe to use - `--head :` (which would point `gh` at a fork). -- **Do not run the orchestration twice while a stuck issue is open.** The - step-1 preflight catches it, but a human bypassing that gate manually - would overwrite the stuck branch and lose their in-progress resolution. -- **Cherry-pick over `git revert` style commits is intentional, not skipped.** - We only skip revert-pairs where **both** sides are inside the pending - range. A revert of a commit we already merged last week must land as a - normal pick — otherwise the fork diverges silently. +- **Never squash-merge the sync PR.** Squashing destroys per-commit + attribution AND collapses the trailers the next run uses as its + watermark. The PR body opens with a banner; step 8 arms + `gh pr merge --rebase --auto`. - **Never strip the `(cherry picked from commit )` trailer** when - hand-resolving a stuck pick. That trailer IS the watermark the next - sync run reads. -- **Build logs live under `Generated Files/upstream-sync//`.** - This directory is gitignored at the repo root (`**/Generated Files/`). - Do not check these in. -- **Single-host scheduler.** The stuck-lock (open labeled issue) is a - read-then-check gate, not an atomic lease — two hosts running on the - same tick can both observe "no open issue" and proceed in parallel. - Run the scheduler from ONE host. For multi-host fan-out, layer atomic - locking on top (GitHub Actions `concurrency: upstream-sync` group is - the easiest). -- **CRLF/LF on manifest files.** Cherry-picks normally preserve upstream - line endings, but any in-flight resolution touched by an LLM may - downgrade to LF. If a Tier-2 resolution touches a - `.yml`/`.xml`/`.csproj`/winget manifest, re-normalize before staging — - see [references/03-conflict-triage.md](./references/03-conflict-triage.md#line-endings). + hand-resolving a stuck pick. That trailer IS the watermark. +- **Never rebase `upstream/main` onto this fork.** Use cherry-pick — + rebase replays old "Merge upstream" commits and explodes. +- **Don't amend substantive review fixes into the sync PR.** Only + build-blocking fixes (max one extra commit). Everything else → follow-up PR. +- **`.github/workflows/spelling2.yml` always conflicts** and is always + "take upstream wholesale". The Tier-0 list in + [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) + handles it — extend the list when you find the next file with this pattern. +- **`gh pr create` on Windows can fail with "Head sha can't be blank"** + on freshly-pushed branches. The step-8 recipe wraps it in a 3× retry. + Do not "fix" it to use `--head :` (that points `gh` at a fork). +- **Cherry-pick over `git revert`-style commits is intentional, not + skipped.** We only skip revert-pairs where **both** sides are inside + the pending range. A revert of an already-merged commit must land — + otherwise the fork diverges silently. +- **Single-host scheduler.** The stuck-lock is a read-then-check gate, + not an atomic lease. Run from ONE host. For multi-host fan-out, layer + atomic locking on top (GitHub Actions `concurrency: upstream-sync` is + easiest). +- **CRLF/LF on manifest files.** Cherry-picks preserve upstream endings, + but Tier-2 LLM-touched resolutions on `.yml`/`.xml`/`.csproj`/winget + manifests may downgrade to LF. Re-normalize before staging — see + [`references/03-conflict-triage.md`](./references/03-conflict-triage.md#line-endings). ## Troubleshooting | Issue | Solution | |---|---| -| `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer ..." | The fork has never used `cherry-pick -x` for an upstream commit yet. Run the one-time seeding pick described in [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | -| Stuck issue prevents new run | Resolve the conflict on the stuck branch, open a PR, merge it (keep the `(cherry picked from commit )` trailer!), then **close the stuck issue**. The next scheduler tick proceeds. | -| Cherry-pick reports "empty commit" | Expected for upstream no-op commits and for fork-already-applied patches; `03-cherry-pick-one.ps1` returns `"skipped-empty"` and the agent's loop skips it. No action needed. | -| Same file conflicts every run | Add it to the Tier-0 list in [references/03-known-conflicts.md](./references/03-known-conflicts.md) with the correct resolution strategy (`take-upstream`, `take-ours`, or `union`). | -| `gh pr create` returns "Head sha can't be blank" | The step-8 recipe retries 3× automatically. On slow networks, the operator may need a manual second run of the whole sync. | +| `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer …" | Fork has never used `cherry-pick -x` yet. Run the one-time [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | +| Stuck issue prevents new run | Resolve on the stuck branch, open + merge a PR (keep the trailer, don't squash), then **close the stuck issue**. The next scheduler tick proceeds. | +| `03-cherry-pick-one.ps1` returns `"skipped-empty"` | Expected for upstream no-op commits and fork-already-applied patches. The loop skips and continues. | +| Same file conflicts every run | Add it to [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) with the right strategy (`take-upstream`, `take-ours`, `union`). | +| `gh pr create` returns "Head sha can't be blank" | Step 8 retries 3×. On slow networks, the operator may need a manual second run. | ## References -- [references/03-conflict-triage.md](./references/03-conflict-triage.md) — Tier 0/1/2/3 resolution rubric with examples. -- [references/03-known-conflicts.md](./references/03-known-conflicts.md) — files that always need a fixed resolution. -- [references/follow-up-pr.md](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. -- [references/recovery-procedures.md](./references/recovery-procedures.md) — direct-to-main, first-time seed, squash-merge recovery. -- [scripts/02-compute-pending.ps1](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). -- [scripts/03-cherry-pick-one.ps1](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. -- [scripts/04-try-build.ps1](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. +- [`references/run-a-sync.md`](./references/run-a-sync.md) — full eight-step procedure with commands. +- [`references/03-conflict-triage.md`](./references/03-conflict-triage.md) — Tier 0/1/2/3 conflict-resolution rubric. +- [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) — files with fixed Tier-0 resolutions. +- [`references/follow-up-pr.md`](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. +- [`references/recovery-procedures.md`](./references/recovery-procedures.md) — direct-to-main, first-time seed, squash-merge recovery. +- [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). +- [`scripts/03-cherry-pick-one.ps1`](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. +- [`scripts/04-try-build.ps1`](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. diff --git a/.github/skills/upstream-sync/references/run-a-sync.md b/.github/skills/upstream-sync/references/run-a-sync.md new file mode 100644 index 000000000..232a62eff --- /dev/null +++ b/.github/skills/upstream-sync/references/run-a-sync.md @@ -0,0 +1,336 @@ +# Run a sync — full procedure + +This is the orchestration the agent executes step-by-step. The high-level +flow and invariants are in [`SKILL.md`](../SKILL.md); this file is the +full runbook, with commands. **Do not skip steps.** Each step's "On +failure" path is mandatory. + +## 1. Preconditions (bail fast) + +```pwsh +# (a) working tree clean +git status --porcelain +# → empty? continue. nonempty? bail and tell the operator. + +# (b) on main, fast-forward +git switch main +git pull --ff-only + +# (c) no open stuck-lock +gh issue list -R microsoft/intelligent-terminal --label upstream-sync-stuck --state open --json number,title,url +# → []? continue. +# → nonempty? STOP. Surface the issue URL(s) to the operator and exit. +# The lock-clear signal is the human closing the issue, not a script. +``` + +## 2. Build a branch name + +```pwsh +$utc = (Get-Date).ToUniversalTime() +$date = $utc.ToString('yyyy-MM-dd') +$tstamp = $utc.ToString('HHmmss') +$rand = [guid]::NewGuid().ToString('N').Substring(0,4) +$branch = "upstream-sync/$date-$tstamp-$rand" +git switch -c $branch +``` + +The fresh suffix means re-runs never collide with stale local branches. + +## 3. Fetch upstream + +```pwsh +# Accept any equivalent form pointing at microsoft/terminal — HTTPS with +# or without .git, SSH form, etc. We only care about owner/repo identity. +$expectedOwnerRepo = 'microsoft/terminal' +$existing = git remote get-url upstream 2>$null +function Get-OwnerRepo([string] $url) { + if (-not $url) { return $null } + $u = $url.Trim() + # https://host/owner/repo[.git] + if ($u -match '^https?://[^/]+/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } + # ssh://[user@]host[:port]/owner/repo[.git] + if ($u -match '^ssh://(?:[^@/]+@)?[^/:]+(?::\d+)?/([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } + # scp-style: [user@]host:owner/repo[.git] + if ($u -match '^(?:[^@:/]+@)?[^:/]+:([^/]+/[^/]+?)(?:\.git)?/?$') { return $Matches[1].ToLowerInvariant() } + return $null +} +if (-not $existing) { + git remote add upstream "https://github.com/$expectedOwnerRepo.git" +} elseif ((Get-OwnerRepo $existing) -ne $expectedOwnerRepo) { + throw "Remote 'upstream' points at '$existing' but this skill requires $expectedOwnerRepo. Aborting so we don't silently sync from the wrong fork." +} +git fetch upstream main --no-tags 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git fetch upstream main failed." } +$upstreamSha = (git rev-parse upstream/main).Trim() +``` + +## 4. Compute pending + +```pwsh +$pendingJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/02-compute-pending.ps1 +if ($LASTEXITCODE -ne 0) { throw "02-compute-pending.ps1 exited $LASTEXITCODE." } +$pending = $pendingJson | ConvertFrom-Json +# $pending.from = last-synced watermark SHA +# $pending.to = upstream/main SHA +# $pending.pending = SHAs to pick, oldest-first +# $pending.dropped_pairs = [[picked, reverted], ...] — auto-cancelled +# $pending.skipped_empty = SHAs the script can statically prove are no-ops +``` + +If `$pending.pending.Count -eq 0`: nothing to do. Delete the branch, tell +the operator "fork is up to date with ", exit. + +## 5. Cherry-pick loop + +For each SHA in `$pending.pending`: + +```pwsh +$pickJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 -Sha $sha +if ($LASTEXITCODE -ne 0) { throw "03-cherry-pick-one.ps1 exited $LASTEXITCODE for $sha." } +$pick = $pickJson | ConvertFrom-Json +``` + +Branch on `$pick.status`: + +- `"picked"` — record `$sha` in `$picked`, continue. +- `"skipped-empty"` — record `$sha` in `$skippedEmpty`, continue. +- `"stuck"` — **stop the loop** and go to step 5a. + +### 5a. On stuck — open the Tier-3 issue and exit + +Push the branch, ensure the lock label exists, and file the issue with a +plain-markdown body. Closing the issue is the lock-clear signal — no +machine-readable metadata is needed. + +```pwsh +git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } + +# Idempotent — `gh label create` returns non-zero when the label already +# exists. We swallow that specific case but surface everything else so an +# auth/repo-typo failure doesn't get silently absorbed. +$labelOut = gh label create 'upstream-sync-stuck' ` + --color B60205 ` + --description 'Upstream sync paused — close to clear the lock' ` + -R microsoft/intelligent-terminal 2>&1 +$labelOutText = ($labelOut | Out-String) +if ($LASTEXITCODE -ne 0 -and $labelOutText -notmatch 'already exists') { + throw "gh label create failed: $labelOutText" +} + +$author = git log -1 --format='%an <%ae>' $sha +$subject = git log -1 --format='%s' $sha +$shortSha = $sha.Substring(0,9) +$hasConflicts = $pick.conflict_paths -and $pick.conflict_paths.Count -gt 0 +$pathLines = if ($hasConflicts) { + ($pick.conflict_paths | ForEach-Object { "- ``$_``" }) -join "`n" +} else { '_(none — cherry-pick failed without producing unmerged paths)_' } +$headline = if ($hasConflicts) { + 'Upstream sync stopped at a conflict that needs human judgment.' +} else { + 'Upstream sync stopped on a non-conflict cherry-pick failure (e.g. merge commit without `-m`, unsupported git option, hook failure).' +} +$body = @" +> [!CAUTION] +> $headline +> **Close this issue when resolved — that IS the lock-clear signal.** + +**Stuck on:** ``$sha`` — $subject +**Upstream commit:** https://github.com/microsoft/terminal/commit/$sha +**Author:** $author +**Sync branch:** ``$branch`` + +**Conflicting paths:** +$pathLines + +$(if ($pick.error) { "**Error:** ``$($pick.error)``" }) + +**To resume** (the script ran ``git cherry-pick --abort`` — the branch +has no in-progress conflict state to resolve directly): + +``````pwsh +git fetch origin +git switch $branch +git cherry-pick -x $sha # re-run to reproduce the conflict; -x keeps the trailer +# ... resolve conflicts ... +git add -A +git cherry-pick --continue +git push +# Open a PR against main, merge without squashing (the trailer must survive), +# then close this issue to clear the lock. +`````` + +See [references/03-conflict-triage.md](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/03-conflict-triage.md) for the resolution rubric. +"@ + +$bodyFile = New-TemporaryFile +try { + Set-Content -Encoding utf8 -Path $bodyFile -Value $body + $issueUrl = gh issue create -R microsoft/intelligent-terminal ` + --title "Upstream sync stuck at $shortSha" ` + --label upstream-sync-stuck ` + --body-file $bodyFile + if ($LASTEXITCODE -ne 0) { throw "gh issue create failed." } +} finally { + Remove-Item -Force -ErrorAction SilentlyContinue $bodyFile +} +``` + +Surface `$issueUrl` and `$branch` to the operator. The human: + +1. `git fetch origin && git switch $branch` to pick up the sync branch. +2. **Re-run the cherry-pick to reproduce the conflict** — `03-cherry-pick-one.ps1` + calls `git cherry-pick --abort` before returning `stuck`, so the branch + itself has no conflict markers / `MERGE_MSG`. Use the stuck SHA from + the issue body: `git cherry-pick -x ` (the `-x` preserves + the `(cherry picked from commit )` trailer — critical for the + watermark). +3. Resolve the conflicts, `git add` the resolutions, `git cherry-pick --continue`. +4. Push, open a PR against `main`, merge it (the `-x` trailer must + survive the merge — DO NOT squash). +5. Close the stuck issue. + +EXIT — do not attempt the build. +See [`03-conflict-triage.md`](./03-conflict-triage.md) for the Tier-0/1/2/3 +resolution rubric. + +## 6. (No commits picked? exit clean.) + +If the loop emitted only `empty` results (rare — every pending commit was +a no-op) there is nothing to build or finalize. Delete the branch, report +"sync completed with 0 commits picked (all empty)", exit. + +## 7. Build + +```pwsh +$buildJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/04-try-build.ps1 +if ($LASTEXITCODE -ne 0) { throw "04-try-build.ps1 exited $LASTEXITCODE before producing a JSON result." } +$build = $buildJson | ConvertFrom-Json +``` + +Build runs BEFORE finalize on purpose — if it fails, the fix lands as ONE +extra commit on the same branch so it ends up in the same PR. + +Branch on `$build.kind`: + +- `"build-ok"` — continue to step 8. +- `"build-failed"` — try ONE focused build-fix: + 1. Read `$build.log_tail`. Decide if the failure is small, mechanical, and + clearly caused by the cherry-pick batch (e.g. duplicate `.resw` key + from a fork-local commit colliding with an upstream rename, missing + `#include` resolved by the upstream batch, etc.). + 2. If yes — fix it, `git add` only the affected files, commit with + subject `chore(upstream): fix build after upstream sync` (carry the + `Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>` + trailer if you authored the fix), re-run `04-try-build.ps1`, go back + to start of step 7. + 3. If no (large change, scope creep, requires design decisions) — go to step 7a. + 4. If the fix attempt itself fails to compile — go to step 7a. Do NOT + pile up multiple fix commits; the one-fix-per-PR rule is policy. +- `"build-inconclusive"` — go to step 7a (timeout, treated as a real failure + unless the operator explicitly opts out). + +### 7a. On build failure — surface and exit + +The fix attempt didn't land or the failure is too big for a one-commit +fix. Don't open an issue. Surface the failure to the operator and exit — +they'll either fix the underlying defect on `main` and re-run, or push a +manual fix commit on top of `$branch` and continue to step 8 by hand. + +```pwsh +[Console]::Error.WriteLine("Build failed after upstream sync:") +[Console]::Error.WriteLine(" branch: $branch") +[Console]::Error.WriteLine(" kind: $($build.kind)") +[Console]::Error.WriteLine(" exit_code: $($build.exit_code)") +[Console]::Error.WriteLine(" log_path: $($build.log_path)") +[Console]::Error.WriteLine("--- log tail ---") +[Console]::Error.WriteLine($build.log_tail) +exit 1 +``` + +## 8. Finalize the PR + +The agent pushes the branch and opens the PR directly with `git` and +`gh`. There is no wrapper script — there's nothing here that needs more +than `gh pr create` plus a body string composed from `$picked` / +`$build` / `$pending`. + +Compose the body: + +```pwsh +$banner = @" +> [!WARNING] +> **DO NOT squash-merge this PR.** Squashing collapses every cherry-picked +> upstream commit into one, destroying per-commit attribution, original +> author dates, the ``(cherry picked from commit )`` trailers that the +> NEXT upstream sync uses as its watermark, and ``git bisect`` resolution. +> Merge with **"Rebase and merge"** (preferred — flat history, all +> $($picked.Count) commit(s) land individually) or **"Create a merge commit"**. + +> [!NOTE] +> **Review-fix policy.** Only build-blocking fixes belong on this branch +> as **one** focused extra commit. All other Copilot / human review +> feedback goes into a **follow-up PR** based on this PR's head. See +> [``.github/skills/upstream-sync/references/follow-up-pr.md``](https://github.com/microsoft/intelligent-terminal/blob/main/.github/skills/upstream-sync/references/follow-up-pr.md). + +--- + +"@ + +$summary = @" +Syncs **$($picked.Count)** commit(s) from microsoft/terminal up to ``$($upstreamSha.Substring(0,9))``. + +## Picked +$( ($picked | ForEach-Object { "- ``$($_.Substring(0,9))`` $(git log -1 --format='%s' $_)" }) -join "`n" ) + +$(if ($pending.dropped_pairs.Count) { @" +## Auto-dropped revert pairs +$( ($pending.dropped_pairs | ForEach-Object { "- ``$($_[0].Substring(0,9))`` ↔ ``$($_[1].Substring(0,9))`` (cancel out within range)" }) -join "`n" ) +"@ }) + +$(if ($skippedEmpty.Count) { @" +## Skipped (empty / already applied) +$( ($skippedEmpty | ForEach-Object { "- ``$($_.Substring(0,9))``" }) -join "`n" ) +"@ }) + +## Build +``04-try-build.ps1`` → ``$($build.kind)`` (exit $($build.exit_code), $($build.duration_ms) ms). Log: ``$($build.log_path)`` (gitignored). +"@ + +$bodyFile = New-TemporaryFile +[System.IO.File]::WriteAllText($bodyFile, ($banner + $summary), (New-Object System.Text.UTF8Encoding($false))) +``` + +Push and create the PR. `gh pr create` on Windows can occasionally fail +with "Head sha can't be blank" right after a push — retry up to 3× with +a short delay: + +```pwsh +git push -u origin $branch 2>&1 | ForEach-Object { [Console]::Error.WriteLine($_) } +if ($LASTEXITCODE -ne 0) { throw "git push of $branch failed." } + +$short = $upstreamSha.Substring(0,9) +$title = "chore(upstream): sync microsoft/terminal up to $short" + +$prUrl = $null +for ($attempt = 1; $attempt -le 3; $attempt++) { + $prOut = gh pr create -R microsoft/intelligent-terminal ` + --base main --head $branch --title $title --body-file $bodyFile 2>&1 + $prExit = $LASTEXITCODE + $prText = ($prOut | Out-String).Trim() + if ($prExit -eq 0) { + $urlLine = ($prOut | Where-Object { "$_" -match '^https://github.com/' } | Select-Object -Last 1) + if ($urlLine) { $prUrl = "$urlLine".Trim(); break } + } + Write-Warning "gh pr create attempt $attempt failed (exit $prExit). Full output:`n$prText" + Start-Sleep -Seconds 5 +} +Remove-Item -LiteralPath $bodyFile -Force +if (-not $prUrl) { throw "gh pr create did not return a URL after 3 attempts." } + +# Optional: arm GitHub auto-merge with rebase (NEVER squash). +# Skip this if the operator wants to merge by hand. +gh pr merge -R microsoft/intelligent-terminal $prUrl --rebase --auto --delete-branch +``` + +Surface `$prUrl` to the operator. Done. From 080c9feccc5fb7942952c64a303b7d8fa1e75f3f Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 15:24:16 +0800 Subject: [PATCH 78/82] Round 28b: fix recovery-procedures.md errors flagged by Copilot review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three findings on Squash-merge recovery section, all correct: 1. `-AutoMergeStrategy rebase` is not a real flag in this skill — the step-8 recipe uses `gh pr merge --rebase --auto`. Wording updated. 2. The watermark resolver in 02-compute-pending.ps1 walks trailers bottom-up (newest-first, lines 60-66), so a squash commit with the trailers preserved picks the NEWEST trailer, not the first/oldest. Watermark does NOT move backward in that case. Doc was wrong. 3. Therefore recovery is conditional: needed only when the squash body was hand-edited to strip the newest trailer (or all trailers). Re-seeding unconditionally would be unnecessary work most of the time. Section rewritten to make the condition explicit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../references/recovery-procedures.md | 34 ++++++++++++------- 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/.github/skills/upstream-sync/references/recovery-procedures.md b/.github/skills/upstream-sync/references/recovery-procedures.md index 635597635..871e87428 100644 --- a/.github/skills/upstream-sync/references/recovery-procedures.md +++ b/.github/skills/upstream-sync/references/recovery-procedures.md @@ -46,15 +46,25 @@ run extends it. No script needed — the bootstrap is one ## Squash-merge recovery (don't do this, but if you did) -The PR banner shouts "do not squash" and `-AutoMergeStrategy rebase` -locks in the safe path. If a reviewer squash-merges anyway: - -- The squash commit's body usually concatenates every cherry-picked - message, so it contains MANY `(cherry picked from commit )` - trailers. The watermark resolver matches the FIRST one it sees, - which is the oldest cherry-pick in the squashed batch. That means - the watermark moves backward, and the next sync re-picks everything - between the oldest and newest commits of the squashed batch. -- The recovery is the same as a first-time seed: commit an empty - watermark commit on `main` carrying the trailer for the upstream - HEAD that was actually merged, and push. +The PR banner shouts "do not squash" and step 8 arms +`gh pr merge --rebase --auto` (the merge strategy is ultimately a +GitHub UI choice — there is no `-AutoMergeStrategy` flag). If a +reviewer squash-merges anyway, the outcome depends on what GitHub +preserved in the squash commit's body: + +- **Trailers preserved (common case).** The squash body concatenates + every cherry-picked message, so it contains MANY + `(cherry picked from commit )` trailers. + [`02-compute-pending.ps1`](../scripts/02-compute-pending.ps1) walks + these bottom-up (newest-first) so it picks the **last** trailer — + which corresponds to the newest upstream commit in the batch. The + watermark does NOT move backward, and **no recovery is needed**. + Per-commit attribution on `main` is gone (that's the squash cost), + but the next sync resumes correctly. +- **Trailers stripped or newest trailer removed.** If the reviewer + hand-edited the squash body and dropped the newest trailer (or all + of them), the watermark will resolve to an older SHA and the next + sync re-picks the gap. To prevent that, do a one-time + `commit --allow-empty` on `main` carrying the trailer for the + upstream HEAD that was actually merged — same shape as the first-time + seed above. From 5514cf2e31615fcfae21b54ddc169bd91f857367 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 15:34:02 +0800 Subject: [PATCH 79/82] Round 28c: drop misleading `union` from troubleshooting; accept uppercase revert SHAs Two findings from Copilot: 1. SKILL.md troubleshooting table listed `union` as a Tier-0 strategy. 03-cherry-pick-one.ps1:191 emits "union strategy not implemented yet" and escalates to stuck, and 03-known-conflicts.md:12 says so explicitly. Dropped `union` from the parenthetical and added the "reserved, currently escalates" note so operators don't add entries that won't auto-resolve. 2. Revert-pair regex `[0-9a-f]{40}` is case-sensitive. While `git revert` itself emits lowercase, the body could carry an uppercase SHA (hand-edited message, scripted commits, third-party tools). Switched to `[0-9a-fA-F]{40}` so we don't silently miss a revert-pair and re-apply both halves on the fork. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 2 +- .github/skills/upstream-sync/scripts/02-compute-pending.ps1 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 15df541aa..534f7f6c9 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -141,7 +141,7 @@ Full rubric, worktree mechanics, PR-body template: | `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer …" | Fork has never used `cherry-pick -x` yet. Run the one-time [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | | Stuck issue prevents new run | Resolve on the stuck branch, open + merge a PR (keep the trailer, don't squash), then **close the stuck issue**. The next scheduler tick proceeds. | | `03-cherry-pick-one.ps1` returns `"skipped-empty"` | Expected for upstream no-op commits and fork-already-applied patches. The loop skips and continues. | -| Same file conflicts every run | Add it to [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) with the right strategy (`take-upstream`, `take-ours`, `union`). | +| Same file conflicts every run | Add it to [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) with the right strategy (`take-upstream` or `take-ours`; `union` is reserved and currently escalates instead of auto-resolving). | | `gh pr create` returns "Head sha can't be blank" | Step 8 retries 3×. On slow networks, the operator may need a manual second run. | ## References diff --git a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 index 649dacd7b..6cb5d9af4 100644 --- a/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 +++ b/.github/skills/upstream-sync/scripts/02-compute-pending.ps1 @@ -146,7 +146,7 @@ foreach ($sha in $all) { $subj = $info[$sha].subject $targetSha = $null - if ($body -match 'This reverts commit ([0-9a-f]{40})\b') { + if ($body -match 'This reverts commit ([0-9a-fA-F]{40})\b') { $targetSha = $Matches[1] } elseif ($subj -match '^Revert "') { # Fallback: match by quoted subject, but only against commits From d8527f256bb5b54f542a8fa0e911d5969b1d4bba Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 15:43:13 +0800 Subject: [PATCH 80/82] Round 28d: fix 04-try-build.ps1 synopsis/parameter mismatch Copilot review: synopsis described the default as the full `cmd /c "tools\razzle.cmd && bz no_clean"` wrapper, but `-BuildCommand` and the `command` field in the JSON output are just the inner cmd string (the `/c` wrapper is added at line 106). Clarified the synopsis to match what callers actually pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/04-try-build.ps1 | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/skills/upstream-sync/scripts/04-try-build.ps1 b/.github/skills/upstream-sync/scripts/04-try-build.ps1 index 2df697f2c..4705ae11c 100644 --- a/.github/skills/upstream-sync/scripts/04-try-build.ps1 +++ b/.github/skills/upstream-sync/scripts/04-try-build.ps1 @@ -1,7 +1,9 @@ <# .SYNOPSIS Try-build. Runs the configured build command in a razzle environment and - captures the result. Default: `cmd /c "tools\razzle.cmd && bz no_clean"`. + captures the result. Default `-BuildCommand`: `tools\razzle.cmd && bz no_clean` + (the script wraps it with `cmd /c "..."` internally — pass only the + cmd.exe command string, not the wrapper). .DESCRIPTION Run AFTER cherry-picking (03) and BEFORE finalizing the PR (SKILL.md From 125c379d50a73467badbcbaac996c9a6699d1333 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Sun, 7 Jun 2026 15:52:22 +0800 Subject: [PATCH 81/82] =?UTF-8?q?Round=2028e:=20simplify=20Get-ConflictPat?= =?UTF-8?q?hs=20=E2=80=94=20drop=20redundant=20-split?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Copilot review claimed `$u -split "`n"` produces a bogus single `System.String[]` entry when `$u` is a string[]. Verified interactively that PowerShell's binary -split operator iterates over arrays element-wise, so the original code worked correctly: $u = @('a/b.txt','c/d.txt') @($u -split "`n") -> @('a/b.txt','c/d.txt') (count=2, String) So the bug Copilot described doesn't exist — but the -split was redundant. Removed it; iteration is now obvious to any reader. Also quoted `$_` in the TrimEnd call to defend against the (theoretical) case where 2>&1 pulls an ErrorRecord into the pipeline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 index 398ff82ac..32ae9127f 100644 --- a/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 +++ b/.github/skills/upstream-sync/scripts/03-cherry-pick-one.ps1 @@ -49,10 +49,11 @@ function Get-KnownConflicts { function Get-ConflictPaths { # core.quotepath=off keeps non-ASCII paths in raw UTF-8 so Tier-0 # path matching against 03-known-conflicts.md works without C-quoting. + # PowerShell already streams external-command stdout as string[] + # (one element per line), so iterate directly — no -split needed. $u = git -c core.quotepath=off diff --name-only --diff-filter=U 2>&1 if ($LASTEXITCODE -ne 0) { throw "git diff --name-only --diff-filter=U failed: $u" } - if (-not $u) { return @() } - return @($u -split "`n" | ForEach-Object { $_.TrimEnd("`r") } | Where-Object { $_ }) + return @($u | ForEach-Object { "$_".TrimEnd("`r") } | Where-Object { $_ }) } $result = [ordered] @{ From 6f05260f779cdd9588cd849e23f83f09ee734af4 Mon Sep 17 00:00:00 2001 From: Yee Lam Lee Date: Tue, 16 Jun 2026 11:29:45 +0800 Subject: [PATCH 82/82] upstream-sync: add pre-build PGO database re-pin check When a sync bumps upstream's Windows Terminal version, build/pgo/Terminal.PGO.props must follow (the fork's custom.props stays 0.1 and can't derive it). Add a no-script pre-build step in run-a-sync.md plus a flow line and gotcha in SKILL.md so the agent re-pins the two PGOPackageVersion values before building instead of hitting 'Could not find matching PGO package'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/skills/upstream-sync/SKILL.md | 7 ++++ .../upstream-sync/references/run-a-sync.md | 40 +++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/.github/skills/upstream-sync/SKILL.md b/.github/skills/upstream-sync/SKILL.md index 534f7f6c9..f44a9d73b 100644 --- a/.github/skills/upstream-sync/SKILL.md +++ b/.github/skills/upstream-sync/SKILL.md @@ -68,6 +68,8 @@ failure-handling for every step are in picked / skipped-empty → continue stuck → 5a (push branch, file upstream-sync-stuck issue, EXIT) 6. (No commits picked? exit clean.) + before build: re-pin build/pgo/Terminal.PGO.props if upstream bumped + its Windows Terminal Major.Minor (else PGO build fails) 7. Build ← scripts/04-try-build.ps1 (→ JSON) build-ok → step 8 build-failed → ONE focused fix commit, re-build; else 7a @@ -133,6 +135,11 @@ Full rubric, worktree mechanics, PR-body template: but Tier-2 LLM-touched resolutions on `.yml`/`.xml`/`.csproj`/winget manifests may downgrade to LF. Re-normalize before staging — see [`references/03-conflict-triage.md`](./references/03-conflict-triage.md#line-endings). +- **PGO pin follows upstream.** `build/pgo/Terminal.PGO.props` hard-pins the + PGO database to upstream's Windows Terminal `Major.Minor` (our `custom.props` + stays `0.1`, so it can't be derived). When a sync bumps the upstream version, + re-pin those two values or the build fails with `Could not find matching PGO + package`. Step 7's pre-build check covers this. ## Troubleshooting diff --git a/.github/skills/upstream-sync/references/run-a-sync.md b/.github/skills/upstream-sync/references/run-a-sync.md index 232a62eff..b39a3bb3d 100644 --- a/.github/skills/upstream-sync/references/run-a-sync.md +++ b/.github/skills/upstream-sync/references/run-a-sync.md @@ -202,6 +202,46 @@ a no-op) there is nothing to build or finalize. Delete the branch, report ## 7. Build +### Pre-build check — re-pin the PGO database if upstream bumped its version + +This fork keeps its own product version (`custom.props` → `VersionMajor.VersionMinor` += `0.1`), so `build/pgo/Terminal.PGO.props` **cannot** derive the PGO database +package version automatically — it carries a hard-coded pin to the upstream +Windows Terminal `Major.Minor`. When a sync pulls in an upstream version bump +(e.g. `1.25 → 1.26`), that pin must follow, or the build fails with: + +``` +Microsoft.PGO-Helpers.Cpp.targets : Error : Could not find matching PGO package. +``` + +(No `Microsoft.Internal.Windows.Terminal.PGODatabase` is published for `0.1` on the +TerminalDependencies feed — only upstream versions exist.) + +Before building, compare the two values — no script needed, just read both: + +``` +# upstream product version the picked commits may have advanced: +git show upstream/main:custom.props # → / + +# current pin in build/pgo/Terminal.PGO.props: +# / +``` + +If upstream's `Major.Minor` is newer than the pin, edit those two +`` / `` values in +`build/pgo/Terminal.PGO.props` to match, and land it as its own mechanical +commit on the sync branch (it's a deterministic, upstream-driven re-pin, not a +judgment fix, so it does not consume the step-7 build-fix budget): + +``` +git add build/pgo/Terminal.PGO.props +git commit -m "chore(upstream): re-pin PGO database to ." +``` + +If the values already match, do nothing and proceed to the build. + +### Run the build + ```pwsh $buildJson = pwsh -NoProfile -File .github/skills/upstream-sync/scripts/04-try-build.ps1 if ($LASTEXITCODE -ne 0) { throw "04-try-build.ps1 exited $LASTEXITCODE before producing a JSON result." }