-
Notifications
You must be signed in to change notification settings - Fork 94
Add upstream-sync agent skill (cherry-pick from microsoft/terminal) #218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
yeelam-gordon
wants to merge
82
commits into
main
Choose a base branch
from
dev/yeelam/upstream-sync-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
a0ccbde
Add upstream-sync agent skill + bootstrap state
yeelam-gordon 91f2fff
Add post-pick validation (static scan + try-build) to upstream-sync s…
yeelam-gordon be2542d
fix(upstream-sync/scan): use absolute paths in resw dedup scan (PR #2…
yeelam-gordon 08a6c16
fix(upstream-sync): address review feedback
yeelam-gordon d6c077e
fix(upstream-sync): cover toolsets spelling identifier
yeelam-gordon 9837b73
fix(upstream-sync): harden stuck-path operations
yeelam-gordon a22edd5
fix(upstream-sync): align reporting docs
yeelam-gordon 67bada9
fix(upstream-sync): guard cleanup and build logging
yeelam-gordon a4f0612
fix(upstream-sync): pull origin main explicitly
yeelam-gordon ad61c93
fix(upstream-sync): make dry-run reports actionable
yeelam-gordon 9bda344
fix(upstream-sync): validate cherry-pick cleanup
yeelam-gordon b12cf07
fix(upstream-sync): stabilize state serialization
yeelam-gordon e14b928
fix(upstream-sync): canonicalize operator SHAs
yeelam-gordon c9baf14
fix(upstream-sync): align spelling patterns
yeelam-gordon fd76e66
docs(upstream-sync): add after-PR review handling step
yeelam-gordon 3bd3187
build(spelling): allow CamelCase *Toolsets identifiers
yeelam-gordon 59cd38f
fix(upstream-sync): reword to satisfy check-spelling
yeelam-gordon 2c821c5
fix(upstream-sync): replace 'exit' with 'throw' in helper scripts
yeelam-gordon d29b822
docs(upstream-sync): align help/retention text with throw-not-exit co…
yeelam-gordon 98c4bd2
fix(upstream-sync): tighten revert-pair fallback + broaden toolset probe
yeelam-gordon 3531230
fix(upstream-sync): existing-PR guard + case-sensitive resw dedup
yeelam-gordon d5fc5cd
fix(upstream-sync): gitignore local-only report suffixes
yeelam-gordon 2b14ccf
fix(upstream-sync): repo-relative git add + bootstrap safety + pr-lis…
yeelam-gordon a9b4075
fix(upstream-sync): prefix-boundary check + ordered state + docs drift
yeelam-gordon af25981
fix(upstream-sync): stable doc link + bypass PR gate on direct-push
yeelam-gordon 21849be
fix(upstream-sync): doc tweaks (retry count + gitignore link path)
yeelam-gordon c509cbb
fix(upstream-sync): absolute follow-up-pr link + try/finally around b…
yeelam-gordon 6aae404
fix(upstream-sync): dispose SHA256 instance in Get-FindingsHash
yeelam-gordon 2654a14
fix(upstream-sync): drop broken head: search qualifier + move real wo…
yeelam-gordon df7db3e
fix(upstream-sync): preserve caller env + stale-probe guard + missing…
yeelam-gordon 19c2bd7
fix(upstream-sync): validate Tier-0 git ops + stable findings hash
yeelam-gordon a257d13
fix(upstream-sync): single active stuck lock at a time
yeelam-gordon 5ba470a
fix(upstream-sync): toolchain-missing resume guidance + correct stuck…
yeelam-gordon 7d07f56
fix(upstream-sync): consistent kind casing + backfill before auto-merge
yeelam-gordon 0b678ad
fix(upstream-sync): handle non-conflict cherry-pick failures + fail-f…
yeelam-gordon 179892f
fix(upstream-sync): array-wrap git log output + preserve upstream ide…
yeelam-gordon dbbf346
docs(upstream-sync): correct 05 help block + 02 return-shape doc drift
yeelam-gordon 548a29e
fix(upstream-sync): use repo-root path in bootstrap hint
yeelam-gordon e7c0216
fix(upstream-sync): bootstrap example + direct-push schema + toolsets…
yeelam-gordon 8e23d74
fix(upstream-sync): bump git prereq to 2.38 + narrow toolsets pattern
yeelam-gordon d758d67
fix(upstream-sync): preserve existing BOM in Tier-2 line-ending snippet
yeelam-gordon 532463b
fix(upstream-sync): document push perms + propagate non-conflict cher…
yeelam-gordon ac2aa28
fix(upstream-sync): trim git rev-parse output for scan baseline
yeelam-gordon ccb0126
fix(upstream-sync): separate stderr from JSON + finally-cleanup PR bo…
yeelam-gordon d4278bc
fix(upstream-sync): emit repo-relative log_path in 10-try-build report
yeelam-gordon 05c3114
fix(upstream-sync): guard git add exit code and tighten invariants
yeelam-gordon b0aa32d
fix(upstream-sync): pscustomobject nesting + pin gh label create to repo
yeelam-gordon b14b7a3
fix(upstream-sync): separate stderr capture for gh pr/issue create
yeelam-gordon 0ca6911
fix(upstream-sync): fast-forward main before reading state.json
yeelam-gordon 15c6cbf
refactor(upstream-sync): derive state from git+gh, drop state.json
yeelam-gordon f99c2c1
fix(upstream-sync): address Copilot review round (15c6cbf14)
yeelam-gordon 2fc5f8e
docs(upstream-sync): sync branch-name docs with f99c2c1fd implementation
yeelam-gordon 38b8317
restructure: orchestration moves to SKILL.md, scripts become atomic ops
yeelam-gordon c3cce2b
fix: address Copilot review on 38b831768
yeelam-gordon efdb6b6
fix: address 6 more Copilot findings (round 5)
yeelam-gordon fe22795
fix: pipe diagnostics to stderr so scripts honor stdout contract
yeelam-gordon bc151b8
remove 05-finalize-pr.ps1; agent calls gh pr create directly (round 7)
yeelam-gordon 55f0026
Round 8: prune the orchestration to its essential algorithms
yeelam-gordon 10c576c
Round 9: address Copilot post-prune findings
yeelam-gordon e5e42f8
Round 10: address Copilot post-restage findings
yeelam-gordon 8e21b47
Round 11: scan all cherry-pick trailers per commit, newest-first
yeelam-gordon 4d88445
Round 12: derive branch date and time from one UTC timestamp
yeelam-gordon 840bc8b
Round 13: exit-code checks + robust label/log/cmd handling
yeelam-gordon 823f2d4
Round 14: fail fast on git errors; drop hardcoded path
yeelam-gordon 28ba860
Round 15: flatten multi-line %B; UTC-date artifacts; permissive upstr…
yeelam-gordon 3402041
Round 16: fail-fast on repo-relative log path; spell out cherry-pick …
yeelam-gordon 6033ca5
Round 17: normalize CR/whitespace; resolve -LogDir relative paths
yeelam-gordon b14aa7b
Round 18: full-history prereq; full gh-pr-create error context
yeelam-gordon 12c8c19
Round 19: drop fragile cmd /c nested quoting in try-build
yeelam-gordon 71b8e08
Round 20: hex case, merge-base error class, diff exit check
yeelam-gordon a967a35
Round 21: capture cherry-pick output; fix array unwrap in Get-Pending…
yeelam-gordon 1f13d1a
Round 22: tolerate cherry-pick --abort failure; stuck-issue wording r…
yeelam-gordon cd6ac7a
Round 23: SSH-URL parsing covers ssh://; exit-check git diff-tree + g…
yeelam-gordon 5d9d280
Round 24: register Process handlers before Start; don't Write-Error i…
yeelam-gordon 853d11d
Round 25: clarify merge-base --is-ancestor comment in 02
yeelam-gordon 2724b5b
Round 27 (post-merge polish): shrink SKILL.md under 500-line cap; del…
yeelam-gordon 2755d17
Round 28 (concision): extract Run-a-sync runbook into references/
yeelam-gordon 080c9fe
Round 28b: fix recovery-procedures.md errors flagged by Copilot review
yeelam-gordon 5514cf2
Round 28c: drop misleading `union` from troubleshooting; accept upper…
yeelam-gordon d8527f2
Round 28d: fix 04-try-build.ps1 synopsis/parameter mismatch
yeelam-gordon 125c379
Round 28e: simplify Get-ConflictPaths — drop redundant -split
yeelam-gordon 6f05260
upstream-sync: add pre-build PGO database re-pin check
yeelam-gordon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -84,6 +84,7 @@ greenfield | |
| greppable | ||
| haikus | ||
| handover | ||
| hashtable | ||
| historicals | ||
| hstrings | ||
| https | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,163 @@ | ||
| --- | ||
| name: upstream-sync | ||
| description: 'Periodically sync new commits from microsoft/terminal into this manually-forked intelligent-terminal repo by cherry-picking commit-by-commit onto a dated sync branch, auto-skipping revert pairs and empty commits, auto-resolving known take-upstream files, and stopping cleanly on genuine conflicts. The agent (you, reading this file) is the orchestrator — the PowerShell scripts are atomic operations you invoke one at a time. Use when the user asks to "sync upstream", "pull from microsoft/terminal", "run upstream sync", "catch up to upstream", or wires this into a scheduler (weekly/daily). Designed to be safe under repeated unattended runs.' | ||
| license: MIT | ||
| --- | ||
|
yeelam-gordon marked this conversation as resolved.
yeelam-gordon marked this conversation as resolved.
|
||
|
|
||
| # Upstream Sync (microsoft/terminal → intelligent-terminal) | ||
|
|
||
| Cherry-pick commit-by-commit from `https://github.com/microsoft/terminal` | ||
| into this fork, preserving per-commit attribution, auto-skipping picks | ||
| that cancel each other out, and stopping cleanly the moment a | ||
| human-judgment conflict appears. | ||
|
|
||
| **You — the agent reading this file — are the orchestrator.** Each step | ||
| in the run is a single atomic call into one of the `scripts/*.ps1` files. | ||
| There is intentionally no PowerShell driver, because every interesting | ||
| decision (build-fix vs. open stuck issue, retry vs. bail, finalize vs. | ||
| dry-run) wants LLM judgment the operator can audit in your transcript. | ||
|
|
||
| ## When to Use This Skill | ||
|
|
||
| - User asks to "sync upstream", "pull from microsoft/terminal", "catch up to upstream", or "run upstream sync". | ||
| - A scheduler invokes the agent on a weekly/daily cadence. | ||
| - The previous run left an `upstream-sync-stuck` labeled issue open and the human has finished resolving the conflict — **close the issue** (that IS the lock-clear signal) and re-run. | ||
|
|
||
| ## When NOT to Use This Skill | ||
|
|
||
| - User wants a **one-shot rebase** of a single feature branch onto upstream — that's a normal `git rebase`, not this skill. | ||
| - An `upstream-sync-stuck` labeled issue is open on `microsoft/intelligent-terminal` — do not re-run; resolve the conflict on the stuck branch first, then close the issue. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - `git` 2.38+ (for `cherry-pick --keep-redundant-commits`), `gh` CLI authenticated against `microsoft/intelligent-terminal` (needs push to `upstream-sync/*` topic branches + issue/label create), PowerShell 7+ (`pwsh`) on PATH. | ||
| - **`origin` MUST point at `microsoft/intelligent-terminal`.** All scripts push to `origin` and pass `-R microsoft/intelligent-terminal` explicitly. Verify with `git remote -v` before running. | ||
| - **Full git history on `origin/main`** (no shallow clone). Watermark discovery walks up to 5000 commits scanning for `cherry picked from commit <sha>` trailers. In CI, use `actions/checkout@v4` with `fetch-depth: 0`. | ||
| - Windows build host with Visual Studio 2022, Windows SDK, `vswhere`, and `tools\razzle.cmd` / `bz`. Build is a hard gate before finalize. | ||
| - **No `state.json` to bootstrap.** Watermark = newest `(cherry picked from commit <sha>)` trailer on `origin/main`. If the fork has never used `cherry-pick -x`, run the one-time [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | ||
|
|
||
| ## State Model (no state file) | ||
|
|
||
| Every persistent fact lives in the source that owns it: | ||
|
|
||
| | Question | Source of truth | | ||
| |---|---| | ||
| | Last-synced upstream commit? | Newest `(cherry picked from commit <sha>)` trailer on `origin/main` whose target is reachable from `upstream/main`. Derived by [`02-compute-pending.ps1`](./scripts/02-compute-pending.ps1). | | ||
| | What's pending? | `git log --cherry-pick --right-only --no-merges origin/main...upstream/main`, drop SHAs at/before the watermark. Patch-id based, so a picked-then-reverted commit correctly re-appears. | | ||
| | Is the scheduler locked? | Any open issue with the `upstream-sync-stuck` label on `microsoft/intelligent-terminal`. Closing it IS the lock-clear signal. | | ||
| | Where do build logs go? | `Generated Files/upstream-sync/<YYYY-MM-DD>/` — gitignored by the repo root's `**/Generated Files/`. Never committed. | | ||
|
|
||
| ### Why cherry-pick (not rebase or merge) | ||
|
|
||
| - **Rebase upstream/main** — ❌ fork history contains old "Merge upstream" commits; rebase replays them and explodes conflicts (verified failure on sister repo). | ||
| - **Merge upstream/main** — ⚠️ works, but collapses the whole sync into one blob commit, killing per-commit review and `git bisect`. | ||
| - **Cherry-pick commit-by-commit** — ✅ preserves authorship + per-commit content, allows mechanical revert-pair skipping, produces a reviewable PR. | ||
|
|
||
| ## Run a sync | ||
|
|
||
| Eight-step orchestration. Full commands, JSON contracts, and | ||
| failure-handling for every step are in | ||
| [`references/run-a-sync.md`](./references/run-a-sync.md). The flow: | ||
|
|
||
| ``` | ||
| 1. Preconditions (clean tree, on main FF, no open stuck issue) | ||
| 2. Create branch upstream-sync/<UTC-yyyy-MM-dd-HHmmss-rand4> | ||
| 3. Fetch upstream (verify owner/repo identity of the `upstream` remote) | ||
| 4. Compute pending ← scripts/02-compute-pending.ps1 (→ JSON) | ||
| 5. Cherry-pick loop ← scripts/03-cherry-pick-one.ps1 per SHA (→ JSON) | ||
| picked / skipped-empty → continue | ||
| stuck → 5a (push branch, file upstream-sync-stuck issue, EXIT) | ||
| 6. (No commits picked? exit clean.) | ||
| before build: re-pin build/pgo/Terminal.PGO.props if upstream bumped | ||
| its Windows Terminal Major.Minor (else PGO build fails) | ||
| 7. Build ← scripts/04-try-build.ps1 (→ JSON) | ||
| build-ok → step 8 | ||
| build-failed → ONE focused fix commit, re-build; else 7a | ||
| build-inconclusive → 7a | ||
| 7a. Surface failure to operator, EXIT (no issue filed for build failures) | ||
| 8. Finalize: push, `gh pr create`, optional `gh pr merge --rebase --auto` | ||
| ``` | ||
|
|
||
| **Key invariants** (the agent MUST hold these — full rationale in the runbook): | ||
|
|
||
| - **Atomic per-commit.** Each pick is one commit on the sync branch carrying the upstream author/date and the `(cherry picked from commit <sha>)` trailer. Never amend across picks. | ||
| - **Stuck → exit clean.** On a Tier-3 conflict (script returns `status: "stuck"`), do NOT continue past it — push the branch, file the labeled issue, exit. The human resolves on the stuck branch, merges (no squash), closes the issue. | ||
| - **One build-fix commit, max.** Build-blocking fixes land as exactly one extra commit on the sync branch. Anything bigger → exit; the operator pushes a manual fix or runs a follow-up PR. | ||
| - **Never squash-merge the sync PR.** Squashing destroys per-commit attribution AND the trailer the next sync uses as its watermark. The PR body banner warns reviewers; the recipe arms `--rebase --auto`. | ||
|
|
||
| ## Recovery procedures (rare) | ||
|
|
||
| Direct-to-main escape hatch, first-time watermark seeding, squash-merge | ||
| recovery: [`references/recovery-procedures.md`](./references/recovery-procedures.md). | ||
| Each requires explicit operator action. | ||
|
|
||
| ## After-PR review handling — fix-in-PR vs. follow-up PR | ||
|
|
||
| Once the sync PR is open, reviewers (Copilot + humans) will comment. | ||
| **Only build-blocking fixes** belong on the sync branch as one focused | ||
| extra commit. Everything else — code-quality, logic-bug suggestions, | ||
| translation corrections, spelling-allowlist migrations, doc nits, design | ||
| feedback — goes into a **follow-up PR** that targets the sync branch | ||
| (not `main`). The cherry-pick PR must stay reviewable as "per-commit, | ||
| faithful to upstream + minimum must-merge delta". | ||
|
|
||
| Full rubric, worktree mechanics, PR-body template: | ||
| [`references/follow-up-pr.md`](./references/follow-up-pr.md). | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - **Never squash-merge the sync PR.** Squashing destroys per-commit | ||
| attribution AND collapses the trailers the next run uses as its | ||
| watermark. The PR body opens with a banner; step 8 arms | ||
| `gh pr merge --rebase --auto`. | ||
| - **Never strip the `(cherry picked from commit <sha>)` trailer** when | ||
| hand-resolving a stuck pick. That trailer IS the watermark. | ||
| - **Never rebase `upstream/main` onto this fork.** Use cherry-pick — | ||
| rebase replays old "Merge upstream" commits and explodes. | ||
| - **Don't amend substantive review fixes into the sync PR.** Only | ||
| build-blocking fixes (max one extra commit). Everything else → follow-up PR. | ||
| - **`.github/workflows/spelling2.yml` always conflicts** and is always | ||
| "take upstream wholesale". The Tier-0 list in | ||
| [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) | ||
| handles it — extend the list when you find the next file with this pattern. | ||
| - **`gh pr create` on Windows can fail with "Head sha can't be blank"** | ||
| on freshly-pushed branches. The step-8 recipe wraps it in a 3× retry. | ||
| Do not "fix" it to use `--head <owner>:<branch>` (that points `gh` at a fork). | ||
| - **Cherry-pick over `git revert`-style commits is intentional, not | ||
| skipped.** We only skip revert-pairs where **both** sides are inside | ||
| the pending range. A revert of an already-merged commit must land — | ||
| otherwise the fork diverges silently. | ||
| - **Single-host scheduler.** The stuck-lock is a read-then-check gate, | ||
| not an atomic lease. Run from ONE host. For multi-host fan-out, layer | ||
| atomic locking on top (GitHub Actions `concurrency: upstream-sync` is | ||
| easiest). | ||
| - **CRLF/LF on manifest files.** Cherry-picks preserve upstream endings, | ||
| but Tier-2 LLM-touched resolutions on `.yml`/`.xml`/`.csproj`/winget | ||
| manifests may downgrade to LF. Re-normalize before staging — see | ||
| [`references/03-conflict-triage.md`](./references/03-conflict-triage.md#line-endings). | ||
| - **PGO pin follows upstream.** `build/pgo/Terminal.PGO.props` hard-pins the | ||
| PGO database to upstream's Windows Terminal `Major.Minor` (our `custom.props` | ||
| stays `0.1`, so it can't be derived). When a sync bumps the upstream version, | ||
| re-pin those two values or the build fails with `Could not find matching PGO | ||
| package`. Step 7's pre-build check covers this. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| | Issue | Solution | | ||
| |---|---| | ||
| | `02-compute-pending.ps1` throws "No 'cherry picked from commit' trailer …" | Fork has never used `cherry-pick -x` yet. Run the one-time [First-time sync](./references/recovery-procedures.md#first-time-sync-seeding-the-watermark). | | ||
| | Stuck issue prevents new run | Resolve on the stuck branch, open + merge a PR (keep the trailer, don't squash), then **close the stuck issue**. The next scheduler tick proceeds. | | ||
| | `03-cherry-pick-one.ps1` returns `"skipped-empty"` | Expected for upstream no-op commits and fork-already-applied patches. The loop skips and continues. | | ||
| | Same file conflicts every run | Add it to [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) with the right strategy (`take-upstream` or `take-ours`; `union` is reserved and currently escalates instead of auto-resolving). | | ||
| | `gh pr create` returns "Head sha can't be blank" | Step 8 retries 3×. On slow networks, the operator may need a manual second run. | | ||
|
yeelam-gordon marked this conversation as resolved.
|
||
|
|
||
| ## References | ||
|
|
||
| - [`references/run-a-sync.md`](./references/run-a-sync.md) — full eight-step procedure with commands. | ||
| - [`references/03-conflict-triage.md`](./references/03-conflict-triage.md) — Tier 0/1/2/3 conflict-resolution rubric. | ||
| - [`references/03-known-conflicts.md`](./references/03-known-conflicts.md) — files with fixed Tier-0 resolutions. | ||
| - [`references/follow-up-pr.md`](./references/follow-up-pr.md) — fix-in-PR vs. follow-up PR rubric and worktree workflow. | ||
| - [`references/recovery-procedures.md`](./references/recovery-procedures.md) — direct-to-main, first-time seed, squash-merge recovery. | ||
| - [`scripts/02-compute-pending.ps1`](./scripts/02-compute-pending.ps1) — derive watermark + pending list (no state file). | ||
| - [`scripts/03-cherry-pick-one.ps1`](./scripts/03-cherry-pick-one.ps1) — cherry-pick one SHA with author/date pinning + Tier-0/Tier-1. | ||
| - [`scripts/04-try-build.ps1`](./scripts/04-try-build.ps1) — run `bz no_clean`; log to `Generated Files/...`. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.