From aa139468ba599a67dae3cce65edfed5a8e263b2e Mon Sep 17 00:00:00 2001 From: Manu Date: Fri, 7 Aug 2026 19:47:24 -0600 Subject: [PATCH] chore(spec): reconcile HIVE-267 with what actually shipped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #333. All six [AGENT-DRAFT] markers resolved, and the spec's state brought in line with reality: the A3 mechanism shipped in #290, #294 and #302 (src/hive/_runtime.py, 25 passing tests) and is load-bearing in production — _service._current_layout_exec() resolves through it and ADR-019 builds on it. status: draft -> verifying, NOT archived. Two concrete gates remain, both outside this repo's unit-test reach: - AC4, real non-admin Windows re-validation, has never been performed. The 2026-06-24 hardware pass was the *feasibility spike*, before any code existed; the shipped implementation has only ever run on CI and a Linux dev box. Blocked in practice because that machine is in the orphaned-trampoline state (dotfiles#791). - AC1's `hive --version` clause cannot be verified: no launcher is on PATH. Acceptance criteria are now marked against evidence rather than intent, with [~] for the two that are partially met and why. The load-bearing correction is in tasks.md. That file asserted "The launcher (~/.local/bin) + supervisor already resolve through `current`". The supervisor half is true; the launcher half was false — nothing ever installed one, so a successful upgrade left `current` pointing at a working venv no shell could find. That false claim is why this spec's own "+ own launcher" promise went unnoticed until it resurfaced as #328. `~/.local/bin` is also the wrong directory: ADR-019 chose %LOCALAPPDATA%\hive\bin and forbids hive touching ~/.local/bin at all. Both facts are now recorded at the point of the claim, and the launcher deliverable is explicitly reassigned to HIVE-328 so two specs no longer claim it. The two remaining Risks drafts are resolved from existing evidence rather than new analysis: the supervisor-holds-the-lock risk is the one A3 was chosen to dissolve (the spike repointed `current` under an exclusive FileShare.None lock with no "Access is denied"), and Windows-only scoping follows from POSIX having no in-use-file lock (dotfiles AI-028 AC6, same conclusion as ADR-019) — though that resolves the *scoping*, not the hardware gate, which is called out separately. verification.md now carries real evidence: commit hashes (all verified to resolve), test names, the 25-test suite status replacing "implementation not started (spike only)", and decided promotion candidates — the lesson was already captured in #337, both ADRs are written, and no vault pattern is warranted since the junction-swap technique has not recurred elsewhere. Docs-only; uv.lock deliberately excluded. Master's lock has stale requires-dist for mcp (pyproject says <3.0 since #316, the lock still recorded <2.0), so any uv invocation dirties it. That drift is fixed in #338, not here. --- specs/HIVE-267-upgrade-swap/proposal.md | 126 +++++++++++++++----- specs/HIVE-267-upgrade-swap/tasks.md | 43 +++++-- specs/HIVE-267-upgrade-swap/verification.md | 68 +++++++++-- 3 files changed, 186 insertions(+), 51 deletions(-) diff --git a/specs/HIVE-267-upgrade-swap/proposal.md b/specs/HIVE-267-upgrade-swap/proposal.md index b86404fb..81cd4c7f 100644 --- a/specs/HIVE-267-upgrade-swap/proposal.md +++ b/specs/HIVE-267-upgrade-swap/proposal.md @@ -1,7 +1,7 @@ --- id: "HIVE-267-upgrade-swap" type: spec -status: draft # draft | implementing | verifying | archived +status: verifying # draft | implementing | verifying | archived created: "2026-06-24" issue: "hive#267" # repo#NNN — GitHub issue / Project item that tracks this spec tags: [spec, proposal] @@ -12,10 +12,24 @@ template_version: "1.0" > **Naming**: file lives at `/specs/HIVE-267-upgrade-swap/proposal.md`. -> `[AGENT-DRAFT — review before archive]` — this proposal was drafted from the -> rich content of issue #267. The blocking mechanism decision is **RESOLVED** -> (A3-first, see Risks); the remaining sections are drafted from the issue — -> review them before archive. +> **Reviewed 2026-08-07 (#333); all draft markers resolved.** The A3 mechanism +> shipped in #290, #294 and #302 (`src/hive/_runtime.py`, 25 tests in +> `tests/test_runtime.py`) and is depended on in production by +> `_service._current_layout_exec()` and [ADR-019](../../docs/adr/adr-019-launcher-ownership.md). +> +> **Status is `verifying`, not `archived`, for two concrete reasons** — see +> Acceptance criteria: +> +> 1. **AC1/AC2 cannot be verified end-to-end yet.** They require `hive --version` +> to report the new version from a shell, and no launcher is on `PATH` — this +> spec's `tasks.md` promised "+ own launcher" and it was never built. That gap +> is [#328](https://github.com/mlorentedev/hive/issues/328); its PR1 shipped +> (#330) and ADR-019 settled the ownership question, but PR2 (the launcher +> itself) has not landed. +> 2. **AC4 (real non-admin Windows re-validation) has not been performed.** The +> A3 *feasibility spike* passed on real hardware 2026-06-24, but the shipped +> implementation was never re-validated there, and that box is currently in the +> orphaned-trampoline failure state (dotfiles#791). ## Why @@ -34,16 +48,36 @@ primary vault-access surface. ## What -`[AGENT-DRAFT]` A swap mechanism in hive that lets the install be replaced -**without ever touching in-use files**, so an upgrade applied while the daemon is -running leaves a valid install instead of a locked, malformed one. After this -change, an upgrade-while-running ends with the `hive` entrypoint present and -`hive --version` reporting the new version; a failed swap leaves the previous -working install intact rather than a half-removed directory. +A swap mechanism in hive that lets the install be replaced **without ever +touching in-use files**, so an upgrade applied while the daemon is running leaves +a valid install instead of a locked, malformed one. After this change, an +upgrade-while-running ends with the `hive` entrypoint present and `hive --version` +reporting the new version; a failed swap leaves the previous working install +intact rather than a half-removed directory. + +**Shipped** as `src/hive/_runtime.py` — `runtime_root()`, `versions_dir()`, +`version_path()`, `current_link()`, `_make_junction()`, `repoint()`, +`build_version()`, `current_version()`, `remove_version()`, `latest_version()`, +`_gc_other_versions()`, `self_upgrade()` — plus the `hive self-upgrade [version]` +subcommand (`server._run_self_upgrade`). PRs #290, #294, #302. + +**One clause of the "After this change" sentence is still unmet:** `hive +--version` cannot be invoked from a shell, because nothing installs a launcher on +`PATH`. The layout is correct and `current` resolves to a working venv; it is +simply unreachable by name. Tracked as [#328](https://github.com/mlorentedev/hive/issues/328). ## Out of scope -`[AGENT-DRAFT]` — confirm the boundary, especially the cross-repo split. +**Boundary confirmed 2026-08-07.** The cross-repo split held up in practice: the +hive side shipped independently, and the dotfiles side is tracked separately as +AI-028 (dotfiles#791). One item has since been carved out into its own hive spec +rather than staying here: + +- **The PATH launcher** — promised by this spec's `tasks.md` ("hive owns the layout + … **+ own launcher**") but never built. Now owned by + `specs/HIVE-328-runtime-launcher/` and decided by + [ADR-019](../../docs/adr/adr-019-launcher-ownership.md). Keeping it here would + have left two specs claiming the same deliverable. - The **dotfiles rollout** that triggers the upgrade (`setup-windows.ps1` supervision/upgrade block, `tests/hive-upgrade-timer.bats`, `mcp-servers.json` @@ -96,26 +130,60 @@ working install intact rather than a half-removed directory. `uv tool list` shows a stale/unmanaged entry; documented as expected. (An upstream `uv` issue — replace-while-running on Windows — is filed regardless, per ADR-015.) -- `[AGENT-DRAFT]` **The supervisor holds the lock.** The swap must succeed while - the Startup supervisor keeps `python.exe` running, or coordinate a bounded - stop/restart (exit-75 contract) without a window where the MCP is dead. -- `[AGENT-DRAFT]` **Windows-only, hardware-gated.** Like the other ADR-015 - mechanisms, this needs validation on a real non-admin Windows box, not just CI. +- **RESOLVED — the supervisor holding the lock is a non-issue for A3 (spike, + 2026-06-24; design confirmed at implementation).** This was the risk A3 was + chosen to dissolve rather than manage: junction operations touch only the reparse + point, never the locked target files, so the repoint succeeds *while* the + supervisor keeps `python.exe` running. The spike proved it directly — `current` + was repointed from `versions/1.41.5` to `1.41.6` while an exclusive + `FileShare.None` lock was held on `1.41.5/core.pyd`, with no "Access is denied". + No bounded stop/restart is needed, so there is no window where the MCP is dead. + `self_upgrade` deliberately does **not** restart the daemon; the supervisor's + pre-existing exit-75 restart-on-upgrade contract relaunches `hive serve` through + the freshly repointed `current`. GC of a still-locked old version returns `False` + and defers to the next run rather than failing the upgrade. +- **RESOLVED as scoping, NOT as validation — Windows-only is confirmed; the + hardware re-validation is still owed.** Windows-only is settled and no longer + open: A3 exists solely to work around Windows' in-use-file lock, and POSIX has no + such constraint, so **Linux keeps `uv tool`** (dotfiles AI-028 AC6). ADR-019 + reaches the same conclusion for the launcher. + **But the hardware gate itself is unmet.** What passed on real hardware was the + A3 *feasibility spike*, before any code existed; the shipped `_runtime.py` has + only ever run on CI (Linux/Windows runners) and this Linux dev box. See AC4 — it + is one of the two reasons this spec is `verifying` rather than `archived`, and it + is blocked in practice because the target machine is in the orphaned-trampoline + state (dotfiles#791). ## Acceptance criteria -`[AGENT-DRAFT]` Observable outcomes. Each must be testable. - -- [ ] An upgrade applied **while the daemon is running** leaves a valid install: - the `hive` entrypoint is present and `hive --version` reports the new - version — no malformed/locked state. -- [ ] The documented #267 reproduction (`uv tool install --upgrade` against a - running daemon) no longer fails with "Access is denied / failed to remove - directory" on in-use files. -- [ ] A swap that cannot complete leaves the **previous** working install intact - and surfaces an actionable error — never a silently corrupted, dead MCP. -- [ ] Validated on a real non-admin Windows box (ADR-015 hardware-validation - discipline), not only the CI matrix. +Observable outcomes. Each must be testable. Status reviewed 2026-08-07 — marked +against evidence, not against intent. + +- [~] **AC1 — an upgrade applied while the daemon is running leaves a valid + install:** the `hive` entrypoint is present and `hive --version` reports the + new version, with no malformed/locked state. **Partially met.** The layout + half holds: `build_version()` writes beside the in-use dir and `repoint()` + flips the junction, so the entrypoint exists at + `current/Scripts/hive.exe` — asserted by `tests/test_runtime.py`. The + `hive --version` half **cannot be verified**, because no launcher is on + `PATH` ([#328](https://github.com/mlorentedev/hive/issues/328)). +- [~] **AC2 — the documented #267 reproduction no longer fails on in-use files.** + **Met in mechanism, unverified in the field.** The spike reproduced the exact + failure mode and showed the repoint succeeding under an exclusive lock + (`verification.md`). Not re-run against the shipped code on real hardware — + same gap as AC4. +- [x] **AC3 — a failed swap leaves the previous working install intact and + surfaces an actionable error.** **Met.** Stage-then-flip: the fallible + `_make_junction` runs before `current` is disturbed, and failure raises a + WHY/FIX `RuntimeError`. `build_version()` cleans a half-built dir on failure; + `remove_version()` refuses the active version. Test: + `test_failed_repoint_leaves_the_previous_current_intact`. +- [ ] **AC4 — validated on a real non-admin Windows box** (ADR-015 + hardware-validation discipline), not only the CI matrix. **NOT met.** The + 2026-06-24 real-hardware pass was the *feasibility spike*, before any code + existed. The shipped implementation has run only on CI and a Linux dev box. + Blocked in practice: the target machine is in the orphaned-trampoline state + (dotfiles#791). ## References diff --git a/specs/HIVE-267-upgrade-swap/tasks.md b/specs/HIVE-267-upgrade-swap/tasks.md index 688f3ba0..05ced453 100644 --- a/specs/HIVE-267-upgrade-swap/tasks.md +++ b/specs/HIVE-267-upgrade-swap/tasks.md @@ -6,12 +6,18 @@ created: "2026-06-24" # Tasks - HIVE-267-upgrade-swap > TDD order. One task = one focused commit. Tick as you go. Reorder freely while spec is in `draft` state; freeze once you start `implementing`. +> +> **FROZEN — spec is `verifying` (reviewed 2026-08-07, #333).** The A3 mechanism +> shipped in #290, #294 and #302. Two items remain, both outside this repo's +> unit-test reach: the dotfiles companion trigger, and real-hardware +> re-validation. See `proposal.md` -> Acceptance criteria for what is and is not +> met. ## Setup - [x] Branch created from main: `feat/HIVE-267-upgrade-swap` -- [ ] `proposal.md` is complete and acceptance criteria are testable -- [ ] No open questions left in `proposal.md` "Risks / open questions" +- [x] `proposal.md` is complete and acceptance criteria are testable — reviewed 2026-08-07, marked against evidence +- [x] No open questions left in `proposal.md` "Risks / open questions" — all draft markers resolved 2026-08-07 (#333) ## Implementation @@ -25,7 +31,8 @@ created: "2026-06-24" succeeded on release. Evidence in `verification.md`. **A3 confirmed feasible — proceed; no fallback to A4/A2 needed.** > Interposition = **hive owns the layout** (`%LOCALAPPDATA%\hive\runtime\versions\` -> + `current` junction + own launcher; versions built via `uv venv` + `uv pip +> + `current` junction + own launcher **[moved to HIVE-328 / ADR-019 — never +> built here, see the correction below]**; versions built via `uv venv` + `uv pip > install`; dotfiles upgrade calls `hive self-upgrade`). Likely a new > `src/hive/_runtime.py` (layout/junction/GC) + a `hive self-upgrade` subcommand. @@ -55,6 +62,15 @@ created: "2026-06-24" trigger. The launcher (`~/.local/bin`) + supervisor already resolve through `current` (repoint primitive). Tracked as successor issue #292. Tests in `tests/test_runtime.py` (`self_upgrade` orchestration + CLI wrapper). + **CORRECTION (2026-08-07, #333):** this task claimed "The launcher + (`~/.local/bin`) + supervisor already resolve through `current`". The + supervisor half is true; **the launcher half was false** — nothing ever + installed one, so after a successful upgrade `current` pointed at a + working venv no shell could find. That false claim is why the "+ own + launcher" promise below went unnoticed, and it became + [#328](https://github.com/mlorentedev/hive/issues/328). `~/.local/bin` is + also the wrong directory: [ADR-019](../../docs/adr/adr-019-launcher-ownership.md) + chose `%LOCALAPPDATA%\hive\bin` and forbids hive touching `~/.local/bin`. - [x] **Auto-latest resolution (#292):** make `hive self-upgrade`'s version argument optional — omitted, it resolves the newest `hive-vault` release from PyPI's JSON API (`_runtime.latest_version`, the module's single @@ -66,18 +82,21 @@ created: "2026-06-24" --upgrade` trigger (`setup-windows.ps1` timer / `mcp-servers.json` `prerequisite_command`) with `hive self-upgrade` (now bare — auto-latest landed above). Document the "no-longer-a-uv-tool on Windows" consequence. -- [ ] Real-hardware re-validation: the #267 reproduction no longer reproduces - (manual, recorded in `verification.md`). +- [ ] **Real-hardware re-validation (AC4, gating archive):** the #267 + reproduction no longer reproduces against the *shipped* code (manual, + recorded in `verification.md`). The 2026-06-24 hardware pass covered the + feasibility spike only. Blocked in practice — the target Windows box is in + the orphaned-trampoline state (dotfiles#791). ## Closing -- [ ] Every acceptance criterion from `proposal.md` is covered by at least one test -- [ ] Every acceptance criterion has a matching entry in `features.json` with a non-vacuous verification command -- [ ] Type checks pass -- [ ] Lint passes -- [ ] No unrelated changes in the diff (no scope creep) -- [ ] `verification.md` filled in -- [ ] PR opened referencing this spec folder +- [~] Every acceptance criterion from `proposal.md` is covered by at least one test — AC3 fully; AC1/AC2 at unit level only (the end-to-end half needs #328's launcher); AC4 is inherently manual +- [x] Every acceptance criterion has a matching entry in `features.json` with a non-vacuous verification command — 5 features, all `pending`; the two manual ones say so explicitly +- [x] Type checks pass — `mypy --strict src/` clean +- [x] Lint passes — `ruff check` + `ruff format --check` clean +- [x] No unrelated changes in the diff (no scope creep) +- [x] `verification.md` filled in — evidence recorded 2026-08-07 +- [x] PR opened referencing this spec folder — #290, #294, #302 ## Machine-readable features diff --git a/specs/HIVE-267-upgrade-swap/verification.md b/specs/HIVE-267-upgrade-swap/verification.md index 61b9d970..ff554818 100644 --- a/specs/HIVE-267-upgrade-swap/verification.md +++ b/specs/HIVE-267-upgrade-swap/verification.md @@ -9,14 +9,46 @@ created: "2026-06-24" Map every acceptance criterion from `proposal.md` to concrete proof (commit hash, test name, or observed behavior). -- [ ] Criterion 1 (upgrade-while-running leaves a valid install) -> commit `` / test `` -- [ ] Criterion 2 (#267 reproduction no longer fails on in-use files) -> commit `` / test `` -- [ ] Criterion 3 (failed swap leaves previous install intact + actionable error) -> commit `` / test `` -- [ ] Criterion 4 (validated on real non-admin Windows hardware) -> observed behavior +Recorded 2026-08-07 (#333). Marked against evidence, not intent — see +`proposal.md` -> Acceptance criteria for the same status with reasoning. + +- [~] **Criterion 1** (upgrade-while-running leaves a valid install) -> **partially met.** + Layout half: `8355b9f` (#290) `repoint()` + versioned paths, `6c41c8b` (#294) + `build_version()` / `self_upgrade()`. Tests: `tests/test_runtime.py` (25 passing). + `hive --version` half: **unverifiable** — no launcher on `PATH` + ([#328](https://github.com/mlorentedev/hive/issues/328); PR1 `#330` shipped, + PR2 pending, directory settled by ADR-019). +- [~] **Criterion 2** (#267 reproduction no longer fails on in-use files) -> **met in + mechanism, unverified in the field.** The spike below reproduced the exact + failure mode and showed the repoint succeeding under an exclusive lock. Never + re-run against the shipped code on real hardware — see Criterion 4. +- [x] **Criterion 3** (failed swap leaves previous install intact + actionable error) + -> **met.** `8355b9f` (#290), stage-then-flip: the fallible `_make_junction` + runs before `current` is disturbed; failure raises a WHY/FIX `RuntimeError`. + Test: `test_failed_repoint_leaves_the_previous_current_intact`. Supporting: + `build_version()` cleans a half-built dir on failure; `remove_version()` + refuses the active version and defers a locked one. +- [ ] **Criterion 4** (validated on real non-admin Windows hardware) -> **NOT met.** + The 2026-06-24 hardware pass was the *feasibility spike*, before any code + existed. The shipped implementation has run only on CI (Linux + Windows + runners) and a Linux dev box. Blocked in practice: the target machine is in + the orphaned-trampoline state (dotfiles#791). **This is the gate on archiving + this spec.** ## Test status -- Test suite: not yet — implementation not started (spike only). +- Test suite: **25 passing** in `tests/test_runtime.py` (`uv run pytest + tests/test_runtime.py -q`, verified 2026-08-07). Covers the repoint primitive, + the failed-repoint rollback, `build_version` / `current_version` / + `remove_version`, `self_upgrade` orchestration, `latest_version` (happy / + timeout / bad-payload) and the CLI wrapper omitted-vs-explicit paths. + `mypy --strict src/` and `ruff` clean. +- Shipped in three PRs: **#290** (`8355b9f`, layout + junction repoint), **#294** + (`6c41c8b`, `hive self-upgrade` end-to-end), **#302** (`e0001f9`, auto-latest + from PyPI). All three commit hashes verified to resolve. +- **Production dependency:** `_service._current_layout_exec()` resolves + `current/{Scripts,bin}/hive[.exe]`, and ADR-019 builds the launcher decision on + this layout. The mechanism is load-bearing, not experimental. - **A3 feasibility spike — PASSED** (real non-admin Windows `TDY\MLORENTE`, uv 0.10.5, 2026-06-24; sandbox `scratchpad/a3-spike.ps1`, never touched the real install): 1. Created a `current` junction → `versions/1.41.5` **as non-admin** — read through it OK. 2. Held an **exclusive in-process lock** on `versions/1.41.5/core.pyd` (simulating a loaded `.pyd` / running venv python). Confirmed locked: `Remove-Item` failed with "being used by another process". @@ -41,13 +73,29 @@ Brief log of non-obvious trade-offs or course corrections taken during the work. Before archiving, flag what (if anything) should be promoted to the vault. If all three are "no", archive in repo is the only persistence. -- [ ] Lesson for the repo's `docs/lessons.md`? -- [ ] ADR-worthy decision for the repo's `docs/adr/adr-XXX.md`? Likely yes — the A3/A4 choice updates/closes ADR-015 mechanism (A). -- [ ] New pattern candidate for `00_meta/patterns/`? Only if this recurs in >1 project. +Decided 2026-08-07 (#333). + +- [x] Lesson for the repo's `docs/lessons.md`? **Yes — already captured.** The + 2026-08-07 entry *"A ticket's premise expires; the code moves on and the + instruction does not"* (#337) draws on this spec: its `tasks.md` asserted the + launcher already resolved through `current`, which was false and is why the + "+ own launcher" promise went unnoticed until it became #328. +- [x] ADR-worthy decision? **Yes — two, both written.** The A3 choice is recorded + in [ADR-015](../../docs/adr/adr-015-windows-daemon-supervision-upgrade.md) + mechanism (A), whose AC-3 is closed by #328 PR2 rather than here. The + launcher-ownership question this spec opened is + [ADR-019](../../docs/adr/adr-019-launcher-ownership.md). +- [x] New pattern candidate for `00_meta/patterns/`? **No.** The junction-swap + technique is specific to Windows in-use-file locking and has not recurred in + another project; A3 is documented in ADR-015 where it belongs. ## Archive checklist -- [ ] `proposal.md` frontmatter set to `status: archived` +**Not yet archivable.** Blocked on Criterion 4 (real-hardware re-validation of +the shipped code) and, for AC1's `hive --version` clause, on #328 PR2. Frontmatter +is `verifying` as of 2026-08-07. + +- [ ] `proposal.md` frontmatter set to `status: archived` — currently `verifying` - [ ] Folder moved: `specs/HIVE-267-upgrade-swap/` -> `specs/archive/HIVE-267-upgrade-swap/` - [ ] Bitácora board ticket for this spec moved to Done / closed with PR link (ADR-018) -- [ ] Promotions above executed (if any) +- [x] Promotions above executed — lesson captured in #337; ADR-015 and ADR-019 both written