Skip to content

πŸ”„ sync: top up v5 with latest v4 - #5845

Merged
clubanderson merged 318 commits into
v5from
sync/v4-into-v5-topup-4
Sep 3, 2026
Merged

πŸ”„ sync: top up v5 with latest v4#5845
clubanderson merged 318 commits into
v5from
sync/v4-into-v5-topup-4

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Merges the latest v4 (310 commits since the last topup, PR #5472) into v5, preserving the merge topology so future topups stay small.

Conflict resolutions (74 files)

Policy: keep v5's structure, adopt v4's new functionality.

Validation

  • go build ./... βœ…
  • go vet ./... βœ…
  • go test on all conflict-touched packages (agent, config, dashboard, hub, knowledge, escalation, cmd/hive) βœ…
  • node bin/contributor-relay.test.js β†’ 305/305 βœ…
  • (pre-existing, unrelated: cmd/bd TestResolveDirFallbackCwd fails on macOS due to /private/var symlink; file untouched by this merge)

clubanderson and others added 30 commits September 1, 2026 12:44
…le the nightly, turn the coverage ratchet

The PR gate's wall clock was the single unsliced `test (hub)` job: pkg/hub is
one package of ~2,700 test functions that takes ~125s serially under -race,
so that job ran ~3.3 min while the widest rest bucket ran ~2.4 min and every
agent slice ~2.0 min. Slice pkg/hub across three jobs by test function, exactly
the scheme #4118 introduced for pkg/agent (deterministic round-robin over the
sorted `go test -list` output, anchored -run regex, fail-closed on an empty
list). The coverage job already merges partial profiles by summing per-block
counts, so the hub floor is scored on the union as before. list-packages now
excludes exactly pkg/hub (not its subpackages), so a future subpackage of
either sliced package lands in a rest bucket instead of nowhere.

Every shard now runs `go test -v` with the `=== RUN/PAUSE/CONT` lines stripped
at tee time, and a new .github/scripts/slowest-tests.sh ranks the slice's
slowest top-level test functions into the step summary. Until now the shard
logs carried only the package total, so "what is the gate waiting on" needed a
local reproduction. The reporter never fails a job.

Non-pull_request runs (the hourly schedule, manual dispatch) add -shuffle=on.
Order-dependent tests (#5102, #4795) pass in a fixed order and only fail when
it moves; shuffling the post-merge net finds them and files the issue with the
seed in the log, while the PR gate stays deterministic.

Five coverage floors had drifted far above their recorded values and were no
longer catching a drop: cmd/hive 17β†’35 (measures 37.5), cmd/bd 58β†’75 (78.8),
cmd/hive-backup 68β†’78 (81.9), dashboard 82β†’84 (86.2), github 88β†’89 (91.8),
all re-measured on the 2026-09-01 hourly run and set 2–4 points under. Applied
to both v2-tests.yml and coverage-hourly.yml, which the files say to keep in
sync. docs/development.md updated to match the new shard shape.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
writeMergeEligible decides which PRs the sweep may merge and which go to the
fix loop. Every exclusion branch in it is annotated with a dated production
incident β€” three green PRs frozen for hours on 2026-08-04, sixteen dependabot
PRs accumulating for eleven days on 2026-08-28, DIRTY go.mod bumps pinning the
eligible count on 2026-08-31 β€” and a dozen fix PRs have patched it, yet the
only test exercised the intent-verdict branch. A patch to one branch could
move a PR between buckets on another and nothing would notice.

Add a truth table over the single-PR decisions (draft, hold, red with and
without a required-check set, optional-only red with and without GitHub's
mergeable verdict, pending Γ— each mergeability, green Γ— conflicting/unknown),
plus focused tests for the evidence a ci_failing entry carries to the fix
agent (check names, excerpt, head SHA, the escalation flag with a positive
control), the fields an eligible entry carries to the merge step (org-qualified
repo, head SHA, tri-state mergeability spelled out, DCO from labels), the
fail-closed review-approval gate (no artifact β†’ nothing eligible; approval at
a moved head β†’ excluded; positive control without the gate), and that both
files are rewritten on an empty cycle so no consumer reads a stale bucket.

cmd/hive runs in a rest bucket at ~10s; these tests do no I/O beyond a
TempDir and add no wall clock the gate can notice.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
handleGovernorConfigGet (src/pkg/dashboard/api.go) feeds every tab of the
dashboard Settings UI and has taken 16 fix commits over time, but only its
litellm and hub-namespace sections had tests pinning their shape. A change
to any other section's field names, polarity, or masking could regress
silently.

This adds governor_config_get_contract_test.go with focused tests, each
carrying a positive assertion (the right value appears) and a negative one
(the wrong/raw value does not), so a handler returning the wrong thing
cannot pass silently:

- TestHandleGovernorConfigGet_AgentsListsConfiguredNamesOnly: agents lists
  every configured agent name and nothing else.
- TestHandleGovernorConfigGet_ThresholdsExcludeIdleAndEffectiveThresholdsMatchConfig:
  thresholds carries the configured non-idle modes and excludes idle even
  when cfg.Governor.Modes has an idle entry; effectiveThresholds has exactly
  quiet/busy/surge, each equal to cfg.Governor.EffectiveThreshold(mode,
  repoCount); repoCount matches cfg.Project.RepoCount().
- TestHandleGovernorConfigGet_ReposOrgQualifiesBareNamesOnly /
  TestHandleGovernorConfigGet_PrimaryRepoAlreadyQualifiedUnchangedAndEmptyStaysEmpty:
  repos and primaryRepo org-qualify bare names but never double-prefix an
  already-qualified name, and primaryRepo stays empty when unset.
- TestHandleGovernorConfigGet_NotificationsEmptyWhenUnconfigured /
  TestHandleGovernorConfigGet_DiscordWebhookMaskedNeverRaw /
  TestHandleGovernorConfigGet_NtfyConfiguredReflectsServerAndTopic: hasNtfy/
  hasDiscord and the string fields reflect configuration state, and the raw
  Discord webhook never appears in the response body β€” only maskSecret's
  output does.
- TestHandleGovernorConfigGet_LabelPolaritySplit: requireLabels mirrors
  cfg.Project.IssueFilter.RequireLabels, labels mirrors
  cfg.Governor.Labels.Exempt (opposite polarity, asserted not-equal), and
  holdLabels equals github.HoldLabels.
- TestHandleGovernorConfigGet_TopLevelSectionsPresent: decodes into
  map[string]any and checks presence of the top-level keys the UI depends
  on (agents, thresholds, effectiveThresholds, repos, primaryRepo,
  notifications, budget, health, sensing, logging, litellm, review,
  auto_merge, hub, attribution).

Test-only change: no CHANGELOG.md entry per repo policy. Uses t.Setenv
where env is touched (none in this file, kept for future sections), no
time.Sleep, no network calls.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
πŸ› test: revert the #5388 demo regression that reached v4, and land the guard fixes
bin/enumerate-actionable.sh writes the actionable.json that merge-gate.sh,
kick-agents.sh and the dashboard contribute lane (contribute_ws.go) read
instead of running their own gh queries. It has been patched by fix PRs
(#245, #260, #274, #394, #398, #4772) and had no tests.

bin/test_enumerate_actionable.sh EXECUTES a path-rewritten copy (the script
hardcodes /usr/bin/gh, /var/run/hive-metrics and /var/log/kick-agents.log)
against a stub gh that serves canned REST/GraphQL fixtures and runs the
script's own --jq filters through real jq. Every exclusion has a positive
control that IS enumerated. Contracts pinned:

  1. Issue label exclusions: hold substring (case-insensitive), blocked
     (#4772), do-not-merge, auto-qa-tuning-report, LFX* prefix; cross-repo
     created_at ordering; count == len(items).
  2. Pull requests returned by the issues API are dropped
     (.pull_request == null).
  3. PR exclusions: draft, hold, blocked; the per-PR files endpoint is hit
     only for survivors (#394). do-not-merge PRs are pinned as currently
     NOT excluded (header/code mismatch).
  4. ADOPTERS exclusion via the file list, case-insensitive.
  5. Output shape: top-level/issues/prs/hold keys, item keys (body and
     author_type stripped, age_minutes added), org/repo qualification,
     flattened labels/assignees, primary-repo-only sla_violations,
     hold.items/total accounting, generated_at.
  6. External-issue SHA hold: withheld, hold added + kind/bug removed +
     one comment, exactly once (marker); Bot / PROJECT_AI_AUTHOR /
     with-SHA / non-primary-repo issues kept. The GraphQL re-check is
     pinned at its CURRENT behaviour: xargs -I {} rewrites the python {}
     literals inside the bash -c script, so the child always reports
     skip and unhold never fires (documented bug, not fixed here).
  7. Partial API failure: MAX_RETRIES attempts, failed repo contributes
     nothing, healthy repo still published, WARN logged.
  8. Total API failure: exit 0, previous actionable.json preserved,
     no temp file left behind.
  9. Empty result publishes an empty document; empty PROJECT_REPOS
     fails closed with exit 1 and touches nothing.

Wired into v2-ci.yml build-and-test right after the gh-wrapper gate step.
Hermetic: no network, retry sleep stubbed, never touches /var/run, /data
or /tmp/hive.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
πŸ› fix(config): warn on absurdly small budget limits at load, reject them on save
StartLatestSHAPoller's ticker loop body was only reachable by waiting on
a live 2-minute ticker, so the existing pre-loop test
(TestStartLatestSHAPollerPreLoop) never exercised it. Extract the loop
body into pollLatestSHAsTick(ctx, now) and have the loop call it; add
sha_poller_tick_test.go covering:

- a changed SHA is persisted and an unchanged SHA is not re-persisted
- the throttled reconciliation lanes (*IfDue) run each tick, observed
  via their own s.last<Lane> throttle timestamps
- the hub auto-upgrade check runs every tick, not only when the SHA
  just changed (the regression the inline comment documents), observed
  by stubbing the existing hubImageExists test seam and counting calls
  across two ticks with an unchanged upstream SHA

Behavior-preserving: the tick body is unchanged, only lifted into its
own method with now injected the same way maybeSnapshotImagePulls
already takes it.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…ppression

πŸ› fix(governor): stop advisory digests repeat-posting to the same issue
🌱 ci(tests): slice pkg/hub 3-way, rank slowest tests per shard, shuffle the nightly, turn the coverage ratchet
…no-undef

node --check only parses syntax, so a call to a helper that was never
defined (apiFetch is not defined, #5516) sailed through dashboard-lint and
shipped in a release. Add a second pass to check-inline-js.js: concatenate
every inline <script> block of a file (classic scripts share one global
scope) and run ESLint with ONLY the no-undef rule, browser globals,
ecmaVersion 2022, sourceType script.

- Findings are reported at the HTML line/column, not the extracted-block
  line, so the operator can jump straight to the call site.
- A reference guarded on its own line by `typeof NAME === 'function'` is
  exempted (the dashboard's optional-navigation hooks use this pattern);
  an unguarded call to the same name elsewhere still fails. Neither HTML
  file has an external <script src> or dynamic loader, so the extra-globals
  allowlist is empty.
- The Linter cwd is anchored at the HTML file's directory so any path
  (repo-relative in CI, absolute locally) lints; a flat-config
  no-match fatal cannot masquerade as a clean pass.
- If eslint is not installed the checker FAILS with instructions rather
  than silently skipping the pass (#5388 was a checker that quietly ran
  nothing).

eslint 10.9.1 and globals 17.12.0 are exact-pinned in a CI-only
.github/scripts/package.json with a lockfile (npmjs.org only, integrity
hashes); the workflow runs npm ci --ignore-scripts there with setup-node
22 and npm cache. dashboard/ and src/proxy/ package.json are runtime
trees, so a lint tool does not belong in either. The path filter now
covers .github/scripts/** so an edit to the pins or lockfile runs the job
it changes.

Calibration: 0 findings on both v4 HTML files; 1 finding (apiFetch at the
injected HTML line) on a scratch copy with the #5516 call reintroduced.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Found by the new no-undef pass on its first run against v4. The shipped
dashboard already replaced this call with refreshStatus() (guarded by
src/pkg/dashboard/budget_reset_ui_test.go), but dashboard/index.html kept
the stale callee, so toggling 'ignore budget' threw ReferenceError after
the POST succeeded. Mirror the shipped fix.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
πŸ§ͺ test(hive): pin every bucket decision in writeMergeEligible
Behaviour-preserving refactor: move the work-source overlay, resume-kick
merge, kickable-agent filter, provider-budget kick gate, SLA-breach page
selection, and advisory-post error classification out of runEvalCycle into
eval_cycle_seams.go as pure functions, and add table tests for each citing
the incident it guards (#4731/#4975, #2627, #815/#2573, #4294, 915bb96,
#1699/#2353). Call sites, ordering, log messages and error semantics are
unchanged; main() is untouched.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…ntract

πŸ§ͺ test(dashboard): pin the /api/config/governor response contract
…cal path

Both image jobs in v2-ci.yml carried needs: build-and-test although
neither consumes anything that job produces β€” each compiles the PR's own
tree from scratch inside src/Dockerfile. The dependency serialised the
test gate's wall time plus a second trip through the shared runner queue
onto every PR's time-to-green. Measured on 2026-09-01 across three PR
runs: ~2.75 min of build-and-test, then 10-11 min queued before either
image job started, then 3.5-4.5 min of build. With the edge removed all
three jobs start at t=0 and the guard (~4.5 min) becomes the ceiling
rather than the sum. nightly-release still requires both build-and-test
and docker, so the v2 nightly gate is unchanged.

The guard's buildx step also gains a read-only cache-from on the
tmux-linux-amd64 gha scope that docker.yml already warms on every PR and
v4 push. That scope only ever receives --target tmux-builder exports, so
no layer carrying /usr/local/bin/hive can be served from it and the
stale-COPY hazard behind the stale-binary guard cannot arise; every other
stage is still built from scratch on the PR's own commit. Measured: the
runtime stage blocks on tmux-builder (~51s) at its third layer while its
own apt layer takes ~20s, so a hit removes ~30s of the ~207s build. A
whole-image cache-to was rejected on docker.yml's own measurement (130-225s
export per job for ~0 reuse) and because it reintroduces the #3816
stale-layer path. What the guard asserts is untouched.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
NewHubServer unconditionally reads/writes the master secret at
/data/saas/hub-secret.key, reads nine other /data paths, and starts the
debounced registry save loop. Only ~6 of ~33 test files that construct a
server ever called StopSaveLoop, so on a hive host the loop outlives its
test and races t.TempDir cleanup (#4774), and every server reads the real
/data.

- Introduce the hubSecretPath package var (default unchanged) and use it
  in NewHubServer and provisionMasterSecret, with MkdirAll on its dir.
- Add newHubServerForTest(t, opts...): per-test t.TempDir, redirects the
  construction-time path vars for the test's lifetime, registers
  StopSaveLoop so the loop is joined before the dir is removed, and
  refuses t.Parallel via t.Setenv. Per-test rather than a shared TestMain
  dir: a shared dir would make tests inherit each other's registry and
  secret, which today's failing /data writes accidentally prevent.
- Migrate 29 test files (344 call sites) mechanically; assertions
  unchanged.
- Scrub OPENAI_API_KEY, CODEX_API_KEY, ANTHROPIC_API_KEY, GITHUB_TOKEN,
  GH_TOKEN and HIVE_HUB_SECRET in TestMain.
- Pin the helper's invariants: distinct paths per test, secret under the
  TempDir, save loop joined and dir removed after cleanup.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
… unit tests

Nine of the last 21 flaky-test tickets were timing margins, and every fix
added another local waitFor* helper (16 across dashboard/hub/github/
watchdog/notify) next to 210 time.Sleep calls in *_test.go. This adds one
shared poll helper and a contract test that stops the count from growing.

internal/testutil:
- Eventually(t, timeout, cond, msg, args...) polls at 10ms, t.Helper(),
  Fatalf on deadline; EventuallyValue[T] returns the observed value. The
  doc comment says when a sleep is still right (negative waits) and when a
  Go 1.25 testing/synctest bubble is the better tool.
- eventually_test.go: positive case plus a negative case through a
  recording testing.TB double (no real failure).
- sleep_ratchet_test.go: walks src/pkg and src/cmd *_test.go, counts
  time.Sleep( occurrences, fails above sleepBaseline=198 with a message
  pointing at testutil.Eventually and at the files with the most sleeps.

Migrated fixed sleeps >= 500ms (assertion semantics unchanged):
- dashboard contribute_ws_{token_refresh x2,credential_accept,maybe_refresh}:
  2-3s "keep handler alive" sleeps -> handlerDone channel closed at cleanup
- proxy_deep8 NonGitHubWithTCPServer: 3s -> wait on handler done, 3s bound
- snapshot builder_test x2: 1s -> poll for the next second boundary
- hub_heartbeat StartHeartbeatWithCancel: 500ms -> first heartbeat, 500ms bound
- discord bot_test x4: 5.1-10.5s -> Eventually on the fetch/send counters,
  bounded at 2-3 ticker periods

Left in place, deliberately: agent tmux_coverage cleanupAgent (5s drain of
non-ctx-aware launch goroutines), agent kick_restart_recovery (4s poller
drain), config watcher_test Debounce (1s negative wait spanning the 500ms
debounce window), agent poll_coverage (3.5s inject-between-ticks
choreography).

CI: ./internal/... added to the rest-bucket go list and to the hourly
coverage runs so the new package and the ratchet actually run.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…ontract

πŸ§ͺ test(bin): contract test for enumerate-actionable.sh
src/deploy/ suites are wired individually, by name, in a workflow step. A new
suite is therefore never picked up automatically, and an omission produces no
failure β€” nothing fails when a test file is simply never named. The file looks
maintained and CI is green because it never ran. That is the property #4363
calls out for bin/, with no equivalent here until now.

The gap was live, not hypothetical: of 30 src/deploy/test_*.sh suites,
test_entrypoint_symlinks.sh was referenced by NO workflow at all. Two earlier
instances (test_entrypoint_system_gitconfig.sh, test_hive_snapshot_unit_contract.sh)
were each caught by accident rather than by a gate.

Adds src/deploy/test_deploy_suites_wired.sh, the src/deploy/ sibling of
bin/test_bin_suites_wired.sh. The bar is "referenced anywhere under
.github/workflows/", not "referenced in v2-ci.yml": four suites live correctly
in specialised lanes (suid-contract.yml, docker.yml, quadlet-gate.yml) because
they need a runtime, an image build, or a generator those workflows stand up,
and demanding v2-ci.yml would fail all four for being in the right place.

Wires test_entrypoint_symlinks.sh into v2-ci.yml rather than deleting it: it
passes 8/0 and guards the Copilot token-persistence symlinks, a recurring
incident surface where a lost link drops the CLI's stored token on every
restart and presents as an agent that silently stops authenticating. It reads
entrypoint.sh and bin/agent-launch.sh from the checkout, so it needs no runtime.

The guard registers itself in v2-ci.yml, which is what makes it pass its own
check β€” exempting itself would be the hole it exists to close. It refuses to
report success when the workflow directory or the suite glob matches nothing,
since a checker aimed at an empty set exits 0 indistinguishably from "all
wired" (#5388). test_lib.sh is named as a helper rather than a suite: it is
sourced, not run, and satisfied a naive reference check only because a v2-ci.yml
comment happened to mention it.

Fixes #5529
Refs #4363, #5388, #5504

Signed-off-by: Andy Anderson <andy@clubanderson.com>
TestPollLatestSHAsTick_HubAutoUpgradeCheckedEveryTick fed the fake GitHub a
branch SHA of "newhubsha00000", which rolloutHubToSHA's validateImageTag
rejects before the hubImageExists call the test counts β€” so the test failed
for a reason unrelated to what it asserts. Use a 40-hex SHA so the tag
validator passes and the auto-upgrade decision is actually exercised.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…e-check

enumerate-actionable.sh's SHA re-check fan-out piped each marker
through 'xargs -I {} bash -c ...  _ {}'. xargs -I rewrites every {}
in the command string, including the python dict literals inside the
quoted script body (d.get("data",{}), (issue.get("author") or {}),
issue.get("comments",{})), so every child hit a SyntaxError that was
silently swallowed by a trailing '|| echo "skip"'.

Switch to 'xargs -n 1 bash -c ... _' so each marker arrives as $1
with no text substitution into the script body. Also stop masking
both failure paths (a failing gh api graphql call, and a python
parse failure) behind a bare 'skip' β€” both now log the actual error
to stderr.

The ADOPTERS batch at line ~208 uses the same xargs -I {} bash -c
shape but has no {} literal in its body, so it is unaffected and
left as-is to keep this diff targeted at the actual bug.

Flips bin/test_enumerate_actionable.sh's pinned-bug assertions (from
PR #5527) to prove the fix: the SHA re-check now actually unholds an
issue once its reporter supplies a commit SHA, and a new negative
control (#112, SHA never supplied) proves the hold correctly survives
the re-check when no SHA is present.

Fixes #5528

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
The terminal dashboard refreshes itself on a timer, and slows that timer
down while its live event stream is healthy β€” the stream is doing the work,
so polling hard would be wasted requests. That was correct when the timer
only fetched things the stream also carries. T30 and T31 then hung token
counts, estimated cost and the audit feed off the same timer, and the stream
carries none of those: a connected stream stretched them from 5s to 60s, so
the Tokens and Events panes went twelve times staler at exactly the moment
the header started reporting `ws: connected`.

Split the one timer into two classes. The reconciliation loop (/api/agents,
/api/status, /api/config/governor, /api/hive-id) keeps the existing
behaviour: 5s while the stream is down, 60s while it is healthy, an
immediate fallback fetch on drop, and generation-based retirement of the
superseded chain. The activity loop (/api/tokens, /api/cost, /api/audit)
runs at 5s unconditionally, with its own message type, interval and
generation.

The separate generation counter is load-bearing rather than symmetry: the
drop path bumps the reconcile generation to retire the stretched chain and
re-arms a reconcile chain only, so a shared counter would retire the
in-flight activity tick with nothing to replace it β€” the Tokens and Events
panes would go dark for the rest of the session at the first stream blip.

The drop path's immediate fallback is now pollReconcile rather than the full
poll, since the activity loop never stretched and is already mid-interval.
poll() itself still issues all seven reads and stays the one-shot refresh
used by Init and by every action handler, where a write moves the roster and
appends to the audit log at once.

No cadence values changed, and no client or pane code was touched.

Signed-off-by: Douglas Baggett <doug.baggett@gmail.com>
Operators seeing the dashboard warning 'no enabled review-capable agents were
detected' had no documentation explaining what makes an agent review-capable.

Add a 'Reviewer selection' section documenting the two paths (explicit
reviewer_agents list vs. keyword scan), which agent fields are searched,
and minimal config examples for both approaches.

Source function: dashboardAgentReviewCapable in src/pkg/dashboard/status_builder.go

Refs #5542

Signed-off-by: sec-check <sec-check@hive.kubestellar.io>
retro.analysis_model requires a configured governor.litellm gateway but the
doc gave no path from intent to configuration. Add a Prerequisites subsection
explaining how to configure the LiteLLM gateway (env vars, dashboard API,
test endpoint) and what value to supply for analysis_model.

Refs #5543

Signed-off-by: sec-check <sec-check@hive.kubestellar.io>
Adds bin/test_kick_governor.sh, a behavioural contract test for
bin/kick-governor.sh (923 lines, called from bin/hive.sh, run every
15 min via kick-governor.timer), and wires it into v2-ci.yml's
build-and-test job.

See the PR body for the full contract list and the bugs pinned.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
ci(dashboard-lint): catch calls to undefined identifiers with ESLint no-undef
ci: take docker and overlayfs-exec-guard off the build-and-test critical path
… agy agent

Two defects that compounded into "the login just fails", with no error
anywhere naming a cause.

1. /data/home/.gemini IS GROUP READ-ONLY AND IN NO PERM GUARD. It appears in
none of the five places entrypoint.sh manages CLI credential dirs β€” the
mkdir, the chmod/chown, the NEED_PERM_FIX sample, the inotify guard, and the
polling repair. So agy creates it itself and nothing ever fixes it. Measured
on a live hive: .claude 2775, .copilot 2770, .gemini 2750 β€” and `touch` into
it as the agent uid returns Permission denied. agy persists its session to
$HOME/.gemini/antigravity-cli/antigravity-oauth-token, so every sign-in
succeeded in-process and evaporated on exit; `agy models` in any new process
reported "Please sign in" while the running agent looked authenticated. One
chmod produced the token file immediately and it survived a restart.

.gemini is added to all five sites at 2770 β€” it holds an OAuth token, so it
follows .copilot/.bob in keeping world off it rather than .codex's 2775.

The ongoing guard matters as much as the create-time mode: agy writes the
token 0600 owned by whichever agent signed in, so without the guard every
agy agent needs its own login and each re-auth locks the others out. That is
the same shape as copilot's config.json, which the guard already re-opens.

2. THE TOKEN-RESTART HEAL IS BACKEND-BLIND. configHasTokens() answers true
when EITHER the shared claude or copilot credential is usable, whatever
backend the agent in front of it runs. So an agy agent parked at its Google
OAuth prompt was restarted because an unrelated CLAUDE login was valid. A
restart cannot mint a Google session, so it looped β€” and each relaunch minted
a fresh PKCE challenge, invalidating the code the operator was mid-way
through pasting. Observed: 15 restarts in an hour, 8 distinct challenges, and
an operator seeing oauth2 "invalid_grant" "Malformed auth code." who
reasonably concluded the code had been mangled in transit. It had not; the
challenge no longer existed.

The gate is now AgentHasValidCredential, which resolves the agent's OWN
backend and is positive-evidence-only, so a backend this process cannot
verify (agy, gemini) answers false and is left alone. That is the correct
answer: do not restart when you cannot show a restart would help. The heal's
documented case β€” a claude agent with a usable shared credential β€” is
unchanged and pinned by a test.

This only became reachable when #5494 made agy's login prompt visible to the
detector; before that agy was invisible and idle, so the heal never fired.

Fixes #5551

Signed-off-by: Douglas Baggett <doug.baggett@gmail.com>
πŸ§ͺ test(governor): extract six testable seams from runEvalCycle
test(hub): per-test HubServer construction with isolated /data footprint
clubanderson and others added 10 commits September 2, 2026 23:39
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes #5775

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes #5779

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update the v4 Go module and repository references from github.com/kubestellar/hive to github.com/hivecommons/hive.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update repository and GHCR references from kubestellar/hive to hivecommons/hive across operational configs, docs, workflows, and tests.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ansfer as one (#5821)

The kubestellar to hivecommons migration took out every agent branch push
in the fleet (#5774). Issue creation kept working, so the hive watcher
looked alive; every PR flow β€” strategist, quality, architect, scanner, and
any human relying on App-authored pushes β€” was dead. Installing and
permissioning the App on the new org is an org-side action and is not
what this change is. What this change fixes is that NOTHING in the
credential surface described the failure, and what it did describe was
about to be wrong in the other direction.

THE WRITE PATH WAS NEVER OBSERVED.

DiagnoseAppAuth classified solely on the Issues permission. It records
Actions and Commit-statuses for the optional Visual Hive App (#4030), and
it never read Contents, Pull requests, or Workflows at all β€” the three
grants an agent PR flow actually runs on: push the branch, open the PR,
and (when the diff touches .github/workflows/**) update a workflow file.
So a hive that could file issues and could not push a single branch
reported exactly what a healthy hive reported: "ok".

All three are now recorded on the same API call that already fetched the
installation, exposed as GrantsAgentPushFlow()/PushFlowGrants(), and
logged on EVERY credential verdict including AppStateOK β€” gating them on
a fault would never emit them for the case that motivated them, an
installation that looks healthy and cannot push.

They are RECORDED, NEVER ENFORCED, following the #4030 precedent for the
same reason. The read-only advisory tier holds Contents: read by design
and never pushes anything; requiring these would flip it to
AppStateInsufficientPerms for operating exactly as intended. And an
installation that holds both grants can still be unable to push for a
reason no permission bit describes, which is what the repo-scope states
exist to say.

AN ORG TRANSFER WAS ABOUT TO BE MISDIAGNOSED.

The deterministic coverage check (#4360) compares configured repos
against GET /installation/repositories and reports AppStateRepoNotCovered
for the difference, with the remedy "tick the repo in the installation's
repository access". That is right for the case it shipped for. It is
impossible for a repository that has been TRANSFERRED: once the App is
installed on the new org while a hive's config still names the old one,
every configured repo reads as not-covered and the banner points at a
settings page for an account the repository has left. Nothing can be
ticked there. Issuing that instruction costs an operator precisely the
debugging time this check was written to give back β€” the same class of
confident misdirection that made #4360 necessary.

InstallationCoverage.MovedTo classifies that shape, from data the
coverage listing already fetched β€” no new API call, no new permission, no
new credential. The rule is deliberately narrow and all three clauses are
required:

  1. the listing is complete (a truncated set proves no absence β€” the
     rule Missing already follows);
  2. the installation covers NOTHING under the configured owner (a
     transfer moves the whole repository out of that account; if other
     repos there are still covered, the account is reachable and a single
     miss is an ordinary scope gap, which is Missing's story);
  3. exactly ONE covered repository carries the configured repo's name
     (two accounts owning a repo called "tools" is unremarkable, and
     picking one would be a coin flip presented as a diagnosis).

Anything less returns nothing and the not-covered verdict stands.

The new AppStateRepoMoved names where each repository lives now, states
plainly that the App, its key and the installation are all healthy, and
never carries the not-covered remedy. It is user-actionable (repoint the
hive's configured org) and never operator-actionable: no key upload can
help, and claiming otherwise is exactly the misattribution this family of
states exists to prevent.

Hub-side, "repo-moved" is added to appCanWriteForAdvisory's non-writable
set, because clause 2 means no repo this hive is pointed at β€” the
advisory repo included β€” is writable, so a stale digest is a consequence
and not a fault. "repo-not-covered" is deliberately left OUT of that set
and a test now pins the asymmetry, so the omission does not read as an
oversight and get "fixed" into a false negative: that state fires when
ANY configured repo is unticked, which need not be the advisory one.

Tests assert the verdict that actually changes, not field presence. The
transfer case asserts BOTH that Missing still reports the repo (so the
old verdict genuinely would have fired) and that MovedTo reports where it
went; every ambiguity clause is pinned individually so a dropped guard
names itself; the copy test forbids the not-covered remedy and requires
the credential exoneration; and the grants table asserts the
classification is UNCHANGED while the push-flow answer moves.

Signed-off-by: Danathar <doug.baggett@gmail.com>
Merge origin/v4 into v5. Conflict resolution policy: keep v5's structure
(split files, seams, edge channel), adopt v4's new functionality.

Highlights:
- workflows: v4's owner-derived dual-publish image names + cross-org
  mirror, with v5's RELEASE_BRANCH=v5 / CHANNELS=edge
- pkg/config: ported v4's pause ownership, kick-only channels +
  ValidateChannels, cadence owners, budget floor warnings, curator
  opt-in into v5's split config files
- pkg/agent: kept v5 terminal seam (removed v4 terminal.go); ported
  v4 thrash breaker, credential probe recovery, coverage preamble
- pkg/dashboard: ported v4 branding (#5752) into webstatic
  (InjectBranding + branded CSP index); kept v5 seams
  (SchedulerControl, deps.Watsonx, openrouter alias); v4 crash-safe
  lease persist (#5625); union of capability consts
- pkg/hub: ported v4's pollLatestSHAsTick extraction into
  saas_sha_poller.go
- CHANGELOG: per-subsection union of Unreleased + v4 release sections

Validation: go build ./..., go vet ./..., go test on all touched
packages, contributor-relay JS tests 305/305.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Sep 3, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 3, 2026
Signed-off-by: Andy Anderson <andy@clubanderson.com>

# Conflicts:
#	src/docs/README.md
#	src/docs/hivecommons-migration.md
#	src/pkg/convergence/mutation/ledger.go
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2026
clubanderson and others added 3 commits September 3, 2026 09:14
… turnEnvelopePath

- changelog-fragment-guard.yml / testutil-guard.yml: release-lines-in-sync
  requires every line in .github/release-lines.yml to appear in hand-written
  branch lists; add v5.
- contribute_turn.go: remove unused turnEnvelopePath (turn.FileStore owns
  envelope paths); flagged by golangci-lint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2026
Signed-off-by: Andy Anderson <andy@clubanderson.com>

# Conflicts:
#	.github/workflows/v2-tests.yml
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2026
Signed-off-by: Andy Anderson <andy@clubanderson.com>

# Conflicts:
#	src/docs/env-vars.md
@clubanderson
clubanderson merged commit 11a14c5 into v5 Sep 3, 2026
53 of 56 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the sync/v4-into-v5-topup-4 branch September 3, 2026 15:10
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

Check out what's new:

Stay connected: Slack #kubestellar-dev | Multi-Cluster Survey

clubanderson added a commit that referenced this pull request Sep 5, 2026
…nc-policy.md) (#6052)

* [strategist] planning: document v4β†’v5 forward-port sync policy

Adds src/docs/v5-sync-policy.md proposing cadence (v4 minor release or
50-commit drift), single-owner rotation, the merge-commit top-up
procedure already used by #5845/#6046, DCO expectations, and explicit
PR-body and review contracts for sync PRs. Links it from the docs index.

Refs #6051

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: sec-check <sec-check@hive.kubestellar.io>

* πŸ“– document active v5 sync policy

Align the v4-to-v5 sync policy with the current merge-commit top-up process, weekly/release cadence, maintainer ownership, verification checklist, and zero-delta check.

Closes #6051

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>

---------

Signed-off-by: sec-check <sec-check@hive.kubestellar.io>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: sec-check <sec-check@hive.kubestellar.io>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Andy Anderson <andy@clubanderson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants