feat(pipeline): add configurable multi-reviewer review panels#356
feat(pipeline): add configurable multi-reviewer review panels#356henrylaih41 wants to merge 10 commits into
Conversation
Let the review step fan out to N reviewers (mixed model families, e.g. codex + claude) reviewing the same diff independently; their findings are merged into one attributed union and the single configured agent reconciles and fixes. With no review.reviewers configured the behavior is byte-identical to today. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…alidation, and gate source rendering
…d-config parse path
…IDs, and merge evidence fields
…tive reviewer; share validation
…onal Reviewers inspect the diff and return findings; they never write the worktree. The shared review CWD is intentional and safe, so we do not isolate or clean up a per-reviewer worktree. A reviewer that writes is a misconfiguration, not a case this code defends against. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
For context: #327 is the previous CR for this feature, and #356 is the improved, gated re-raise of it. The core behavior is the same — opt-in cross-family multi-reviewer panel, attributed-union merge, single-fixer reconciliation, fail-closed default, and re-review-on-fix — with the no-reviewers case staying byte-identical to today. Core differences from #327
On the remaining warningsMost of the flagged issues are edge cases that don't impact core correctness — e.g. same-family reviewer The reason these aren't addressed in this CR: it has been through several rounds of review and has not converged after 8 rounds of loop. We decided to open the PR now since the remaining issues are not critical and not a blocker, and can follow up on them separately. |
|
Test step The The suite was run directly on this branch with the race detector enabled, and all tests pass: CI on this PR will verify this again. |
|
Thanks for the PR! It looks like this branch has a merge conflict with the base branch right now. When you get a chance, could you rebase onto (or merge in) the latest base branch, resolve the conflict, and push? Once GitHub shows the PR as mergeable again, it'll be picked back up for review. Noted for no-mistakes#356 at |
Resolve conflicts between the multi-reviewer review panel and upstream's ordered fallback agent list (kunchenguid#379). Both features coexist: - daemon/manager.go: build the fallback impl-agent list (upstream), then the cross-family reviewer panel (this branch); reviewer-creation failure closes the fallback agent via ag.Close(). - config.go EffectiveRepoConfig: take BOTH Agents (fallback) and Review (panel) from the trusted default-branch copy; force both empty when untrusted. - Semantic conflict: upstream's new RepoConfig.UnmarshalYAML shadow-struct decoder dropped the `review` key, silently nil-ing the panel and disabling trusted reviewer validation. Added Review to repoConfigRaw so the pointer (absent=nil inherit, present=preserved) round-trips as before. - docs: combine fallback-list + reviewer-panel wording in 4 pages. Validated: go build, go vet, gofmt, make lint, and `go test -race ./...` (31 packages) all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address the verified findings of a multi-angle re-review of the panel feature on the merged head (panel + kunchenguid#379 fallback lists): - agent/fanout: recover() on each fan-out goroutine so a reviewer-backend panic fails that reviewer's slot instead of killing the shared daemon (child goroutines bypass the pipeline goroutine's recover). - steps/review: dispatch on the new StepContext.ReviewPanel flag instead of len(reviewers)<=1, so a configured single-reviewer panel (or a panel deduped to one) keeps Source attribution, namespaced IDs, and the audit log; the no-panel default path stays byte-identical. Single-reviewer panels keep live streaming (one fan-out goroutine is race-free). - steps/review_panel + agent/fanout: fail-closed panels (fail_open=false, the default) cancel remaining reviewers on the first error instead of burning the slowest reviewer's full runtime for a discarded result; the step error names the originally-failing reviewer, not a canceled sibling. - daemon/manager: a semantically invalid trusted default-branch review block now fails the run loudly (config.ErrRepoConfigInvalid) instead of silently dropping ALL trusted config; unparseable YAML keeps the legacy warn-and-nil path. Reviewer panel construction is skipped when the review step is skipped, so `rerun --skip review` no longer fails on an unavailable reviewer. - config: ResolveReviewers availability-checks every native reviewer at run start (not just rovodev) after dedup; ReviewerPath/ReviewerArgs delegate to AgentPathFor/AgentArgsFor instead of duplicating them; `agent: auto` reviewers inherit the pipeline agent's ordered fallback chain (explicit families stay pinned); unknown-agent error text now lists copilot; restore the EffectiveRepoConfig security comment's opening line lost in the upstream merge. - tui/review: the findings-row ref budget accounts for the rendered source-tag width, so long reviewer tags no longer overflow the box at narrow widths. - docs/AGENTS.md: trust-boundary field list includes review; document auto-reviewer fallback inheritance. Validated: gofmt, make lint, go test -race ./... (31 pkgs), make e2e. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What Changed
Risk Assessment
Testing
Pipeline
Updates from git push no-mistakes
⏭️ **intent** - skipped
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
internal/pipeline/steps/review_panel.go:37- runReviewPanel fans N reviewer agents out concurrently against one shared worktree (CWD = sctx.WorkDir). The code comment defends this on the data-safety axis ("reviewers are READ-ONLY by contract"), but the distinct operational risk is git-lock contention: real coding agents (claude/codex) routinely run subcommands that take .git/index.lock even for nominally read-only inspection (git status, git add -A, git stash), and two reviewers colliding on the lock surface an error. Under the DEFAULT fail-closed policy (review.fail_open=false) any single reviewer error fails the entire review step (processReviewerResults returns on first Err). Because the panel re-runs on every post-fix re-review, this is a repeated flakiness vector for opted-in repos. Worth confirming the tradeoff is acceptable or whether per-reviewer worktree isolation (or a lock-tolerant default) is wanted; the existing defense addresses corruption, not lock-contention-induced step failure.internal/types/findings.go:346- SeverityRank is newly exported and unit-tested but never referenced in production code. combineReviewerFindings explicitly states it does NO severity-escalation, so the intended consumer never shipped. It's harmless dead code (RiskRank, its sibling, is used); flagging only so it isn't mistaken for wired-up behavior. Either remove it or wire it into the intended severity reconciliation.internal/config/config.go:522-ReviewerArgsonly treats non-emptyargsas a per-reviewer override, soargs: []still inheritsagent_args_override. That makes it impossible to opt a reviewer out of a global model/flag override and can even dedup away an intended same-family reviewer because the effective args collapse to the inherited value. Consider checking whetherspec.Argsis nil rather thanlen(spec.Args) > 0so an explicit empty list means 'no extra args'.internal/pipeline/steps/review_panel.go:96- Panel attribution stamps every finding with onlyAgent.Name(). For the supported case of two same-family reviewers with differentargsorpath, both findings render/log as the same source such ascodex, so the human gate cannot tell which model/config produced which finding. Include the stable slot or an explicit reviewer label in the source/log label when same-family reviewers are distinct.⏭️ **Test** - skipped
Step was skipped.
✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.