Triage should suppress ready-to-code when issue proposes multiple approaches without clear priority

[fullsend-ai-retro]

What happened

Issue #626 proposed three potential solutions of varying complexity: (1) reviewer-facing documentation in CONTRIBUTING.md, (2) warning annotations in check-e2e-authorization.sh, and (3) making the functional-tests-complete consolidate job fail when tests are skipped. The issue phrased these as "Some combination of" without indicating which was primary. The triage agent applied ready-to-code and the code agent auto-dispatched within ~5 minutes. It implemented options 1 and 2 (the simpler, agent-implementable changes) and created PR #628. The review agent then spent 14 minutes reviewing. The maintainer then closed the PR without merging, stating that option 3 (the structural CI fix) was the right approach — the palliative measures alone were not valuable. Total wasted effort: ~6 min code agent + ~14 min review agent + human review time.

What could go better

The triage agent should detect when an issue proposes multiple candidate approaches with significantly different scope or complexity and no clear priority signal from the author. In such cases, auto-dispatching the code agent risks the agent picking the easiest approach rather than the most impactful one. This is a specific instantiation of a broader pattern: existing issues #605 (human-approval gate) and #259 (code agent surfacing approach) address related aspects but from different stages. #605 proposes a blanket configuration gate on all auto-dispatches. #259 proposes the code agent post a comment about its chosen approach. Neither addresses the triage agent's ability to internally detect multi-approach ambiguity and conditionally suppress dispatch. Confidence is medium-high: the issue's phrasing ('Some combination of') with three options of varying complexity was a clear signal of ambiguity that the triage could have detected. The uncertainty is whether a heuristic for detecting multi-approach issues would be reliable enough to avoid false positives on issues where the approach is actually clear despite listing alternatives.

Proposed change

In the triage agent definition (agents/triage.md) or the triage skill, add guidance for multi-approach detection. When the issue body contains multiple proposed solutions (heuristic: numbered lists of approaches, 'Option 1/2', 'Approach A/B', 'Some combination of', or 'Either X or Y'), the triage agent should assess whether there is a clear primary approach. If the approaches have significantly different complexity or scope (e.g., documentation vs. CI workflow changes), and the issue does not indicate which is preferred, the triage should: (a) apply triaged instead of ready-to-code, (b) note in its triage comment which approach it recommends and why, and (c) invite the issue author to confirm the approach before code dispatch. This complements #605 (which proposes a blanket gate) by making the triage agent's own dispatch decision smarter without requiring a global configuration change. It also complements #259 (which operates at the code agent level) by preventing the wasted code+review cycle entirely rather than surfacing the choice after work has begun.

Validation criteria

Over the next 10 issues where the body proposes 2+ distinct solution approaches of different complexity, the triage agent should either: (a) identify a clear primary approach and recommend it in its triage comment, applying ready-to-code with rationale, or (b) apply triaged instead of ready-to-code and note that the approach needs human confirmation. Success metric: zero code agent PRs closed by humans for 'wrong approach' on multi-approach issues within 30 days of the change. Secondary metric: no increase in triage-to-code latency on single-approach issues (the heuristic should not trigger false positives).

---

Generated by retro agent from #628

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 3:56 PM UTC · Completed 4:00 PM UTC
Commit: f76c1fc · View workflow run →

[fullsend-ai-triage]

Triage summary

This is a well-scoped enhancement to the triage agent's dispatch logic. The concrete example (#626 → PR #628 closed without merge) demonstrates the cost clearly.

Relationship to existing work:

• triage: add configurable human-approval gate before ready-to-code auto-dispatch #605 (configurable human-approval gate) — blanket gate approach; this issue proposes smart detection at triage time, complementary rather than overlapping
• Code agent should surface chosen approach before implementing multi-option issues #259 (code agent surfaces chosen approach) — operates downstream; this prevents the wasted cycle entirely
• Triage agent should assess whether issue-proposed fixes target the appropriate architectural layer #590 (assess fix architectural layer) — related pattern but different signal: Triage agent should assess whether issue-proposed fixes target the appropriate architectural layer #590 checks where a fix belongs, this checks which approach to take
• PRs feat(scripts): gate triage-triggered code/fix PRs behind write+ approval #523, feat(#1754): add TRIAGE_AUTO_CODE configuration for ready-to-code #567, feat: triage variable to control auto-coding #331 — all address auto-dispatch control via configuration knobs; this issue adds triage-internal heuristic intelligence independent of those knobs

Proposed test case:

Eval case: multi-approach-ambiguity

Input: Issue body with 3 numbered approaches of varying complexity
  (docs fix, warning annotations, CI structural change)
  and phrasing: "Some combination of these"

Expected output:
  - label_actions does NOT add 'ready-to-code'
  - label_actions DOES add 'triaged'
  - comment recommends a specific approach with rationale
  - comment invites author to confirm before code dispatch

Edge cases:
  - Clear author preference ("the primary fix is X") → ready-to-code OK
  - Multiple approaches of similar scope → no suppression needed

---

Labels: This is a feature request targeting the triage agent definition. Applying 'feature' for category and 'triage-agent' for component area.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance

[fullsend-ai-prioritize]

RICE Priority Score: 1.2

Score breakdown

Dimension	Score	Reasoning
Reach	1.5	This affects all users of the fullsend-ai agent platform whenever they file issues proposing multiple approaches with different complexity levels. The concrete example (#626) shows this is a recurring pattern, and the triage agent processes every incoming issue, so the improvement has platform-wide applicability for a meaningful subset of issues.
Impact	1	Medium — the improvement prevents wasted code-agent and review-agent cycles (the cited example wasted ~20+ min of agent time plus human review), which is a noticeable efficiency gain. However, the core triage and coding workflows still function; this is an optimization to reduce rework rather than fixing a broken or blocking capability.
Confidence	0.8	High — the issue is exceptionally well-described with a concrete triggering incident (PR #628 closed without merge), a clear proposed change targeting agents/triage.md, specific heuristic signals to detect ('Option 1/2', numbered lists, 'Some combination of'), and measurable validation criteria. The main uncertainty is whether the multi-approach detection heuristic can avoid false positives on issues that list alternatives but have a clear preference.
Effort	1	Medium — requires modifying the triage agent definition to add multi-approach detection guidance, designing reliable textual heuristics, and handling edge cases (clear author preference, similar-scope alternatives). The scope is well-bounded to the triage agent's dispatch logic and doesn't require architectural changes, but the heuristic design and testing against false positives adds moderate complexity.

Formula: (1.5 x 1 x 0.8) / 1 = 1.2