Skip to content

Code agent should compare forge implementations when fixing cross-forge gaps #898

Description

@fullsend-ai-retro

What happened

Issue #6418 reported that GitLab retro dispatch was missing ORIGINATING_URL and REPO_FULL_NAME. The triage agent correctly identified the root cause and recommended a fix for those two variables, plus noting GITLAB_TOKEN should be a pre-configured CI/CD variable. The code agent implemented exactly those fixes.

However, the human (ggallen) discovered an additional gap: RETRO_COMMENT needs to be extracted from EVENT_PAYLOAD_B64 on GitLab for retro stages, matching what GitHub's reusable-retro.yml does with event_payload.comment.body. Neither triage nor code agent identified this because neither compared the full GitHub retro flow with the GitLab equivalent. The human added the extraction in a force-push at 17:16 UTC.

The triage agent even noted that "both this dispatch fix and [agents#817] need to land for GitLab retro to function end-to-end" — acknowledging cross-dependency — but did not enumerate what the GitLab path was still missing relative to the GitHub path.

What could go better

When an issue reports a feature that works on one forge (GitHub) but fails on another (GitLab), agents should systematically compare the two implementation paths rather than fixing only what's in the error message or issue description.

In this case, the harness env validation caught ORIGINATING_URL (the first missing required var), and the human author had already identified REPO_FULL_NAME and GITLAB_TOKEN. But RETRO_COMMENT is not a harness-required var — it's a functional variable the retro agent uses that's set by the GitHub workflow but had no GitLab equivalent. This class of gap (optional but functionally important variables) won't surface through validation errors and requires active comparison.

Confidence: High that this is the root cause of the miss. Both agents analyzed "what does the harness require" rather than "what does the working forge path provide." The triage agent's analysis was correct for required vars but incomplete for the full retro flow.

Uncertainty: I don't have the code agent's transcript to confirm it didn't read reusable-retro.yml. It's possible the agent read it but didn't recognize RETRO_COMMENT as needing a GitLab equivalent because it's not in the harness env section.

This pattern will recur — agents#816 (multi-forge retro) is still open, and additional forge-parity work is expected as GitLab support matures.

Proposed change

Add forge-parity analysis guidance to the code-implementation skill in fullsend-ai/agents. When the code agent detects it is working on a cross-forge fix (signals: issue mentions one forge failing where another works, files being modified are forge-specific like fullsend-agent.yml or reusable-*.yml, or the forge-abstraction.md contributing guide is relevant), it should:

  1. Identify the working forge's implementation path for the same stage/feature (e.g., reusable-retro.yml for GitHub retro)
  2. Read that file and enumerate all variables, setup steps, and configuration it provides
  3. Compare against the broken forge's equivalent path (e.g., the retro section of fullsend-agent.yml)
  4. Fix ALL gaps — not just the ones that cause validation failures

This could be added as a conditional step in the code-implementation skill (e.g., after the planning phase, before implementation), or as guidance in the code agent definition that triggers when forge-specific files are in scope.

The triage agent could also benefit from similar guidance — when triaging cross-forge issues, enumerate all variables/setup in the working path, not just the ones in the error message. This would be a separate change to the triage agent definition or triage skill.

Related but distinct: agents#808 (review agent detecting cross-variant drift) covers the review phase, not the code/triage phase. fullsend#6275 (automated cross-forge parity tests) covers CI-time detection, not agent-time prevention.

Validation criteria

The next 3 code agent runs that fix cross-forge gaps (GitLab feature parity issues) should include evidence of comparing the working forge's implementation path. Specifically: (1) the agent's commit should address all variable/setup gaps between forges, not just those in the error message, and (2) the review agent should not find missing forge-parity items that a simple diff between the two paths would have caught. Track against issues linked to agents#816 or any new GitLab parity issues.


Generated by retro agent from fullsend-ai/fullsend#6419

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions