Automate cross-forge harness passthrough variable parity test

[fullsend-ai-retro]

What happened

Issue #6273 reported that the GitLab code agent failed because CODE_ALLOWED_TARGET_BRANCHES was declared in reusable-code.yml (GitHub) but missing from the GitLab CI template. The issue author documented a broader pattern: fix.yaml and scribe.yaml have similar ${VAR} passthroughs that will break when multi-forged.

The triage agent (run 31988031730) proposed a comprehensive test shape that would parse harness YAMLs for ${VAR} patterns and verify both forge templates declare them. The code agent (run 31988273425) implemented a simpler approach: a manually-maintained passthroughVars list in TestGitLabAgentTemplateHarnessPassthroughVars with a comment instructing future developers to update it.

Related issue #5799 covers ${VAR} substitution semantics for unset vars but does not address cross-forge parity detection.

What could go better

The manually-maintained passthroughVars list in TestGitLabAgentTemplateHarnessPassthroughVars requires a developer to remember to add new entries when passthrough variables are introduced. This is the same class of error that caused #6273 — a variable was added to the GitHub workflow but the GitLab template was missed. A comment telling developers to update a list is helpful but not a structural guarantee.

The GitHub reusable workflows (reusable-code.yml, reusable-dispatch.yml, etc.) and the GitLab agent template both live in this repo. Passthrough variables in the GitHub workflows follow a recognizable pattern: they are set to an empty string '' in the env: block specifically so the harness engine's os.Expand succeeds. The test could parse these workflows to extract such variables automatically and verify the GitLab template declares them too.

Confidence: High. The issue author explicitly documented that fix.yaml and scribe.yaml will hit the same bug when multi-forged. An automated parity check would catch those before they surface as runtime failures.

Proposed change

Enhance TestGitLabAgentTemplateHarnessPassthroughVars in internal/scaffold/scaffold_gitlab_test.go to replace the manually-maintained passthroughVars list with automated extraction:

1. Parse each reusable-*.yml workflow file under .github/workflows/ for env: blocks.
2. Extract variable names that are set to empty string ('' or ""), which is the passthrough convention.
3. For each extracted variable, verify it is declared in the GitLab agent template's variables: section (read via GitLabPerRepoFile).
4. Fail with a descriptive message naming the workflow file, variable, and the GitLab template path when a declaration is missing.

This keeps the test entirely within this repo (no external dependencies on fullsend-ai/agents) and catches mismatches at go test time rather than at runtime on a real GitLab pipeline.

Validation criteria

1. The enhanced test passes with the current set of passthrough variables.
2. Adding a new empty-string env var to any reusable-*.yml workflow without a corresponding GitLab template declaration causes the test to fail with a clear error message.
3. The manually-maintained passthroughVars slice is removed or reduced to an override/exclusion list rather than the primary source of truth.

---

Generated by retro agent from #6274

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 3:38 AM UTC · Completed 3:41 AM UTC

Commit: 7aa8587 · View workflow run →

[fullsend-ai-triage]

The existing TestGitLabAgentTemplateHarnessPassthroughVars at internal/scaffold/scaffold_gitlab_test.go:481 currently maintains a hardcoded passthroughVars slice with just CODE_ALLOWED_TARGET_BRANCHES. The proposed enhancement replaces this with automated extraction.

Implementation notes

The reusable workflows to parse are:

.github/workflows/reusable-{code,dispatch,fix,prioritize,retro,review,triage}.yml

The passthrough convention is env vars set to '' in env: blocks — these exist specifically so os.Expand in the harness engine succeeds. A simple YAML parse or regex scan for VAR_NAME: '' patterns in the env: section would work. The GitLab template to verify against is read via GitLabPerRepoFile(".gitlab/ci/fullsend-agent.yml").

Consider whether an exclusion list is needed for variables that are intentionally GitHub-only (not used in any multi-forge harness). If all current empty-string env vars are passthrough candidates, no exclusion list is needed.

Related issues

• GitLab agent template missing CODE_ALLOWED_TARGET_BRANCHES (harness env validation failure) #6273 (closed) — the original bug this test was created to prevent
• Verify ${VAR} substitution semantics for unset harness env vars #5799 (open) — covers ${VAR} substitution semantics for unset vars (different concern: behavior when vars are unset, not cross-forge declaration parity)

Proposed test shape

func TestGitLabAgentTemplateHarnessPassthroughVars(t *testing.T) {
    // Parse reusable-*.yml for env vars set to empty string
    workflows, err := filepath.Glob("../../.github/workflows/reusable-*.yml")
    require.NoError(t, err)
    require.NotEmpty(t, workflows)

    var passthroughVars []struct{ workflow, varName string }
    for _, wf := range workflows {
        // Parse YAML, extract env: block entries where value is ""
        // Append {workflow, varName} for each match
    }

    glContent, err := GitLabPerRepoFile(".gitlab/ci/fullsend-agent.yml")
    require.NoError(t, err)
    s := string(glContent)

    for _, pv := range passthroughVars {
        assert.Contains(t, s, pv.varName,
            "GitLab agent template missing %s (declared in %s)",
            pv.varName, pv.workflow)
    }
}

---

Labels: This is a test enhancement for the harness/scaffold component at medium priority — it prevents cross-forge parity regressions but is not itself a runtime bug.

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance