Skip to content

docs(agents): correct the runOnce key rule this repo hands contributors - #3388

Merged
gustavolira merged 1 commit into
mainfrom
agents-runonce-key-rule
Aug 25, 2026
Merged

docs(agents): correct the runOnce key rule this repo hands contributors#3388
gustavolira merged 1 commit into
mainfrom
agents-runonce-key-rule

Conversation

@gustavolira

Copy link
Copy Markdown
Member

One file, 19 lines. RHIDP-16456.

rhdh-e2e-test-utils#149 fixed this in the library's own guide and merged. But the file someone writing an overlay spec actually reads — this repo's AGENTS.md — still says:

Prefix with workspace name: "tech-radar-setup", "argocd-deploy"

That is the advice that produced #3318. Worse, both examples above it wrap configure() and deploy() in a literal key, so rule and examples together read as an endorsement of the shape that breaks.

What changes

The rule now explains why the "and projects" half matters, rather than only asserting it:

const flagDir = path.join(os.tmpdir(), `playwright-once-${process.ppid}`);
const flagFile = path.join(flagDir, `${key}.done`);

Nothing in that path comes from the project. One spec in two projects — which is what adding an -app-next lane does — and the first project's setup satisfies the second, which then deploys nothing and fails much later on a missing element.

The fix is stated as `${key}-${rhdh.deploymentConfig.namespace}`, which is what deploy() does internally and why deploy() was never affected. Both examples carry it now.

A literal key stays correct for setup that is genuinely shared — bulk-import has one of each on purpose, so the contrast is named rather than reduced to "avoid literals". Nesting is marked as no rescue: a project-shared outer key skips before deploy() is reached, so its internal protection never gets a say.

Scope

Documentation only, no code. The static check that would have enforced this was #3375, closed — it flagged nothing real and produced a false positive, so the convention is carried by the docs, which is where the reviewer said it belonged.

🤖 Generated with Claude Code

…he broken half

rhdh-e2e-test-utils#149 fixed this in the library's own guide, but the file
someone writing an overlay spec actually reads still said:

    Prefix with workspace name: "tech-radar-setup", "argocd-deploy"

That is the advice that produced #3318. Both examples above it used a literal
key around configure() and deploy(), so the two together read as an
endorsement of exactly the shape that breaks.

The rule now says why the "and projects" half matters — the flag directory is
keyed on the runner PID alone, so nothing in the path comes from the project —
and gives the fix as `${key}-${rhdh.deploymentConfig.namespace}`, which is what
deploy() has always done internally and why deploy() was never affected. Both
examples carry it.

A literal key stays correct for setup that is genuinely shared; bulk-import has
one of each on purpose, so the contrast is named rather than left as "avoid
literals". And nesting is marked as no rescue: a project-shared outer key skips
before deploy() is reached.

RHIDP-16456.
@gustavolira
gustavolira requested review from a team, gashcrumb and kadel as code owners August 25, 2026 13:06
@sonarqubecloud

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 25, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:07 PM UTC · Completed 1:11 PM UTC

Commit: 3562675 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $0.72

@gustavolira

Copy link
Copy Markdown
Member Author

/publish

@github-actions

Copy link
Copy Markdown
Contributor

PR action (/publish) cancelled: PR doesn't touch only 1 workspace.

@fullsend-ai-review

Copy link
Copy Markdown

Review — approve

Documentation-only change to AGENTS.md (1 file, +19/−4). Corrects the test.runOnce key uniqueness guidance and examples to prevent the class of silent deployment failures seen in #3318.

What was reviewed

Correctness of the technical explanation. The PR adds the runOnce flag-file implementation detail:

const flagDir = path.join(os.tmpdir(), `playwright-once-${process.ppid}`);
const flagFile = path.join(flagDir, `${key}.done`);

This matches the mechanism described in #3318 and #3375. The key insight — that nothing in the flag path comes from the Playwright project, so a literal key is shared across projects running the same spec — is accurate and well-motivated.

Alignment with existing code. Verified against actual runOnce call sites in the repo:

  • workspaces/bulk-import/e2e-tests/tests/specs/bulk-import.spec.ts:76 — already uses namespace-scoped key (bulk-import-rhdh-setup-${rhdh.deploymentConfig.namespace}), consistent with the updated guidance.
  • workspaces/homepage/e2e-tests/tests/specs/homepage.spec.ts — demonstrates both patterns: literal "homepage-keycloak-groups" for cluster-scoped Keycloak setup, and homepage-deploy-${namespace} for per-project deployment. This validates the PR's nuance about when each form is appropriate.
  • workspaces/bulk-import/e2e-tests/tests/specs/bulk-import-orchestrator.spec.ts:25 — uses literal "bulk-import-install-orchestrator-and-test-workflow" for genuinely shared orchestrator deployment into a fixed namespace. Confirms the PR's claim that "bulk-import has one of each."

Updated examples. Both code examples now use template literals with namespace suffix (tech-radar-setup-${rhdh.deploymentConfig.namespace}, my-setup-${rhdh.deploymentConfig.namespace}), matching the recommended pattern.

Nesting clarification. The updated nesting bullet correctly notes that a project-shared outer key prevents deploy()'s internal namespace-scoped runOnce from ever executing — the outer guard skips the entire callback before the inner guard is reached.

Impact scope. CLAUDE.md is @AGENTS.md, so this file is the primary instruction source for AI tools writing E2E tests in this repo. Accurate examples here directly prevent the class of bug that caused #3318.

Findings

No blocking findings. The change is factually accurate, well-scoped, and aligned with existing code patterns.


🤖 Reviewed with fullsend


Protected paths detected — this PR modifies files under one or more
protected paths. The review agent cannot approve PRs that touch these paths.
A human reviewer must approve this PR.

Protected files in this PR:

  • AGENTS.md

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 25, 2026
@github-actions github-actions Bot added the non-workspace-changes PR changes files outside workspace directories label Aug 25, 2026
@gustavolira
gustavolira merged commit efbd210 into main Aug 25, 2026
13 checks passed
@gustavolira
gustavolira deleted the agents-runonce-key-rule branch August 25, 2026 13:16
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 25, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 1:17 PM UTC · Completed 1:27 PM UTC

Commit: 3562675 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.18

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #3388 — docs(agents): correct the runOnce key rule

Verdict: clean workflow, no proposals.

This was a docs-only PR by gustavolira correcting the test.runOnce key scoping guidance in AGENTS.md. The change implements the documentation improvement proposed in issue #3333, which was filed by the retro agent after PR #3318 exposed a silent deployment failure caused by project-shared runOnce keys.

Timeline

Time (UTC) Event
13:06 PR created — 1 file changed, +19/−4 lines in AGENTS.md
13:07 SonarQube passes
13:07–13:11 Review agent runs (run 32851389850) — claude-opus-4-6, $0.72, 32 turns
13:11 Review posted — approves content, but protected-path policy downgrades to comment; requires-manual-review label applied
13:16 PR merged by author

Review quality

The review agent performed thorough verification for a docs PR:

  • Cross-referenced 3 spec files (bulk-import.spec.ts, homepage.spec.ts, bulk-import-orchestrator.spec.ts) to verify the updated guidance matches actual runOnce usage patterns in the repo.
  • Fetched linked issues #3318 and #3375 to validate the PR's root-cause claims.
  • Correctly confirmed both namespace-scoped keys (per-project setup) and literal keys (genuinely shared setup) are present in the codebase, matching the new documentation's two-pattern guidance.
  • The protected-path mechanism correctly flagged AGENTS.md as requiring human review.

No false positives, no missed issues in the content itself. The review agent's analysis was substantive — more detailed than the human review (author self-merged without a separate reviewer).

Agents repo

Discovered from run 32851389850: agent definitions, skills, and harness configs resolved from fullsend-ai/agents at 4bbe4f5.

Issue #3333 remains open

PR #3388 implements exactly what issue #3333 (retro-filed, labeled ready-to-code) proposed, but the PR body doesn't include closing keywords (Closes #3333). The issue remains open and could trigger redundant code agent work. This gap is already tracked by existing platform issues: fullsend#961 (review agent escalation for missing closing keywords), fullsend#6145 (post-merge guard for unclosed issues), and fullsend#1647 (code agent pre-creation check). No new proposal needed.

Closed-loop success

This PR completes a full retro-to-resolution cycle: retro agent analyzed PR #3318 → filed issue #3333 → human implemented the fix in PR #3388. The retro's original finding (that the runOnce key guidance in AGENTS.md was the root cause of silent deployment failures) was accurate and actionable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-workspace-changes PR changes files outside workspace directories requires-manual-review Review requires human judgment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant