Skip to content

chore: add e2e validation as required final phase in plan-adr #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jamesc merged 2 commits into from
Mar 22, 2026
Merged

chore: add e2e validation as required final phase in plan-adr #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9b1e1bf
pick-epic: clean up agent worktrees after PR merge
jamesc Mar 19, 2026
8364168
chore: add e2e validation as required final phase in plan-adr
jamesc Mar 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions skills/pick-epic/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,11 @@ Use `Agent` tool with `isolation: "worktree"` and `run_in_background: true` for
```bash
gh pr merge <PR> --squash --admin
```
- After merge, clean up the agent's worktree to reclaim disk space:
```bash
git worktree remove --force <worktree-path>
git branch -D <branch-name>
```

**4d. Once all PRs in the wave are merged**, start Wave N+1.

Expand Down
13 changes: 8 additions & 5 deletions skills/plan-adr/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,12 @@ Take an **accepted ADR** and break it into a set of **sequenced, agent-ready Lin
BT-D: Codegen for Y (blocked by BT-A, BT-B)

Phase 3 (integration, depends on Phase 2):
BT-E: E2E tests (blocked by BT-C, BT-D)
BT-F: REPL integration (blocked by BT-D)

Phase 4 (polish):
BT-G: Documentation and examples (blocked by BT-E)
BT-E: REPL integration (blocked by BT-D)
BT-F: MCP tools if applicable (blocked by BT-D)

Phase 4 (validation + polish):
BT-G: E2E btscript test (blocked by BT-E) — REQUIRED for user-facing features
BT-H: Documentation and examples (blocked by BT-E)
```

**Issue sizing rules:**
Expand Down Expand Up @@ -200,6 +201,8 @@ Every issue must be `agent-ready` — an agent should be able to pick it up with

4. **REPL integration is not optional**: If the ADR affects user-facing behavior, include a REPL integration issue. Features aren't done until users can interact with them.

5. **E2E validation as the final phase**: The last issue in every epic MUST include an e2e btscript test (`tests/e2e/cases/<feature>.btscript`) that exercises the feature end-to-end from the Beamtalk REPL. This catches integration failures invisible to unit tests — missing code path dependencies, serialization bugs, broken describe/list-tools responses. The ADR 0069 tracing epic shipped 7 PRs that all passed CI but were non-functional from the REPL because no e2e test existed. **EUnit and BUnit alone are insufficient for user-facing features.**

Comment on lines +204 to +205

@coderabbitai coderabbitai Bot Mar 22, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Clarify whether E2E test should be the last issue or in the last issue.

The principle states "The last issue in every epic MUST include an e2e btscript test," but the phase example (lines 68-70) shows the E2E test as a separate issue (BT-G) followed by documentation (BT-H). This creates ambiguity:

  • If BT-H is the last issue, should it include the E2E test alongside documentation?
  • Or should the E2E test always be the final issue, with no work items after it?

The intent is clear from context, but precise wording would prevent confusion during implementation.

📝 Suggested clarification

Consider one of these alternatives:

Option 1 (if E2E should be a separate final issue):

-5. **E2E validation as the final phase**: The last issue in every epic MUST include an e2e btscript test
+5. **E2E validation as the final phase**: Every epic MUST include an e2e btscript test as the final validation issue in the last phase

Option 2 (if E2E can be earlier in the final phase):

-5. **E2E validation as the final phase**: The last issue in every epic MUST include an e2e btscript test
+5. **E2E validation as the final phase**: Every epic MUST include an e2e btscript test in the final validation phase

And update the example to match the chosen interpretation.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@skills/plan-adr/SKILL.md` around lines 204 - 205, The wording "The last issue
in every epic MUST include an e2e btscript test" is ambiguous versus the phase
example that lists BT-G (E2E) followed by BT-H (documentation); update the
SKILL.md sentence and the example so they match a single clear rule: either
require the E2E test to be the absolute final issue (change the sentence to "The
final issue in every epic MUST be an e2e btscript test" and move/remove BT-H
after it) or allow E2E as the final phase item (change the sentence to "Every
epic's final phase MUST include an e2e btscript test" and ensure BT-G appears
within the final phase before BT-H), and adjust the example lines showing
BT-G/BT-H to reflect the chosen option.

### Splitting Large Issues

If an issue feels like **L or XL**, split along these seams:
Expand Down
Loading