fix(e2e): use workflow_run trigger pattern to support fork PRs#10
Merged
Conversation
Fork PRs cannot access repository secrets, so agent E2E tests that require
an OpenAI API key silently skip (empty key). This patch applies the same
two-workflow trigger pattern already used by self-review-pr.
Change 1 — new test-e2e-trigger.yml:
Lightweight pull_request workflow with zero permissions. Writes PR number
and head SHA to flat files, uploads as the 'e2e-test-context' artifact
(1-day retention). Runs in the base repo context so it can be referenced
by workflow_run.
Change 2 — test-e2e.yml converted to workflow_run:
- Remove pull_request trigger; add workflow_run referencing 'Test E2E Trigger'
- Add resolve-context job: downloads e2e-test-context artifact via OIDC
(setup-credentials + GITHUB_APP_TOKEN), outputs pr-number and pr-head-sha
- test-pirate-agent and test-invalid-agent now depend on resolve-context,
checkout the PR head ref, use setup-credentials for OIDC, and fall back
through OPENAI_API_KEY_FROM_SSM before secrets.OPENAI_API_KEY
- test-output-extraction and test-job-summary run on both push and
workflow_run (no secrets needed); checkout PR head SHA on workflow_run
- mention-reply jobs unchanged (workflow_dispatch only)
Change 3 — self-review-pr-trigger.yml:
Narrow pull_request types from [ready_for_review, opened, review_requested]
to [review_requested] to reduce unnecessary trigger noise.
resolve-context has no checkout step so it must keep the pinned ref. The four jobs that do check out the PR head (test-pirate-agent, test-invalid-agent, test-mention-reply-toplevel, test-mention-reply-inline) now use ./setup-credentials, consistent with how ./ and ./review-pr/mention-reply are referenced in the same file.
./setup-credentials runs node dist/credentials.js, so dist/ must exist first. Reorder steps in test-pirate-agent and test-invalid-agent: checkout → pnpm setup → node setup → build → setup-credentials → run test. The mention-reply jobs already had the correct order; no change there.
Re-run E2E tests when a draft PR is marked ready for review.
aheritier
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test-pirate-agentandtest-invalid-agentusesecrets.OPENAI_API_KEY, which is empty on fork PRs. Tests silently fail (empty key → agent errors).Solution
Three-part change mirroring the existing
self-review-pr/self-review-pr-triggerpattern.1. New
test-e2e-trigger.ymlLightweight
pull_requestworkflow withpermissions: {}. Writes PR number and head SHA to flat files and uploads as thee2e-test-contextartifact (1-day retention). Runs in the base-repo context so the main workflow can download it.2.
test-e2e.ymlconverted toworkflow_runpull_requesttrigger; addsworkflow_runreferencing "Test E2E Trigger"resolve-contextjob: fetches OIDC token viasetup-credentials, downloads the artifact usingGITHUB_APP_TOKEN, outputspr-numberandpr-head-shatest-pirate-agent/test-invalid-agent: depend onresolve-context, checkoutrefs/pull/<N>/head, useOPENAI_API_KEY_FROM_SSM || secrets.OPENAI_API_KEYtest-output-extraction/test-job-summary: no secrets needed, run on bothpushandworkflow_runworkflow_dispatchonly)3.
self-review-pr-trigger.ymltypes narrowedtypes: [ready_for_review, opened, review_requested]→types: [review_requested]to reduce unnecessary trigger noise.