Skip to content

Commit fa931ca

Browse files
committed
feat(ci): add issue auto-implement action and workflow
- Add workflow issue-auto-implement.yml (label, comment, PR review triggers) - Add composite action: team check, labels, assess script, implement placeholder, verify loop, create PR - Add assess script (TypeScript/tsx, Claude, fixtures) with vitest tests - Add workflow issue-auto-implement-test.yml to run assess tests on PR - Add README (usage, inputs, testing) and AGENTS.md (flows, verification, next steps) - Use AUTO_IMPLEMENT_* prefix for secrets/vars; require github_allowed_trigger_team Made-with: Cursor
1 parent 5e323d8 commit fa931ca

16 files changed

Lines changed: 3239 additions & 0 deletions
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# AGENTS.md — Issue auto-implement action
2+
3+
For agents making changes to this action. This file summarizes flows, design decisions, and implementation details.
4+
5+
## Flows
6+
7+
### 1. Issue to first PR
8+
9+
- **Triggers:** `issues.labeled` (prefixed trigger label), `issue_comment.created` on a labeled issue when **no PR exists yet**.
10+
- **Flow:** Normalize event → assess (enough info?) → if `request_info`: post comment, add needs-info label, exit. If `implement`: implement step (push to branch `auto-implement-issue-<N>`) → verify → on fail retry (cap `max_implement_retries`); on pass create PR with "Closes #N", add pr-created label, optional comment.
11+
12+
### 2. Issue comment when PR already exists
13+
14+
- **Trigger:** `issue_comment.created` on an issue that **already has an open PR** for that issue.
15+
- **Flow:** Post a short reply on the issue directing the user to the PR; exit. No assessment or implement.
16+
17+
### 3. PR review → iteration
18+
19+
- **Triggers:** `pull_request_review.submitted`, `pull_request_review_comment.created` when the PR is from an automation branch or body contains "Closes #N".
20+
- **Flow:** Resolve issue number from PR (body "Closes #N"/"Fixes #N" or head branch `auto-implement-issue-<N>`) → assess with issue + review content → implement ("address review feedback"), push to same branch → verify → on pass: do **not** create PR; post comment on the PR summarizing the new commit(s).
21+
22+
## Event normalization
23+
24+
From the workflow event payload, derive:
25+
26+
- **Issue number:** For `issues` or `issue_comment`: `event.issue.number`. For `pull_request_review` or `pull_request_review_comment`: parse PR body for "Closes #N" or "Fixes #N", or PR head branch for `auto-implement-issue-<N>`.
27+
- **PR exists for issue (issue_comment only):** Check whether an open PR exists for that issue (e.g. head branch `auto-implement-issue-<N>` or body "Closes #<N>").
28+
29+
## Assess script
30+
31+
- **Path:** `assess/index.ts` (TypeScript), run with `npx tsx assess/index.ts` (no build).
32+
- **Input:** Reads event from `GITHUB_EVENT_PATH`; optional context files from input.
33+
- **Output:** JSON with `action` (`implement` | `request_info`), `comment_body` (if request_info), `verification_notes` (optional). Written to file or GITHUB_OUTPUT.
34+
- **When triggered by PR review:** Include PR review body and review comments in the payload sent to Claude.
35+
36+
## Branch and PR
37+
38+
- Branch name: `auto-implement-issue-<issue_number>`.
39+
- PR title or body must include "Closes #N" (or "Fixes #N") so merging auto-closes the issue.
40+
- On iteration (PR already exists): do not create a new PR; post a comment on the PR summarizing the new commit(s).
41+
42+
## Restricting who can trigger
43+
44+
Only members of the `github_allowed_trigger_team` (input; set via repo variable `AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM`) may trigger the flow. First step checks `github.actor` against that team; if the variable is unset or the actor is not a member, fail immediately. Token must have `read:org`.
45+
46+
## Labels
47+
48+
All automation labels use `label_prefix` (default `automation`): `{prefix}/auto-implement`, `{prefix}/needs-info`, `{prefix}/pr-created`. Create via API if missing.
49+
50+
## Verification
51+
52+
When changing this action or the assess script:
53+
54+
1. **Run the assess unit tests locally** before committing: `cd .github/actions/issue-auto-implement/assess && npm ci && npm test`. Do not rely on CI alone—catch failures locally first, then push.
55+
2. **CI** runs the same tests in `.github/workflows/issue-auto-implement-test.yml` when you push or open a PR that touches `.github/actions/issue-auto-implement/**`.
56+
3. **Optional:** Run the assess script with a fixture and `ANTHROPIC_API_KEY` for end-to-end assessment behavior.
57+
4. **Full workflow:** Trigger manually on a test issue (trigger label or comment). Ensure repo secrets/variables are set. Inspect the Actions run and the issue/PR; re-run after changes to the implement or verify loop.
58+
59+
## Next steps (implementation backlog)
60+
61+
Recommended order:
62+
63+
1. **Context files in assess** — Pass the `context_files` input into the assess script (e.g. via env) and include those file contents (AGENTS.md, REFERENCE.md, etc.) in the Claude assessment prompt so the model has full repo guidance.
64+
2. **Fetch all issue comments** — For `issues` and `issue_comment` events, optionally call the GitHub API to list all comments on the issue and include them in the assessment payload (not only the single comment from the event).
65+
3. **Implement step (real)** — Replace the placeholder with Claude generating and applying code changes. Options: call Anthropic API to produce a patch or file edits from the issue body + verification_notes (and repo context), then apply, commit, and push; or integrate a Claude Code Action / external tool. Branch is already checked out; implement must make commits and push.
66+
4. **True implement–verify loop** — On verify failure, re-run the **implement** step with the verify failure output in the prompt (then verify again), up to `max_implement_retries`. Currently only the verify command is retried; the plan requires re-implementing with failure context.
67+
5. **PR review iteration path** — When the trigger is `pull_request_review` or `pull_request_review_comment`: after verify passes, do **not** create a new PR; instead post a comment on the existing PR summarizing the changes in the new commit(s). Detect "PR already exists" (e.g. from event type or by checking for an open PR for this branch) and branch the flow: create PR vs. comment on PR.
68+
6. **Optional comment when PR is created** — Add an input (e.g. `post_pr_comment`) and, when creating a PR, optionally post a short comment on the issue linking to the new PR.
69+
7. **Comment when retries exhausted** — When the verify loop fails after all retries, post a comment on the issue so the run is visible and explainable from the issue thread. Add a step that runs on failure (e.g. `if: failure() && steps.assess.outputs.action == 'implement'`) and posts a comment on the issue: e.g. "The auto-implement run could not complete: verification failed after N attempts. See the [workflow run](link) for logs. You can address the failure and re-trigger by adding a comment or re-applying the label."
70+
8. **Secrets and variables** — Document in README: add `AUTO_IMPLEMENT_ANTHROPIC_API_KEY` as a repo secret; set `AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM` (required) as a repo variable; note that the default `GITHUB_TOKEN` may need to be replaced with a PAT that has `read:org` for the team check.
71+
9. **Local run with fixture and Claude** — Add an npm script or small wrapper (e.g. `npm run assess:fixture issue-labeled`) to run the assess script with a fixture and real `ANTHROPIC_API_KEY` for manual end-to-end assessment testing.
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Issue auto-implement action
2+
3+
Reusable composite action for label-triggered issue automation: assess (request more info or implement), implement-verify loop, then create PR or iterate on an existing PR after review.
4+
5+
## Usage
6+
7+
Used by `.github/workflows/issue-auto-implement.yml`. Requires `anthropic_api_key` (e.g. from repo secret `AUTO_IMPLEMENT_ANTHROPIC_API_KEY`), `github_allowed_trigger_team` (e.g. from repo variable `AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM`), and `github_token` from the workflow.
8+
9+
## Inputs
10+
11+
| Input | Required | Default | Description |
12+
|-------|----------|---------|-------------|
13+
| `anthropic_api_key` | Yes | - | Claude API key. Set via repo secret `AUTO_IMPLEMENT_ANTHROPIC_API_KEY` so multiple actions can use different keys. |
14+
| `github_token` | Yes | - | Token with contents, issues, pull-requests, read:org |
15+
| `context_files` | No | AGENTS.md,REFERENCE.md | Comma-separated paths for assessment context |
16+
| `assessment_reference_issue` | No | 192 | Reference issue number for "enough information" |
17+
| `label_prefix` | No | automation | Prefix for labels (e.g. automation/auto-implement) |
18+
| `verify_commands` | No | go test ./... | Commands run for verification |
19+
| `max_implement_retries` | No | 3 | Max retries on verify failure (cap 5) |
20+
| `github_allowed_trigger_team` | Yes | - | GitHub Team slug (e.g. org/team); only members can trigger. Set via repo variable `AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM`. |
21+
22+
Secrets and variables use an action-specific prefix (e.g. `AUTO_IMPLEMENT_`) so each action can have its own keys/variables and it's clear which workflow uses which. This also avoids clashing with platform-reserved names (e.g. `GITHUB_*`).
23+
24+
## Triggers
25+
26+
- **issues.labeled** — prefixed trigger label (e.g. `automation/auto-implement`)
27+
- **issue_comment.created** — on an issue with that label (redirect to PR if PR exists)
28+
- **pull_request_review.submitted** / **pull_request_review_comment.created** — PR from automation branch or with "Closes #N"
29+
30+
## Labels
31+
32+
The action ensures these labels exist (creates them if missing): `{prefix}/auto-implement`, `{prefix}/needs-info`, `{prefix}/pr-created`.
33+
34+
## Testing
35+
36+
From the repo root, run the assess script tests:
37+
38+
```bash
39+
cd .github/actions/issue-auto-implement/assess && npm ci && npm test
40+
```
41+
42+
CI runs these in `.github/workflows/issue-auto-implement-test.yml` when you push or open a PR that touches this action. To run the assess script with a real Claude call, set `ANTHROPIC_API_KEY` and use a fixture:
43+
44+
```bash
45+
cd .github/actions/issue-auto-implement/assess
46+
GITHUB_EVENT_PATH=./fixtures/issue-labeled.json GITHUB_EVENT_NAME=issues npx tsx index.ts
47+
```
48+
49+
For implementation details, verification steps, and the implementation backlog, see `AGENTS.md`.
Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
name: 'Issue auto-implement'
2+
description: 'Assess labeled issues (or PR reviews), request more info or implement with verify loop; create PR or iterate on existing PR.'
3+
inputs:
4+
anthropic_api_key:
5+
description: 'API key for Claude (assessment and implement step)'
6+
required: true
7+
github_token:
8+
description: 'GitHub token (contents, issues, pull-requests, read:org)'
9+
required: true
10+
context_files:
11+
description: 'Comma-separated paths to repo files to include in assessment (e.g. AGENTS.md, REFERENCE.md)'
12+
required: false
13+
default: 'AGENTS.md,REFERENCE.md'
14+
assessment_reference_issue:
15+
description: 'Reference issue number for "enough information" example (e.g. 192)'
16+
required: false
17+
default: '192'
18+
label_prefix:
19+
description: 'Prefix for automation labels (e.g. automation/auto-implement, automation/needs-info, automation/pr-created)'
20+
required: false
21+
default: 'automation'
22+
verify_commands:
23+
description: 'Shell commands to run for verification (e.g. go test ./..., go build)'
24+
required: false
25+
default: 'go test ./...'
26+
max_implement_retries:
27+
description: 'Max retries for implement step on verify failure (default 3, max 5)'
28+
required: false
29+
default: '3'
30+
github_allowed_trigger_team:
31+
description: 'GitHub Team slug whose members may trigger the flow (e.g. hookdeck/automation-triggers). Set via repo variable AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM.'
32+
required: true
33+
runs:
34+
using: 'composite'
35+
steps:
36+
- name: Check allowed trigger team
37+
shell: bash
38+
run: |
39+
TEAM="${{ inputs.github_allowed_trigger_team }}"
40+
if [ -z "$TEAM" ]; then
41+
echo "::error::github_allowed_trigger_team must be set (e.g. repo variable AUTO_IMPLEMENT_ALLOWED_TRIGGER_TEAM). Only team members can trigger this workflow."
42+
exit 1
43+
fi
44+
ORG="${TEAM%/*}"
45+
SLUG="${TEAM#*/}"
46+
HTTP=$(curl -s -o /dev/null -w "%{http_code}" \
47+
-H "Authorization: Bearer ${{ inputs.github_token }}" \
48+
-H "Accept: application/vnd.github+json" \
49+
"https://api.github.com/orgs/${ORG}/teams/${SLUG}/members/${{ github.actor }}")
50+
if [ "$HTTP" != "204" ]; then
51+
echo "::error::Actor ${{ github.actor }} is not in team $TEAM (HTTP $HTTP). Only team members can trigger this workflow."
52+
exit 1
53+
fi
54+
echo "Actor ${{ github.actor }} is in team $TEAM."
55+
- name: Ensure labels exist
56+
shell: bash
57+
env:
58+
GITHUB_TOKEN: ${{ inputs.github_token }}
59+
REPO: ${{ github.repository }}
60+
LABEL_PREFIX: ${{ inputs.label_prefix }}
61+
run: |
62+
for LABEL in auto-implement needs-info pr-created; do
63+
NAME="${LABEL_PREFIX}/${LABEL}"
64+
curl -s -o /dev/null -w "%{http_code}" \
65+
-H "Authorization: Bearer $GITHUB_TOKEN" \
66+
-H "Accept: application/vnd.github+json" \
67+
"https://api.github.com/repos/$REPO/labels" \
68+
-d "{\"name\":\"$NAME\",\"color\":\"ededed\"}" | grep -q 201 || true
69+
done
70+
- name: Install assess script dependencies
71+
shell: bash
72+
run: cd .github/actions/issue-auto-implement/assess && npm ci --omit=dev
73+
- id: assess
74+
name: Run assess
75+
shell: bash
76+
env:
77+
ANTHROPIC_API_KEY: ${{ inputs.anthropic_api_key }}
78+
GITHUB_TOKEN: ${{ inputs.github_token }}
79+
GITHUB_REPOSITORY: ${{ github.repository }}
80+
ASSESSMENT_REFERENCE_ISSUE: ${{ inputs.assessment_reference_issue }}
81+
run: |
82+
RESULT=$(cd .github/actions/issue-auto-implement/assess && npx tsx index.ts)
83+
echo "action=$(echo "$RESULT" | jq -r '.action')" >> $GITHUB_OUTPUT
84+
echo "issue_number=$(echo "$RESULT" | jq -r '.issue_number // empty')" >> $GITHUB_OUTPUT
85+
echo "pr_url=$(echo "$RESULT" | jq -r '.pr_url // empty')" >> $GITHUB_OUTPUT
86+
echo "comment_body<<BODY_EOF" >> $GITHUB_OUTPUT
87+
echo "$RESULT" | jq -r '.comment_body // empty' >> $GITHUB_OUTPUT
88+
echo "BODY_EOF" >> $GITHUB_OUTPUT
89+
echo "verification_notes<<NOTES_EOF" >> $GITHUB_OUTPUT
90+
echo "$RESULT" | jq -r '.verification_notes // empty' >> $GITHUB_OUTPUT
91+
echo "NOTES_EOF" >> $GITHUB_OUTPUT
92+
- name: Handle redirect to PR
93+
if: steps.assess.outputs.action == 'redirect_to_pr'
94+
shell: bash
95+
env:
96+
GITHUB_TOKEN: ${{ inputs.github_token }}
97+
REPO: ${{ github.repository }}
98+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
99+
PR_URL: ${{ steps.assess.outputs.pr_url }}
100+
run: |
101+
BODY="A PR is open for this issue. Please review and comment on the PR: $PR_URL"
102+
curl -s -X POST \
103+
-H "Authorization: Bearer $GITHUB_TOKEN" \
104+
-H "Accept: application/vnd.github+json" \
105+
"https://api.github.com/repos/$REPO/issues/$ISSUE_NUMBER/comments" \
106+
-d "{\"body\":$(echo "$BODY" | jq -Rs .)}"
107+
echo "Redirected user to PR. Exiting."
108+
exit 0
109+
- name: Handle request more info
110+
if: steps.assess.outputs.action == 'request_info'
111+
shell: bash
112+
env:
113+
GITHUB_TOKEN: ${{ inputs.github_token }}
114+
REPO: ${{ github.repository }}
115+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
116+
LABEL_PREFIX: ${{ inputs.label_prefix }}
117+
run: |
118+
COMMENT_BODY="${{ steps.assess.outputs.comment_body }}"
119+
curl -s -X POST \
120+
-H "Authorization: Bearer $GITHUB_TOKEN" \
121+
-H "Accept: application/vnd.github+json" \
122+
"https://api.github.com/repos/$REPO/issues/$ISSUE_NUMBER/comments" \
123+
-d "$(jq -n --arg b "$COMMENT_BODY" '{body: $b}')"
124+
NEEDS_LABEL="${LABEL_PREFIX}/needs-info"
125+
curl -s -X POST \
126+
-H "Authorization: Bearer $GITHUB_TOKEN" \
127+
-H "Accept: application/vnd.github+json" \
128+
"https://api.github.com/repos/$REPO/issues/$ISSUE_NUMBER/labels" \
129+
-d "{\"labels\":[\"$NEEDS_LABEL\"]}"
130+
echo "Posted request for more info. Exiting."
131+
exit 0
132+
- name: Checkout branch for implement
133+
if: steps.assess.outputs.action == 'implement'
134+
shell: bash
135+
env:
136+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
137+
run: |
138+
BRANCH="auto-implement-issue-${ISSUE_NUMBER}"
139+
git fetch origin main
140+
if git show-ref --verify --quiet refs/remotes/origin/"$BRANCH"; then
141+
git checkout "$BRANCH"
142+
git merge origin/main --no-edit
143+
else
144+
git checkout -b "$BRANCH" origin/main
145+
fi
146+
- name: Implement
147+
if: steps.assess.outputs.action == 'implement'
148+
shell: bash
149+
env:
150+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
151+
VERIFICATION_NOTES: ${{ steps.assess.outputs.verification_notes }}
152+
ANTHROPIC_API_KEY: ${{ inputs.anthropic_api_key }}
153+
run: |
154+
echo "Implement step: would run Claude Code Action or API here."
155+
echo "Verification notes: $VERIFICATION_NOTES"
156+
# Placeholder until Claude implement is wired; verify step will run next.
157+
- name: Push branch
158+
if: steps.assess.outputs.action == 'implement'
159+
shell: bash
160+
env:
161+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
162+
run: |
163+
BRANCH="auto-implement-issue-${ISSUE_NUMBER}"
164+
git push -u origin "$BRANCH" || true
165+
- id: verify_loop
166+
name: Verify (with retries)
167+
if: steps.assess.outputs.action == 'implement'
168+
shell: bash
169+
env:
170+
MAX_RETRIES: ${{ inputs.max_implement_retries }}
171+
VERIFY_CMDS: ${{ inputs.verify_commands }}
172+
run: |
173+
MAX=${MAX_RETRIES:-3}
174+
for i in $(seq 1 "$MAX"); do
175+
echo "Verify attempt $i of $MAX"
176+
if eval "$VERIFY_CMDS"; then
177+
echo "verified=true" >> $GITHUB_OUTPUT
178+
exit 0
179+
fi
180+
done
181+
echo "verified=false" >> $GITHUB_OUTPUT
182+
echo "::error::Verification failed after $MAX attempts"
183+
exit 1
184+
- name: Create PR
185+
if: steps.assess.outputs.action == 'implement' && steps.verify_loop.outcome == 'success'
186+
shell: bash
187+
env:
188+
GITHUB_TOKEN: ${{ inputs.github_token }}
189+
REPO: ${{ github.repository }}
190+
ISSUE_NUMBER: ${{ steps.assess.outputs.issue_number }}
191+
LABEL_PREFIX: ${{ inputs.label_prefix }}
192+
run: |
193+
BRANCH="auto-implement-issue-${ISSUE_NUMBER}"
194+
TITLE="Auto-implement issue #${ISSUE_NUMBER}"
195+
BODY="Closes #${ISSUE_NUMBER}"
196+
PR_JSON=$(curl -s -X POST \
197+
-H "Authorization: Bearer $GITHUB_TOKEN" \
198+
-H "Accept: application/vnd.github+json" \
199+
"https://api.github.com/repos/$REPO/pulls" \
200+
-d "{\"title\":\"$TITLE\",\"body\":\"$BODY\",\"head\":\"$BRANCH\",\"base\":\"main\"}")
201+
PR_URL=$(echo "$PR_JSON" | jq -r '.html_url // empty')
202+
if [ -n "$PR_URL" ]; then
203+
PR_NUM=$(echo "$PR_JSON" | jq -r '.number')
204+
NEEDS_LABEL="${LABEL_PREFIX}/pr-created"
205+
curl -s -X POST \
206+
-H "Authorization: Bearer $GITHUB_TOKEN" \
207+
-H "Accept: application/vnd.github+json" \
208+
"https://api.github.com/repos/$REPO/issues/$ISSUE_NUMBER/labels" \
209+
-d "{\"labels\":[\"$NEEDS_LABEL\"]}"
210+
echo "Created PR: $PR_URL"
211+
fi
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"action": "created",
3+
"issue": {
4+
"number": 42,
5+
"title": "Example issue",
6+
"body": "Description",
7+
"labels": [{ "name": "automation/auto-implement" }]
8+
},
9+
"comment": { "body": "Here is more context for implementation." },
10+
"repository": { "full_name": "hookdeck/hookdeck-cli" }
11+
}
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"action": "labeled",
3+
"issue": {
4+
"number": 192,
5+
"title": "Fix rule-filter-headers storing JSON as escaped string",
6+
"body": "When using --rule-filter-headers the value is stored as an escaped JSON string...",
7+
"labels": [{ "name": "automation/auto-implement" }]
8+
},
9+
"label": { "name": "automation/auto-implement" },
10+
"repository": { "full_name": "hookdeck/hookdeck-cli" }
11+
}
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"action": "submitted",
3+
"pull_request": {
4+
"number": 99,
5+
"body": "Closes #192",
6+
"head": { "ref": "auto-implement-issue-192" },
7+
"html_url": "https://github.com/hookdeck/hookdeck-cli/pull/99"
8+
},
9+
"review": {
10+
"body": "Please add a test for the new behavior.",
11+
"state": "changes_requested"
12+
},
13+
"repository": { "full_name": "hookdeck/hookdeck-cli" }
14+
}

0 commit comments

Comments
 (0)