Skip to content

Commit a82f0ba

Browse files
snapsynapseclaude
andcommitted
Iterate resolve-issue skill: internal consistency checks, stronger Claude rule
- Add Step 2 internal consistency check (gating vs availability table, talking point vs data, platform completeness, source URL health) - Remove aggressive "phrasing/terminology" blanket close heuristic - Make external research mandatory for all Claude/Anthropic issues - Add audit trail requirement for no-change closes - Flag phrasing differences that touch gating/pricing for investigation Based on learnings from resolving 25 issues: the skill was too quick to dismiss conflicts as "phrasing" and missed internal data contradictions (Vision gating=paid vs free availability table, Codex missing terminal/Linux). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 8c97c80 commit a82f0ba

1 file changed

Lines changed: 19 additions & 6 deletions

File tree

skills/resolve-issue/src/SKILL.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,29 +37,40 @@ For batch mode, list all open issues for the platform:
3737
gh issue list --repo snapsynapse/ai-capability-reference --state open --search "<platform>" --json number,title,labels
3838
```
3939

40-
### Step 2: Read current data
40+
### Step 2: Read current data and check internal consistency
4141

4242
Read the relevant `data/platforms/<platform>.md` file. Identify the feature section that matches the issue.
4343

44+
Before assessing the issue itself, scan the feature's data for internal contradictions — these are often more reliable indicators of errors than the model responses:
45+
- Does the **Gating** field match the **availability table**? (e.g., `Gating: paid` but Free tier shows ✅ means one of them is wrong)
46+
- Does the **talking point** match the actual gating/status? (e.g., talking point says "available on all plans" but Gating says `paid`)
47+
- Is the **platforms table** complete? Check whether major surfaces (terminal/CLI, Linux, API) are missing when they plausibly exist for this feature.
48+
- Do the **source URLs** still work? A 404 is worth noting even if it's not the issue being resolved.
49+
4450
### Step 3: Assess — apply these heuristics IN ORDER
4551

4652
**Close immediately as duplicate if:**
4753
- An older `verification-inconclusive` issue exists for the same feature AND a newer `verification-conflict` issue also exists. Close the older one with comment: "Superseded by #[newer]. Consolidating to the newer issue."
4854

4955
**Close with no data change (bump Checked date only) if:**
50-
- Both models agree on the facts but the "conflict" is about phrasing or terminology
5156
- One model (usually Perplexity) says "insufficient sources" but the other confirms our existing data is correct
5257
- The flagged change is an incremental UX improvement, not a change to status, gating, pricing tiers, platform availability, or regional availability
53-
- The issue asks about a feature that IS correctly reflected in our data
58+
- The issue asks about a feature that IS correctly reflected in our data AND the internal consistency check (Step 2) found no contradictions
5459

5560
**Research and update data if:**
61+
- The internal consistency check (Step 2) found contradictions in the data file — even if the models didn't flag them
5662
- A model reports a genuine change to: gating (free/paid), plan availability, platform support, status (ga/beta/preview/deprecated), or regional availability
5763
- Both models report something different from what our data says
5864
- One model makes a specific, sourced claim that contradicts our data
65+
- The models describe a "phrasing/terminology" difference but the underlying facts touch gating, pricing, or platform availability (e.g., "DALL-E 3 vs GPT-4o" may sound like naming but if one model says free and the other says paid, that's a real discrepancy worth investigating)
66+
67+
### Step 4: Research
68+
69+
**For Claude/Anthropic features: ALWAYS research externally.** You are Claude — you cannot objectively assess your own product's data. Every Claude issue must include at least one Perplexity search or official URL fetch, even if the issue looks like an obvious no-change close. This is non-negotiable.
5970

60-
### Step 4: Research (only if needed)
71+
**For all other platforms:** research when Step 3 indicates it's needed, or when the internal consistency check found issues.
6172

62-
When research is needed, check official sources first:
73+
When researching, check official sources first:
6374
1. Fetch the feature's official URL from the data file
6475
2. Use Perplexity via curl for broader search:
6576
```bash
@@ -134,8 +145,10 @@ After resolving, report to the user:
134145

135146
### Important rules
136147

137-
- **Never update Claude/Anthropic data based on your own knowledge.** Always verify against official sources. You ARE Claude — this is a conflict of interest.
148+
- **Never update Claude/Anthropic data based on your own knowledge.** You ARE Claude — every Claude issue requires external verification via Perplexity or official URL fetch. Even for no-change closes, you must show evidence from an external source. See Step 4.
149+
- **Always check internal consistency before assessing.** The most common real errors are contradictions within the data file itself (gating vs availability table, talking point vs actual data, missing platform rows). The model responses in the issue are a starting point, not the whole picture.
138150
- **Always present your assessment to the user before making changes** if the issue requires data updates. For no-change closes and duplicate closes, proceed directly.
139151
- **The talking point must be presenter-ready.** It's used in a classroom setting. Bold the key access/pricing info.
140152
- **Changelog entries are reverse chronological** — newest at top.
141153
- **Don't forget to update `last_verified` in the frontmatter** if you're updating any feature for that platform.
154+
- **When closing with no change, be specific about why.** Don't just say "data is correct." Name which fields you verified and what sources you checked. This creates an audit trail.

0 commit comments

Comments
 (0)