You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Iterate resolve-issue skill: internal consistency checks, stronger Claude rule
- Add Step 2 internal consistency check (gating vs availability table,
talking point vs data, platform completeness, source URL health)
- Remove aggressive "phrasing/terminology" blanket close heuristic
- Make external research mandatory for all Claude/Anthropic issues
- Add audit trail requirement for no-change closes
- Flag phrasing differences that touch gating/pricing for investigation
Based on learnings from resolving 25 issues: the skill was too quick to
dismiss conflicts as "phrasing" and missed internal data contradictions
(Vision gating=paid vs free availability table, Codex missing terminal/Linux).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: skills/resolve-issue/src/SKILL.md
+19-6Lines changed: 19 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,29 +37,40 @@ For batch mode, list all open issues for the platform:
37
37
gh issue list --repo snapsynapse/ai-capability-reference --state open --search "<platform>" --json number,title,labels
38
38
```
39
39
40
-
### Step 2: Read current data
40
+
### Step 2: Read current data and check internal consistency
41
41
42
42
Read the relevant `data/platforms/<platform>.md` file. Identify the feature section that matches the issue.
43
43
44
+
Before assessing the issue itself, scan the feature's data for internal contradictions — these are often more reliable indicators of errors than the model responses:
45
+
- Does the **Gating** field match the **availability table**? (e.g., `Gating: paid` but Free tier shows ✅ means one of them is wrong)
46
+
- Does the **talking point** match the actual gating/status? (e.g., talking point says "available on all plans" but Gating says `paid`)
47
+
- Is the **platforms table** complete? Check whether major surfaces (terminal/CLI, Linux, API) are missing when they plausibly exist for this feature.
48
+
- Do the **source URLs** still work? A 404 is worth noting even if it's not the issue being resolved.
49
+
44
50
### Step 3: Assess — apply these heuristics IN ORDER
45
51
46
52
**Close immediately as duplicate if:**
47
53
- An older `verification-inconclusive` issue exists for the same feature AND a newer `verification-conflict` issue also exists. Close the older one with comment: "Superseded by #[newer]. Consolidating to the newer issue."
48
54
49
55
**Close with no data change (bump Checked date only) if:**
50
-
- Both models agree on the facts but the "conflict" is about phrasing or terminology
51
56
- One model (usually Perplexity) says "insufficient sources" but the other confirms our existing data is correct
52
57
- The flagged change is an incremental UX improvement, not a change to status, gating, pricing tiers, platform availability, or regional availability
53
-
- The issue asks about a feature that IS correctly reflected in our data
58
+
- The issue asks about a feature that IS correctly reflected in our data AND the internal consistency check (Step 2) found no contradictions
54
59
55
60
**Research and update data if:**
61
+
- The internal consistency check (Step 2) found contradictions in the data file — even if the models didn't flag them
56
62
- A model reports a genuine change to: gating (free/paid), plan availability, platform support, status (ga/beta/preview/deprecated), or regional availability
57
63
- Both models report something different from what our data says
58
64
- One model makes a specific, sourced claim that contradicts our data
65
+
- The models describe a "phrasing/terminology" difference but the underlying facts touch gating, pricing, or platform availability (e.g., "DALL-E 3 vs GPT-4o" may sound like naming but if one model says free and the other says paid, that's a real discrepancy worth investigating)
66
+
67
+
### Step 4: Research
68
+
69
+
**For Claude/Anthropic features: ALWAYS research externally.** You are Claude — you cannot objectively assess your own product's data. Every Claude issue must include at least one Perplexity search or official URL fetch, even if the issue looks like an obvious no-change close. This is non-negotiable.
59
70
60
-
### Step 4: Research (only if needed)
71
+
**For all other platforms:** research when Step 3 indicates it's needed, or when the internal consistency check found issues.
61
72
62
-
When research is needed, check official sources first:
73
+
When researching, check official sources first:
63
74
1. Fetch the feature's official URL from the data file
64
75
2. Use Perplexity via curl for broader search:
65
76
```bash
@@ -134,8 +145,10 @@ After resolving, report to the user:
134
145
135
146
### Important rules
136
147
137
-
-**Never update Claude/Anthropic data based on your own knowledge.** Always verify against official sources. You ARE Claude — this is a conflict of interest.
148
+
-**Never update Claude/Anthropic data based on your own knowledge.** You ARE Claude — every Claude issue requires external verification via Perplexity or official URL fetch. Even for no-change closes, you must show evidence from an external source. See Step 4.
149
+
-**Always check internal consistency before assessing.** The most common real errors are contradictions within the data file itself (gating vs availability table, talking point vs actual data, missing platform rows). The model responses in the issue are a starting point, not the whole picture.
138
150
-**Always present your assessment to the user before making changes** if the issue requires data updates. For no-change closes and duplicate closes, proceed directly.
139
151
-**The talking point must be presenter-ready.** It's used in a classroom setting. Bold the key access/pricing info.
140
152
-**Changelog entries are reverse chronological** — newest at top.
141
153
-**Don't forget to update `last_verified` in the frontmatter** if you're updating any feature for that platform.
154
+
-**When closing with no change, be specific about why.** Don't just say "data is correct." Name which fields you verified and what sources you checked. This creates an audit trail.
0 commit comments