Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions SKILL_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,10 @@ Hard rules only — falsifiable and enforceable. No "consider" / "may" language.
## 4. Remediation

What the agent emits or changes when this fires. Keep complex logic in a
reference/script file (§7), not inline. If this skill can modify code or
configuration, classify each remediation path using the repo-level
`docs/fixer-policy.md` before applying changes.
reference/script file (§7), not inline. Every fix recommendation must include
remediation guidance, confidence, blast radius, and behavior-change risk. If this
skill can modify code or configuration, classify each remediation path using the
repo-level `docs/fixer-policy.md` before applying changes.

**Before (vulnerable):**
```
Expand All @@ -78,6 +79,16 @@ configuration, classify each remediation path using the repo-level
<what the agent should produce>
```

**Fix recommendation output:**
```yaml
remediations:
- guidance: "<concrete remediation steps or patch guidance>"
confidence: high # low | medium | high
blast_radius: "<affected files, users, systems, integrations, or workflows>"
behavior_change_risk: medium # low | medium | high
test_strategy: "<optional validation approach>"
```

## 5. Verification (falsifiable)

The skill is not "done" until this passes — binary, not aspirational.
Expand Down Expand Up @@ -133,6 +144,7 @@ skills/<domain>/<skill-name>/
- [ ] At least one machine-matchable detection signal (regex / structural)
- [ ] Rules are hard constraints (no "consider"/"may")
- [ ] Before/after remediation example present
- [ ] Every fix recommendation includes `guidance`, `confidence`, `blast_radius`, and `behavior_change_risk`
- [ ] Falsifiable verification test defined (binary pass/fail)
- [ ] Gotchas: ≥2 false positives + ≥1 precision trap
- [ ] `SKILL.md` stays lean; long detail moved to reference files
Expand Down
16 changes: 16 additions & 0 deletions docs/remediation-output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Remediation Output Fields

SecuritySkills fix recommendations should be emitted as structured remediation
items under each finding. Every remediation item must include:

- `guidance`: concrete remediation steps or patch guidance.
- `confidence`: `low`, `medium`, or `high` confidence that the recommendation is correct for the observed evidence.
- `blast_radius`: the expected scope of systems, files, users, integrations, data, or workflows affected by the change.
- `behavior_change_risk`: `low`, `medium`, or `high` risk that the fix changes intended behavior.

`test_strategy` is optional in the normalized schema. Include it when the skill
can name a concrete validation approach, but keep detailed test strategy policy
in the dedicated test-strategy workstream.

The machine-readable contract lives in
[`schemas/finding.schema.json`](../schemas/finding.schema.json).
114 changes: 114 additions & 0 deletions schemas/finding.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/UnitOneAI/SecuritySkills/schemas/finding.schema.json",
"title": "SecuritySkills Normalized Finding Output",
"description": "Machine-readable contract for normalized security findings and remediation recommendations emitted by SecuritySkills workflows.",
"type": "object",
"additionalProperties": false,
"required": ["findings"],
"properties": {
"findings": {
"type": "array",
"items": {
"$ref": "#/$defs/finding"
},
"description": "Normalized findings produced by a skill run."
}
},
"$defs": {
"riskLevel": {
"type": "string",
"enum": ["low", "medium", "high"]
},
"nonEmptyString": {
"type": "string",
"minLength": 1
},
"finding": {
"type": "object",
"additionalProperties": false,
"required": ["id", "title", "severity", "evidence", "remediations"],
"properties": {
"id": {
"$ref": "#/$defs/nonEmptyString",
"description": "Stable finding identifier within the output."
},
"title": {
"$ref": "#/$defs/nonEmptyString"
},
"severity": {
"type": "string",
"enum": ["info", "low", "medium", "high", "critical"]
},
"cwe": {
"$ref": "#/$defs/nonEmptyString",
"description": "Optional CWE mapping, when applicable."
},
"framework": {
"$ref": "#/$defs/nonEmptyString",
"description": "Optional framework/control reference from the skill frontmatter frameworks."
},
"evidence": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["location", "summary"],
"properties": {
"location": {
"$ref": "#/$defs/nonEmptyString"
},
"summary": {
"$ref": "#/$defs/nonEmptyString"
}
}
}
},
"remediations": {
"type": "array",
"minItems": 1,
"items": {
"$ref": "#/$defs/remediation"
},
"description": "Every fix recommendation for this finding. Each item must include remediation guidance, confidence, blast radius, and behavior-change risk."
}
},
"anyOf": [
{
"required": ["cwe"]
},
{
"required": ["framework"]
}
]
},
"remediation": {
"type": "object",
"additionalProperties": false,
"required": ["guidance", "confidence", "blast_radius", "behavior_change_risk"],
"properties": {
"guidance": {
"$ref": "#/$defs/nonEmptyString",
"description": "Concrete remediation steps or patch guidance for the finding."
},
"confidence": {
"$ref": "#/$defs/riskLevel",
"description": "How confident the skill is that this recommendation is correct for the observed evidence."
},
"blast_radius": {
"$ref": "#/$defs/nonEmptyString",
"description": "Expected scope of systems, files, users, integrations, data, or workflows affected by the change."
},
"behavior_change_risk": {
"$ref": "#/$defs/riskLevel",
"description": "Risk that the remediation changes intended application or operational behavior."
},
"test_strategy": {
"$ref": "#/$defs/nonEmptyString",
"description": "Optional validation strategy for the remediation. Kept optional because test strategy requirements are defined separately."
}
}
}
}
}
Loading