Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,15 @@ If your contribution includes skill test fixtures, also run:
ruby scripts/test_skill_fixtures.rb
```

### Normalized JSON output

Every skill must be able to emit findings as normalized JSON that validates
against [schemas/finding.schema.json](schemas/finding.schema.json). The required
top-level envelope and field semantics are documented in
[docs/normalized-json-output.md](docs/normalized-json-output.md). Existing
human-readable report sections may remain, but machine-readable finding output
must use the normalized contract.

---

## Getting Started
Expand Down
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,14 @@ Validate skill fixture manifests and expected evidence strings with:
ruby scripts/test_skill_fixtures.rb
```

### Normalized finding JSON

Every skill must be able to emit findings as normalized JSON that validates
against [`schemas/finding.schema.json`](schemas/finding.schema.json). The
top-level envelope, required run/skill metadata, finding fields, evidence,
framework/CWE references, remediation fields, and test strategy requirements are
documented in [`docs/normalized-json-output.md`](docs/normalized-json-output.md).

### Progressive disclosure (keep `SKILL.md` lean)

Claude's skill guidance: when a `SKILL.md` would exceed ~500 lines, **don't inline everything** — split detail into sibling reference files in the same directory and link to them from `SKILL.md`. The agent loads a reference only when it needs it, so the entrypoint stays cheap to load.
Expand Down Expand Up @@ -241,7 +249,7 @@ Pre-configured skill sequences for common security roles. Each bundle orchestrat
## What Makes This Different

- **Framework-grounded.** Every skill cites real control IDs from OWASP, NIST, MITRE ATT&CK, or CIS. No invented controls. No hallucinated references.
- **Consistent output format.** Structured findings with severity, CWE mapping, framework reference, evidence, and remediation -- every time.
- **Consistent output format.** Structured findings with severity, CWE mapping, framework reference, evidence, remediation, and normalized JSON -- every time.
- **AI-security skills that don't exist elsewhere.** OWASP LLM Top 10, Agentic AI security, prompt injection testing, model supply chain review.
- **Multi-agent compatible.** Same skill file works with Claude Code, Gemini CLI, Cursor, Codex CLI, OpenClaw, and Kiro.
- **Prompt-injection hardened.** Every skill reviewed against OWASP LLM01:2025. CI scans for injection patterns on every PR.
Expand Down
7 changes: 7 additions & 0 deletions SKILL_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@ modify code or configuration, classify each remediation path using the repo-leve
request approval when the policy requires it, and include review evidence plus
rollback guidance in the handoff.

When machine-readable output is requested, findings MUST be available as JSON
that validates against the repo-level normalized contract:
[`schemas/finding.schema.json`](schemas/finding.schema.json). See
[`docs/normalized-json-output.md`](docs/normalized-json-output.md) for
the top-level envelope and required fields.

**Before (vulnerable):**
```
<minimal vulnerable example>
Expand Down Expand Up @@ -158,6 +164,7 @@ skills/<domain>/<skill-name>/
- [ ] Frontmatter complete; `name` matches the directory
- [ ] Every framework ID is real and resolves (no invented control numbers)
- [ ] At least one machine-matchable detection signal (regex / structural)
- [ ] Findings can be emitted as normalized JSON per `schemas/finding.schema.json`
- [ ] Rules are hard constraints (no "consider"/"may")
- [ ] Before/after remediation example present
- [ ] Every fix recommendation includes `guidance`, `confidence`, `blast_radius`, `behavior_change_risk`, and `test_strategy`
Expand Down
156 changes: 156 additions & 0 deletions docs/normalized-json-output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Normalized JSON Output Contract

SecuritySkills can emit machine-readable findings as a normalized JSON envelope
that validates against [`schemas/finding.schema.json`](../schemas/finding.schema.json).
This contract is independent of SARIF. Downstream systems may map it to SARIF,
ticketing systems, GRC platforms, dashboards, or vulnerability stores without
changing individual skill output rules.

## Envelope

Every JSON response uses this top-level shape:

```json
{
"schema_version": "1.0.0",
"run": {
"id": "run-2026-06-16T12:00:00Z",
"timestamp": "2026-06-16T12:00:00Z",
"tool": "codex",
"target": "github.com/example/service",
"source_ref": "commit-or-build-id"
},
"skill": {
"name": "secure-code-review",
"version": "1.0.0",
"path": "skills/appsec/secure-code-review/SKILL.md",
"frameworks": ["OWASP-ASVS-4.0.3", "CWE"]
},
"findings": []
}
```

- `schema_version` is fixed at `1.0.0` until a breaking contract change is
required.
- `run.id` is the deduplication boundary for one execution. Use an orchestrator
ID, CI job ID, or generated run ID.
- `run.timestamp` should be an ISO 8601 timestamp.
- `run.target` identifies the reviewed repository, project, service, artifact,
environment, or evidence package.
- `skill.name`, `skill.version`, and `skill.frameworks` come from the skill's
`SKILL.md` frontmatter.

## Findings

Each finding must include:

- `id`: stable finding ID within the run.
- `title`: concise finding title.
- `severity`: one of `info`, `low`, `medium`, `high`, or `critical`.
- `status`: one of `open`, `mitigated`, `accepted_risk`, or `false_positive`.
- `evidence`: one or more concrete evidence entries.
- `remediations`: one or more remediation recommendations, each with a test
strategy.

Each finding must include at least one framework/CWE mapping:

- `cwe`: array of CWE IDs such as `CWE-89`, when applicable.
- `framework_refs`: array of framework/control references from the skill's
declared frameworks.

Optional fields such as `fingerprint`, `description`, and `references` support
enterprise deduplication, analyst context, and external advisory linking.

## Evidence

Evidence entries must identify where the issue was observed and summarize the
observation. Locations may be source paths, line ranges, cloud resource IDs, log
sources, policy paths, scanner result identifiers, or evidence package records.

Use `snippet` only for the minimal redacted text needed to prove the finding.
Set `redacted: true` when secrets, tokens, personal data, internal hostnames, or
other sensitive values were removed.

## Remediation And Tests

Each remediation item must include:

- `guidance`: concrete remediation steps or patch guidance.
- `confidence`: `low`, `medium`, or `high`.
- `blast_radius`: expected affected files, systems, users, integrations, data,
or workflows.
- `behavior_change_risk`: `low`, `medium`, or `high`.
- `test_strategy`: validation that proves the issue is fixed.

`test_strategy` must include a `summary` and at least one of
`recommended_tests` or `generated_tests`. See
[`docs/remediation-output.md`](remediation-output.md) for remediation-specific
field guidance.

## Minimal Example

```json
{
"schema_version": "1.0.0",
"run": {
"id": "run-001",
"timestamp": "2026-06-16T12:00:00Z",
"tool": "codex",
"target": "payments-api",
"source_ref": "abc1234"
},
"skill": {
"name": "api-security",
"version": "1.0.0",
"path": "skills/appsec/api-security/SKILL.md",
"frameworks": ["OWASP-API-Top-10-2023", "CWE"]
},
"findings": [
{
"id": "API-SEC-001",
"fingerprint": "api-security:users-delete:missing-admin-auth",
"title": "Administrative endpoint lacks authorization check",
"description": "The delete-user route accepts authenticated requests without verifying administrative privileges.",
"severity": "high",
"status": "open",
"cwe": ["CWE-862"],
"framework_refs": [
{
"framework": "OWASP-API-Top-10-2023",
"control": "API5:2023",
"name": "Broken Function Level Authorization"
}
],
"evidence": [
{
"location": "routes/users.js:42",
"artifact_type": "source",
"summary": "DELETE /users/:id checks authentication but not administrator role.",
"snippet": "router.delete('/users/:id', requireAuth, deleteUser)",
"redacted": false
}
],
"remediations": [
{
"guidance": "Require an administrator role check before invoking deleteUser.",
"confidence": "high",
"blast_radius": "User administration routes only.",
"behavior_change_risk": "medium",
"test_strategy": {
"summary": "Proves non-admin users cannot delete accounts while admins still can.",
"recommended_tests": [
{
"name": "Reject non-admin delete",
"type": "integration",
"purpose": "Confirms the vulnerable authorization bypass no longer succeeds.",
"command": "npm test -- users-authz.test.js",
"expected_result": "Non-admin DELETE /users/:id returns 403 and admin DELETE succeeds."
}
]
}
}
]
}
]
}
```
6 changes: 4 additions & 2 deletions docs/remediation-output.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Remediation Output Fields

SecuritySkills fix recommendations should be emitted as structured remediation
items under each finding. Every remediation item must include:
items under each finding in the normalized JSON envelope documented in
[`docs/normalized-json-output.md`](normalized-json-output.md). Every remediation
item must include:

- `guidance`: concrete remediation steps or patch guidance.
- `confidence`: `low`, `medium`, or `high` confidence that the recommendation is correct for the observed evidence.
Expand Down Expand Up @@ -44,5 +46,5 @@ remediations:
expected_result: "All login redirect regression examples pass."
```

The machine-readable contract lives in
The machine-readable JSON schema lives in
[`schemas/finding.schema.json`](../schemas/finding.schema.json).
Loading
Loading