SARIF rule severity should be configurable per finding
Problem
The two SARIF rules in src/navi_bootstrap/sarif.py (AUDIT_RULES
constant) hardcode defaultConfiguration.level: "warning":
AUDIT_RULES: tuple[dict[str, Any], ...] = (
{
"id": "pack-drift-missing",
...
"defaultConfiguration": {"level": "warning"},
},
{
"id": "pack-drift-changed",
...
"defaultConfiguration": {"level": "warning"},
},
)
SarifResult already carries a level: str = "warning" field that
nothing currently sets, so per-finding override exists in the
dataclass but is unreachable from audit_cmd.
This blocks legitimate severity expressiveness: a missing
SECURITY.md from the security-scanning pack should arguably
land as error, while stylistic README drift should be note.
SARIF supports both per-rule defaults and per-result overrides;
we expose neither.
Proposed fix
Two options, in order of decreasing scope:
A. Per-pack severity in the manifest schema (preferred)
Extend schema/manifest-schema.yaml so each template entry can
declare a severity: key (error | warning | note). Plumb
through RenderEntry → DiffResult → AuditFinding → SarifResult.level.
templates:
- src: SECURITY.md.j2
dest: SECURITY.md
severity: error # new field; defaults to "warning" if absent
B. CLI-level uniform override (cheaper, less expressive)
Add --severity {error,warning,note} to audit_cmd; applies to
all findings emitted by that run. Simpler but loses the
"missing SECURITY.md is more serious than missing README badge"
distinction.
Acceptance criteria
Out of scope (companion follow-ups, not bundled)
These were considered for inclusion but rejected per audit:
- Stdout audit-log line — stdout already emits the full report;
the report IS the log, so a redundant line on stderr would just
add CI noise. Skip unless concrete demand surfaces.
- "No drift" success message in --output file — empty SARIF
already serializes as a valid run with results: [], which any
spec-compliant consumer parses correctly. Spec-compliant; skip.
- CLI error-message hints with doc URLs — covered partially by
the existing "file an issue" hint added in 83d9376; if more is
needed it's a 5-minute follow-up to file separately.
- SARIF extensibility caveat in audit.md — once this issue lands,
the doc should describe the new level mechanism; a separate doc
issue isn't needed.
Effort
- Path A: ~2 hours (schema change + plumbing + tests + doc).
- Path B: ~30 minutes (CLI flag + tests).
Source
Bot review on PR #51 (Grippy sarif.py:23/50 MEDIUM/LOW
"SARIF rule severity is always 'warning'"). Originally drafted as
a polish bundle; sharpened post-Codex review which correctly
identified that the other "polish" items were either already
covered or bot-noise. This is the one item with genuine product
value.
SARIF rule severity should be configurable per finding
Problem
The two SARIF rules in
src/navi_bootstrap/sarif.py(AUDIT_RULESconstant) hardcode
defaultConfiguration.level: "warning":SarifResultalready carries alevel: str = "warning"field thatnothing currently sets, so per-finding override exists in the
dataclass but is unreachable from
audit_cmd.This blocks legitimate severity expressiveness: a missing
SECURITY.mdfrom thesecurity-scanningpack should arguablyland as
error, while stylistic README drift should benote.SARIF supports both per-rule defaults and per-result overrides;
we expose neither.
Proposed fix
Two options, in order of decreasing scope:
A. Per-pack severity in the manifest schema (preferred)
Extend
schema/manifest-schema.yamlso each template entry candeclare a
severity:key (error|warning|note). Plumbthrough
RenderEntry → DiffResult → AuditFinding → SarifResult.level.B. CLI-level uniform override (cheaper, less expressive)
Add
--severity {error,warning,note}toaudit_cmd; applies toall findings emitted by that run. Simpler but loses the
"missing SECURITY.md is more serious than missing README badge"
distinction.
Acceptance criteria
level != "warning".findings_to_sarif()andSarifResult.to_dict()faithfullypropagate the level.
partialFingerprints.primaryLocationLineHashis unchangedregardless of level (level is metadata; identity is rule+path).
when no severity is declared (default =
warning).tests/test_sarif.pycovering the level pass-through.Out of scope (companion follow-ups, not bundled)
These were considered for inclusion but rejected per audit:
the report IS the log, so a redundant line on stderr would just
add CI noise. Skip unless concrete demand surfaces.
already serializes as a valid run with
results: [], which anyspec-compliant consumer parses correctly. Spec-compliant; skip.
the existing "file an issue" hint added in
83d9376; if more isneeded it's a 5-minute follow-up to file separately.
the doc should describe the new level mechanism; a separate doc
issue isn't needed.
Effort
Source
Bot review on PR #51 (Grippy
sarif.py:23/50MEDIUM/LOW"SARIF rule severity is always 'warning'"). Originally drafted as
a polish bundle; sharpened post-Codex review which correctly
identified that the other "polish" items were either already
covered or bot-noise. This is the one item with genuine product
value.