feat(reports): vulnerability_management evidence report (#5816) (#5784 W04)

[ToddHebebrand]

W04 of #5812 / #5784 — the vulnerability_management report type, end to end.
The spec's own worked example (#5573 §15 points a vulnerability-management
deliverable at "the org's vulnerability report definition") now has a report to
point at.

Closes #5816

Spec: docs/superpowers/specs/billing/2026-09-14-service-plan-evidence-reports-spec.md §3.4
Plan: docs/superpowers/plans/billing/2026-09-14-service-plan-evidence-reports-w04-vulnerability-management.md
Plan index: docs/superpowers/plans/billing/2026-09-14-service-plan-evidence-reports.md

Depends on W01 only (#5813, merged as eabb9eb48). Independent of W02/W03/W06.

Relationship to security_compliance_posture

security_compliance_posture keeps its single vulnerability control line;
this is the detail artifact — findings, exceptions and a remediation
ranking. Neither replaces the other.

The shared loader securityComplianceReportVulnerabilities.ts was extended,
not forked: the catalog join and the throw-on-incomplete-catalog rule now live
in one private loadVulnerabilityCatalog, which both loadOpenVulnerabilityCounts
(posture, signature and behaviour unchanged — its tests pass unedited) and
the new loadOpenFindings call. Integration Case 4 generates a posture run and
this report for the same seeded org and asserts the critical/high numbers agree;
if a later change forks the loader, that case goes red. (verified)

What the artifact leads with

• Feed freshness, first. Per-source last_sync_at is on the cover. A report
  that under-counts because MSRC has not synced in a week says so instead of
  printing a reassuring number.
• Unmeasured ≠ zero. When no source has ever synced, every count is null,
  the PDF prints N/A - not measured, and the reason lands in dataGaps.
  Integration Case 3 asserts null, not 0. (verified)
• KEV and high-EPSS are counted separately from severity and are never
  filtered by severityFloor — an actively exploited medium outranks a
  theoretical critical, and the options-form help line says so explicitly.
• Exceptions expiring next period, the section a vCISO buys the deliverable
  for. The next period is derived from the deliverable's own cadence
  (recurrence.ts month-end semantics), never "period end + 30 days"; a
  quarterly deliverable is pinned by a unit test, a monthly one by integration
  Case 2. An ad-hoc staff run with no occurrence falls back to one calendar
  month and discloses that in dataGaps. (verified)

Portal

Label only — no generate button. The type stays out of PORTAL_REPORT_TYPES
and out of both portal-user allowlists in reportGenerationService.ts (OD-10 = A).
PortalRunDto['type'] and the portal ReportRunList union are widened (the type
lie the compiler could not see, since portalRunListPredicate has no type
filter); ReportRunList now separates ReportType (what a row can carry) from
GeneratableReportType (what has a button), so a later wave has to choose a side
rather than inherit a button by accident. OD-12 gates visibility on delivery —
integration Case 5 asserts invisible + renderRunPdf refusal before delivery,
listed + downloadable after.

The silent-failure this wave specifically guards

A type with no buildReportPdf arm falls through to renderGenericReport,
producing a plausible-looking PDF with the entire designed summary — exceptions
section included — dropped, and no unit test catches it. All three render
paths are now proven against the one stored snapshot: (verified)

Path	Proof
portal / server (renderRunPdf)	vulnerabilityManagementEvidence.integration.test.ts Case 5 asserts the produced PDF's text contains the exceptions section
staff / browser (exportReport)	reportExport.vulnerabilityManagement.test.tsx
scheduled email	same buildReportPdf opts — reportPdf.vulnerabilityManagement.test.ts covers the arm and the generic fall-through

Rollout

• No M365_TENANT_SYNC_ENABLED dependency. Unlike W03, W05 and W06 this wave
  reads no M365 table; device_vulnerabilities / vulnerabilities are populated
  by the agent and the NVD/MSRC/Apple/OSV feeds regardless of that flag. Do not
  gate this release on a flag it does not need.
• No feature flag, no env var, no consent change, no agent change.
• This release must run pnpm --filter @breeze/api reports:reprovision-portal-definitions
  (dry run, then --apply) — orgs that enabled portal reports earlier will
  otherwise lack the new definition. (Not required for correctness of the
  evidence path: resolveManagedEvidenceDefinition creates it on first template
  apply.) (inferred — the script was not run here)
• Migration 2026-10-17-110600-report-type-vulnerability-management.sql is an
  ALTER TYPE ... ADD VALUE IF NOT EXISTS alone in its file, writes no rows, and
  is not added to migrationRlsScope.test.ts's baseline. Applied twice
  against a live DB: second run a clean no-op, db:check-drift clean. (verified)
• No new table, no new column, no RLS change — so the cascade / export-policy
  registrations are untouched (tenantCascade + tenant-export-policy
  integration suites re-run green). (verified)

Localization caveat

The eight locale files localize the web UI only; the PDF renderer is English.

Verification

All run in the foreground on this branch. (verified)

Check	Result
apps/api full unit suite	2406 files / 45,331 passed, 3 files + 22 tests skipped
packages/shared full suite	130 files / 2,994 passed
apps/web full suite	969 files / 10,719 passed
apps/portal full suite	71 files / 626 passed
tsc --noEmit — api, shared, web, portal	clean
astro check (web)	0 errors, 0 warnings
pnpm lint	6/6 packages clean
pnpm db:check-drift	clean
vulnerabilityManagementEvidence.integration.test.ts	5 passed on a live stack (non-zero count confirmed; a deliberate mutation of Case 4's parity assertion was confirmed to fail before being reverted)
tenantCascade + tenant-export-policy + orgLifecycleFoundations + managedEvidenceFoundations + portalReportSelfService integration	23 passed
securityComplianceReportVulnerabilities.test.ts + posture suites	green, unedited

Test stack torn down (pnpm test-stack down, verified no Breeze compose project left running).

Shared-file edits (all additive, one entry each)

db/schema/reports.ts, reportGenerationService.ts (+test), managedEvidenceRegistry.ts,
validators/deliverableTemplates.ts, routes/reports/schemas.ts (+test),
securityComplianceReportVulnerabilities.ts (+test), reportPdf.ts + index.ts,
types/index.ts, utils/index.ts, portal/reportsSelfService.ts (+test),
types/portalVisibility.ts, portal ReportRunList.tsx (+test), six web report
components, apps/web/src/locales/*/reports.json.

Three pre-existing assertions were bumped by exactly this wave's addition, not
loosened: the portal provisioning list (3 → 4 definitions), the curated
template-card count (6 → 7 buttons), and W01's deliberate
auto-evidence-empty empty-state assertion, which W01 wrote for the first wave
to flip.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying breeze with    Cloudflare Pages

Latest commit:	c81c939
Status:	✅  Deploy successful!
Preview URL:	https://da76f0f2.breeze-9te.pages.dev
Branch Preview URL:	https://feature-5812-service-plan-ev-dqdk.breeze-9te.pages.dev

View logs

[ToddHebebrand]

Review run: /pr-review-toolkit:review-pr — code-reviewer, silent-failure-hunter, pr-test-analyzer (3 agents, parallel, on 8c84e405).

Findings: 3 consequential raised → all 3 addressed in c9ea189 + f5845d8; 0 outstanding. Both hunters independently raised the same top two, which is why they're treated as confirmed rather than plausible.

1. closedThisPeriod.count ignored the unmeasured gate (silent-failure-hunter + code-reviewer). With no feed ever synced the artifact printed "0 finding(s) were remediated between …" directly beside "N/A - not measured" for the same data source — a definite claim about data we had never read, in the one report type whose reason to exist is not making that claim. Now gated exactly as open is. Regression test added; a deliberate revert of the gate was confirmed to fail it.
2. Restricted-to-zero-sites returned an unshaped result (both). zeroSafeReport's arm handed back emptyRowsReport() with no summary, so buildReportPdf fell through to renderGenericReport and printed "No data available for the selected filters." — indistinguishable from "we checked every device and found none" for a technician whose site grants are empty. Both that arm and the generator's own branch now share one emptyVulnerabilityManagementSummary (nulls + a stated reason in dataGaps).
3. A merged exception row could splice two devices' records (code-reviewer). groupExceptions fixed status from the first-read finding but later overwrote acceptedBy/acceptedUntil from a different device's finding when that one expired sooner — regardless of its status. A CVE accepted on device A until December and mitigated on device B until October rendered as "accepted, by Bob, until October": a record neither device holds, in the section a vCISO reads to decide what to re-approve. Status, approver and expiry now move together. Regression test added, with a mutation control confirming it fails without the fix.

Also closed the test-coverage gaps the analyzer named: mitigated vs accepted exception classification (previously zero fixtures anywhere in the stack), the missing-deliverable-cadence fallback disclosure, the below-floor-but-patchable finding still ranking for remediation, and the topN 1–500 schema bounds for an API caller bypassing the form.

Clean on review: per-branch site scoping (verified non-vacuous — the assertion's needle is a UUID, not a matchable enum value); loadVulnerabilityCatalog's system-context elevation (the vulnerabilities / vulnerability_sources catalogs carry no RLS, confirmed against 2026-06-22-vulnerability-management.sql); the posture control line unchanged by the loader extension; migration naming/idempotency; additive-only registry edits; runAction on the web mutation path; genuine (spot-checked) translations in all eight locales.

Tests (re-run in full after the fixes): apps/api 45,338 passed / 2,406 files; packages/shared 2,996 passed; apps/web 10,721 passed; apps/portal 626 passed. tsc --noEmit clean for api/shared/web/portal, astro check 0 errors, pnpm lint 6/6, pnpm db:check-drift clean. Integration on a live stack: vulnerabilityManagementEvidence 5 passed plus managedEvidenceFoundations / tenantCascade / tenant-export-policy — 19 passed, non-zero counts confirmed. Stack torn down.

Status: review-clean, awaiting maintainer merge. Note the rollout item in the PR body — this release should run reports:reprovision-portal-definitions, and it has no M365_TENANT_SYNC_ENABLED dependency.

[ToddHebebrand]

Closing and reopening to trigger a pull_request CI run on the current head f5845d8 — the two review-fix pushes produced no run (last run is on 8c84e40). Orchestrator action, no code change.

[ToddHebebrand]

Merged main (W02 #6006 and W03 #6007 landed first, same report-type registries). 27 conflicts resolved as unions — threat_detection_review, endpoint_management_review, then vulnerability_management in every enum/registry/switch/render arm; ReportTemplates.savedReportMerge.test.tsx count 7/8 → 9 (8 curated cards + hardware-lifecycle's second saved report); portal ReportRunList kept main's narrow-allowlist GeneratableReportType design and extended it. tsc clean in shared/api/web; portal tsc has one pre-existing error from main (filed separately). Targeted tests: shared 81, api 143, web 120, portal 16 — all green. Head c81c939; CI hand-dispatched (this branch produces no pull_request runs), enqueue-on-green watcher armed.