Rebalance verdicts: PASS for healthy PRs, QUARANTINE for risk, BLOCK for major signals only

[VIVAAN-DHAWAN]

Why

Verdict feedback from dogfooding: healthy pull requests were routinely quarantined or blocked by signals that don't warrant a merge stop. Three rules over-fired:

1. Pending checks quarantined every PR analyzed before CI finished — the most common real-world state at review time.
2. Any dependency-manifest edit quarantined, including version bumps and lockfile refreshes.
3. A credential-like value anywhere hard-blocked — demonstrated when diffly's own Action BLOCKed Count diff content lines that look like ---/+++ headers #17 because that PR's regression fixture contains a fake postgresql://admin:hunter2@… test string.

New policy

Signal	Before	Now
Required checks failed	BLOCK	BLOCK (unchanged)
Credential-like value in production code	BLOCK	BLOCK (unchanged)
Credential-like value in tests/fixtures/docs	BLOCK	QUARANTINE (confirm it's intentional)
Newly added dependency (net of removals)	QUARANTINE	QUARANTINE (unchanged)
Version bump / lockfile refresh only	QUARANTINE	PASS + review note
Pending required checks	QUARANTINE	PASS + review note
Auth/security-sensitive code, DB/migrations	QUARANTINE	QUARANTINE (unchanged)
Missing obvious test coverage, unknown checks	PASS note	PASS note (unchanged)

Net-new dependency detection now subtracts removed packages from added ones (_dependency_base_name strips version constraints), so requests==2.31→2.32 passes while adding left-pad still quarantines.

Surface area

• src/diffly_cli/triage.py: verdict_for() rebalanced; _removed_dependency_names() added; flag codes/severity/JSON schema unchanged (only verdict routing and reasoning text moved).
• cli.py report policy footer + README "The verdict policy" table updated to match.
• CHANGELOG: three Changed bullets under Unreleased.

Validation

• Suite: 69 passed (six tests new or updated for the matrix), including:
  • pending checks alone → PASS with "required checks were still running" note
  • version-bump-only → PASS; net-new package → QUARANTINE
  • fake credential in tests/ → QUARANTINE; same string in production path → BLOCK
• Live end-to-end: the exact local-repo fixture that BLOCKed Count diff content lines that look like ---/+++ headers #17 now returns QUARANTINE with the confirm-it's-fake reasoning.
• Historical demo reports in demo/ intentionally left untouched (captured outputs).

[github-actions]

Diffly verdict: BLOCK

#20 · 6 files · 145 lines changed · checks: PENDING

Risk flags: EXPOSED_SECRET (critical), NO_TEST_COVERAGE (low), CHECKS_PENDING (medium)

Risk flags and reasoning

• EXPOSED_SECRET (critical): Adds a credential-like value in the changed code or configuration. — tests/test_triage.py
• NO_TEST_COVERAGE (low): Changed production files have no obvious neighboring or repository test coverage; this is a review hint, not a verdict gate. — src/diffly_cli/cli.py
• CHECKS_PENDING (medium): Required status checks are still pending. — combined commit status; diffly; test (3.10); test (3.11); test (3.12); test (3.13)
• BLOCK because the pull request appears to add a credential-like value.

Blast-radius summary

File	Change	Symbols	Direct callers	Tests
CHANGELOG.md	+6/-0	<top-level changes>	—	—
README.md	+3/-3	<top-level changes>	—	—
src/diffly_cli/cli.py	+1/-1	<top-level changes>	—	—
src/diffly_cli/triage.py	+38/-9	_test_files	—	tests/test_triage.py, tests/test_triage.py
tests/test_regressions.py	+8/-7	<top-level changes>	—	—
tests/test_triage.py	+64/-5	test_credential_in_tests_quarantines_instead_of_blocking, test_pending_checks_are_a_review_note_not_a_gate	—	—

Deterministic triage is authoritative; any optional LLM explanation cannot change the verdict.