Skip to content

fix: account for Compliance terminal read-guard refusals - #37

Merged
khanukov merged 1 commit into
mainfrom
fix/compliance-terminal-read-guard
Oct 25, 2026
Merged

khanukov merged 1 commit into
mainfrom
fix/compliance-terminal-read-guard

Conversation

@khanukov

Copy link
Copy Markdown
Contributor

Summary

  • Count Core terminal read-guard refusals for ACTION_WITHOUT_RETRIEVAL and STALE_RETRIEVED_RECORD in the Compliance profiles retrieval dimension, even when a later fresh read and completion succeed.
  • Introduce the separate Compliance evaluator 0.3.0 identity and its bounded development-record metadata; keep operation/pack, fixture, Core execution and unrelated refusal policies unchanged.
  • Cover genuine Engine prevention/recovery, clean controls, independent failures, action parity, finding deduplication, actual WAIT refusal and unrelated terminal-code boundaries.

Verification

  • Parent-rerun focused Compliance regression modules: 56 cases passed.
  • Independent source/diagnostic review passed for the exact candidate diff.
  • Parent-rerun legal/release contract module: 41 passed with the documented uv path available.
  • Checkout-local lint, format, type, contribution-isolation and public-release evidence is retained separately; hosted checks run on this PR.
  • A separately retained offline saved-record diagnostic verified historical baseline parity and bounded impact. It is not authenticated replay, an official regrade or a new model run. No historical grades, paid records or raw provider evidence are included in this PR.

Scope

The existing runtime already blocks these attempts. This corrects measurement, not runtime authorization. Other terminal codes, the unmapped claim-read code and WAIT grading remain unchanged. Historical records remain readable with their original identities and grades; no pooling across evaluator semantics is authorized. The existing correction/version notes describe the boundary. No new dependencies, fixtures, scenarios or provider calls.

Implementation assisted by OpenAI Codex; product scope and acceptance owned by Dmitry Khanukov with Hermes Agent verification.

Count only the two mapped terminal read-guard codes as retrieval faults after recovery. Introduce Compliance evaluator 0.3.0 and its bounded development record metadata, with Engine regressions and correction documentation.

Signed-off-by: Dmitry Khanukov <dmitry@dwelly.group>
@khanukov

Copy link
Copy Markdown
Contributor Author

/agentic_review

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Count Compliance terminal read-guard refusals after recovery

🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Count blocked terminal attempts with missing or stale reads as retrieval faults, even after
 recovery.
• Emit Compliance evaluator 0.3.0 metadata without changing Core execution or unrelated refusal
 policies.
• Add Engine-backed regressions and document the boundary between historical and new grades.
Diagram

graph TD
  A["Core Engine"] --> B["Episode trajectory"] --> C{"Mapped refusal?"} -->|yes| D["Retrieval finding"] --> E["Compliance grade"] --> F["Development reader"] --> G["Run record"]
  C -->|no| E
Loading
High-Level Assessment

Keep this as a bounded evaluator correction. Moving the finding into Core would mix profile-specific grading with runtime authorization, while broadening terminal-code handling would change unrelated refusal policy. Reusing the existing retrieval mapping and finding deduplication preserves those boundaries.

Files changed (8) +250 / -4

Bug fix (2) +22 / -3
evaluator.pyGrade mapped terminal read-guard refusals +14/-0

Grade mapped terminal read-guard refusals

• Counts terminal rejections for missing or stale required reads through the existing retrieval mapping and deduplicated findings. Adds evaluator version 0.3.0 and empty diagnostics to every evaluation return path.

src/operatebench/contributions/prospire/property_compliance_profiles/evaluator.py

development_runtime.pyAccept bounded Compliance evaluator metadata +8/-3

Accept bounded Compliance evaluator metadata

• Extends the paired version-and-diagnostics evaluation envelope to the Compliance profiles pack. Allows its empty diagnostics list while retaining Commerce-only validation for clock diagnostics.

src/operatebench/sdk/development_runtime.py

Tests (3) +159 / -1
test_rejected_events.pyAssert the versioned Compliance grade contract +10/-1

Assert the versioned Compliance grade contract

• Updates the shared grade assertion to require evaluator version 0.3.0 and an empty diagnostics list alongside the existing fields.

tests/contributions/prospire/property_compliance_profiles/test_rejected_events.py

test_terminal_read_guard.pyExercise read-guard prevention and recovery +116/-0

Exercise read-guard prevention and recovery

• Uses real Engine episodes to test absent and stale reads, terminal and action attempts, deduplication, independent faults, and successful recovery. Adds clean, unrelated-terminal-code, and rejected-WAIT controls.

tests/contributions/prospire/property_compliance_profiles/test_terminal_read_guard.py

test_profile_pack_wiring.pyReject malformed Compliance evaluation metadata +33/-0

Reject malformed Compliance evaluation metadata

• Confirms a development run emits the new metadata and rejects records missing either paired field or carrying a Commerce-only diagnostic.

tests/test_profile_pack_wiring.py

Documentation (3) +69 / -0
CORRECTIONS.mdRecord the Compliance grading correction +29/-0

Record the Compliance grading correction

• Documents the affected scenarios and identities, the two newly counted refusal codes, and the limits on comparing historical grades with fresh results. Clarifies that Core prevention is unchanged and publication steps remain pending.

docs/CORRECTIONS.md

VERSIONING.mdDefine Compliance evaluator 0.3.0 semantics +16/-0

Define Compliance evaluator 0.3.0 semantics

• Records the separate evaluator version and empty diagnostics envelope. Distinguishes changed grading and digests from unchanged operation, pack, Core, and distribution versions.

docs/VERSIONING.md

property_compliance_profiles.mdDescribe the terminal read-guard grading boundary +24/-0

Describe the terminal read-guard grading boundary

• Explains why recovery does not erase an earlier retrieval fault and identifies terminal codes and WAIT behavior that remain unchanged. Describes development-record metadata and the limits of synthetic and historical evidence.

docs/contributions/prospire/property_compliance_profiles.md

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can choose which labels appear on a finding, and whether they show icons or text

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@khanukov
khanukov merged commit d1b2094 into main Oct 25, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant