aws-samples · neosun100 · Aug 4, 2026 · Aug 4, 2026
diff --git a/README.md b/README.md
@@ -13,7 +13,7 @@
   <img alt="python" src="https://img.shields.io/badge/python-3.10%2B-2997ff"/>
   <img alt="bedrock-agentcore" src="https://img.shields.io/badge/Amazon%20Bedrock-AgentCore%20Harness-ff9900"/>
   <img alt="version" src="https://img.shields.io/badge/version-0.4.0-2997ff"/>
-  <img alt="tests" src="https://img.shields.io/badge/offline%20tests-3711%20passing-1D8102"/>
+  <img alt="tests" src="https://img.shields.io/badge/offline%20tests-3715%20passing-1D8102"/>
   <img alt="coverage" src="https://img.shields.io/badge/coverage-90%25-1D8102"/>
   <img alt="milestones" src="https://img.shields.io/badge/milestones-M0--M15%20delivered-1D8102"/>
   <img alt="hardening" src="https://img.shields.io/badge/adversarial%20audit-100%20defects%20fixed-8b5cf6"/>
@@ -99,7 +99,7 @@ Honest build status per capability — mirrors the self-audit.
 | **Tools** | `nvd_lookup` / `epss_kev` / `attack_lookup` / `web_search` | 🟡 **reference stubs** (offline-safe, contract-tested) | `tools/`, `tests/test_tool_handlers.py` |
 | **Tools** | `siem_query` / `asset_lookup` / `enrich_ioc` / `ops_query` — backend-pluggable | 🟢 **built + tested** (offline mock default; `*_LIVE`=1 switches to a real stdlib-HTTP client — env-driven URL + bearer, timeouts, all failures→`upstream_error` with no silent fallback — proven end-to-end against an in-process 127.0.0.1 mock server, zero external network) | `tools/{siem_query,asset_lookup,enrich_ioc,ops_query}/`, `tests/test_*_live.py` |
 
-🟢 built & validated · 🟡 built, partial · 🟠 designed with loadable config · ⚪ design narrative only. **3711 offline tests pass** (+6 skipped when optional deps absent).
+🟢 built & validated · 🟡 built, partial · 🟠 designed with loadable config · ⚪ design narrative only. **3715 offline tests pass** (+6 skipped when optional deps absent).
 
 ## 🚀 Quickstart
 
@@ -247,7 +247,7 @@ Borrowed patterns (see [`docs/BLUEPRINT.md`](docs/BLUEPRINT.md)): supervisor→s
 | [`docs/GOVERNANCE.md`](docs/GOVERNANCE.md) | Registry dual-gate, HITL, sandbox hooks, and tag-guard controls |
 | [`docs/COMPLIANCE.md`](docs/COMPLIANCE.md) | Capability → SOC 2 / ISO 27001 / NIST CSF 2.0 control mapping (anchors machine-verified) |
 | [`docs/OBSERVABILITY.md`](docs/OBSERVABILITY.md) | Logging (`logutil`), metrics (token/latency/tool-call/error/eval), the OTEL/Transaction-Search path |
-| [`docs/TESTING.md`](docs/TESTING.md) | The 3711-test offline suite: layout, determinism, how to run |
+| [`docs/TESTING.md`](docs/TESTING.md) | The 3715-test offline suite: layout, determinism, how to run |
 | [`docs/FIDELITY-REPORT.md`](docs/FIDELITY-REPORT.md) | The self-audit — real vs. built vs. designed, with limits stated |
 | [`docs/ROADMAP.md`](docs/ROADMAP.md) | Delivered milestones (M0–M12) and what's next |
 | [**API reference (live)**](https://aws-samples.github.io/sample-sentinel-harness/) | Rendered `sentinel_harness` API docs (pdoc → GitHub Pages) |
@@ -285,7 +285,7 @@ sentinel-harness/
 ├── iac-cdk/              L3 CDK stacks (9; guardrail/identity/obs/vpc live) 🟢
 ├── iac-terraform/        deployable Terraform mirror (validate-clean)  🟢
 ├── docs/                 QUICKSTART · ARCHITECTURE · BLUEPRINT · SETUP · HARNESSES · GOVERNANCE · TESTING · FIDELITY-REPORT · ROADMAP
-├── tests/                offline unit + config tests (3711)     🟢
+├── tests/                offline unit + config tests (3715)     🟢
 └── .github/workflows/    CI incl. a customer-name / secret gate
 ```
 

diff --git a/docs/FIDELITY-REPORT.md b/docs/FIDELITY-REPORT.md
@@ -29,7 +29,7 @@ This is real, not narration:
 - **Layer 3 foundation ships runnable, tested code.** The dual-gate tool/skill registry, PreToolUse sandbox hook, and Agent Factory (fleet provision, dry-run, cross-env tag-guard) are built and unit-tested.
 - **Native IaC, not hand-rolled resources.** The gateway / registry / memory / harness stacks use the native `AWS::BedrockAgentCore::*` CloudFormation types. Per the README status matrix, the **Gateway and Memory CFN types are registered**; the Registry type is not yet in CFN (see limitations).
 - **Config path works.** `pip install -e .` succeeds (`[tool.setuptools] packages = ["sentinel_harness", "intake"]`), the `sentinel` console script works, and `sentinel create <harness.yaml>` loads real config via `sentinel_harness/loader.py` (systemPrompt file read, `bedrockModelConfig` / `agentCoreGateway` / `managedMemoryConfiguration` mapping, `${ENV}` expansion, `@gateway/tool` allowedTools grammar).
-- **Scale.** 3711 offline tests pass (+6 skipped when optional deps absent) across 137 test files, with 37 evidence JSON artifacts, 22 scenarios, 20 tools (incl. a 7-tool deterministic detection-engineering suite), an `iac-cdk` project (9 stacks synth-green) and an `iac-terraform` mirror (`validate`-clean).
+- **Scale.** 3715 offline tests pass (+6 skipped when optional deps absent) across 137 test files, with 37 evidence JSON artifacts, 22 scenarios, 20 tools (incl. a 7-tool deterministic detection-engineering suite), an `iac-cdk` project (9 stacks synth-green) and an `iac-terraform` mirror (`validate`-clean).
 - **Clean anonymization.** No real account IDs (only the `000000000000` placeholder), no customer or company names, no secrets. The CI secret-and-name scan is self-non-matching and fails the build on any hit.
 
 ## 4. Live controls retained for demos (us-east-1)

diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md
@@ -51,7 +51,7 @@ live) · 🟡 skeleton / partial · 🔴 gap.
 | `specialists/` | `cve-intel` (docker-build + live-validated on AgentCore Runtime) + `attack-mapper` / `threat-hunt` (real graph/plan builders) + `adversarial-reviewer` (agent_a2a + local_a2a + two-stage Dockerfile + contract test) | ✅ | all four specialists shipped |
 | `longrunning/` | `bas-runner` (BAS case-gen + detection-replay) + `detonation` (full simulated microVM lifecycle + orchestrator) | 🟩 | both built + tested; detonation stays an honest SIMULATED no-op |
 | `iac-cdk/lib/` | 9 synth-green stacks — `gateway` / `registry` / `memory` / `network` / `identity` / `guardrail` / `observability` / `harness` / `runtime` (+ `iam`); `iac-terraform/` mirror is `terraform validate`-clean | ✅ | `guardrail` / `identity` / `observability` LIVE-deployed (us-east-1); the Registry + `runtime` custom-resource/raw-CfnResource stacks synth clean but fail on deploy until their CFN types are GA (both control-plane APIs are separately live-verified — Registry via `registry_live.py`, `CreateAgentRuntime` via a real arm64 microVM that served a live A2A call, HTTP 200, real Bedrock model, on a non-prod test account, then torn down — `evidence/live_a2a_runtime_result.json`) |
-| `tests/` | 137 files, **3711 offline passing** (+6 skipped) | ✅ | add tests with each new module |
+| `tests/` | 137 files, **3715 offline passing** (+6 skipped) | ✅ | add tests with each new module |
 | `evidence/` | 37 evidence sets | ✅ | add one per milestone |
 
 ### 0.3 Fit score (vs. a full three-layer SecOps agent program)
@@ -181,8 +181,8 @@ Each milestone gives: **goal / files / reused APIs / acceptance (live evidence)
 Suggest one feature branch per milestone.
 
 ### M0 — Environment & baseline reproduction (half a day)
-**Goal:** on a fresh machine, get all 3711 offline tests green and reproduce ≥1 live scenario.
-- [ ] `uv sync` + `uv run pytest -q` → 3711 passing (+6 skipped) (offline).
+**Goal:** on a fresh machine, get all 3715 offline tests green and reproduce ≥1 live scenario.
+- [ ] `uv sync` + `uv run pytest -q` → 3715 passing (+6 skipped) (offline).
 - [ ] Configure `SENTINEL_EXECUTION_ROLE_ARN` / `SENTINEL_REGION` / `AWS_PROFILE` (non-prod) — see `docs/SETUP.md`.
 - [ ] Run `scenarios/scenario_cve_triage.py`; compare `evidence/cve_triage_result.json` shape.
 - [ ] Run `scenarios/scenario_hitl_resume.py`; reproduce pause→approve→resume.
@@ -419,7 +419,7 @@ hand-off reuses the live-capable M1/M2 engine (driven offline here, labeled a wi
       (`make deploy`, cost note, `make destroy`) + the no-lock-in export. — `docs/QUICKSTART.md`
 - [x] `tests/smoke/`: offline acceptance suite (default offline; `SENTINEL_SMOKE_LIVE=1` opt-in for live). — `tests/smoke/`
 
-**Acceptance:** `make test` → 3711 offline tests green; `make seed-registry` → dual-gate `ok`;
+**Acceptance:** `make test` → 3715 offline tests green; `make seed-registry` → dual-gate `ok`;
 `make create-harnesses` (DRY_RUN=1) → 8 harnesses validate offline with zero AWS; `sentinel export` → valid
 compilable Strands Python; `make smoke` → the offline acceptance suite green. A fresh non-prod account can then
 run `make deploy` (free-tier foundation) and the live scenarios; `make destroy` tears it all down.

diff --git a/tests/test_zz_process_isolation.py b/tests/test_zz_process_isolation.py
@@ -0,0 +1,130 @@
+"""
+Process-global state must not leak between tests.
+================================================
+Rounds 20-21 introduced three process-globals, and each is a way for one test to weaken a
+security check for every test that runs after it:
+
+    SENTINEL_PROMOTION_GATE_WITNESSED   (env)   INV-OPS-7 — a leak DISABLES the promotion
+                                                gate process-wide, which is the exact
+                                                failure the witness exists to prevent
+    the sentinel_harness.metric log handler     INV-METRIC-1 — a stacking bug duplicates
+                                                every emitted metric line
+    logger.propagate = False                    pre-existing; it silently broke a `caplog`
+                                                assertion in round 20 depending on whether
+                                                an earlier test had called
+                                                configure_logging()
+
+The file name starts with `zz_` so pytest collects it LAST in the default alphabetical
+order: these assertions are about the state the rest of the suite leaves behind, so running
+them first would prove nothing.
+
+Under `pytest-randomly` (this repo's default) collection order is shuffled, so these become
+a sample rather than a guarantee — still useful, and the `-p no:randomly` CI leg gives the
+ordered run. Recorded rather than worked around: an ordering-dependent check that pretends
+to be absolute is worse than one that says what it covers.
+
+A NOTE ON THE SECOND ASSERTION, which was wrong first
+-----------------------------------------------------
+The behavioural check originally read `assert out["ok"] is False`. With the witness
+deliberately leaked, that PASSED — for the wrong reason. The gate had correctly opened, the
+call proceeded to AWS, and it failed on `NoCredentialsError`, so `ok` was False with
+`error == "upstream_error"`. The environment assertion caught the planted leak; the
+behavioural one did not.
+
+So it now asserts the error KIND. Two assertions that look like they test the same thing
+can differ in strength, and the weaker one passing is how a real leak would ship.
+"""
+from __future__ import annotations
+
+import logging
+import os
+
+WITNESS_ENV = "SENTINEL_PROMOTION_GATE_WITNESSED"
+
+
+def test_the_promotion_witness_did_not_leak():
+    """A leaked witness means every later caller in this process can promote unattended."""
+    value = os.environ.get(WITNESS_ENV)
+    assert value is None, (
+        f"{WITNESS_ENV} leaked into the process environment as {value!r}. Some test set "
+        "it without cleanup, and INV-OPS-7's tool-side promotion gate is now open for "
+        "everything that runs afterwards. Use the `gate_witnessed` fixture (monkeypatch, "
+        "auto-reverted) or `agent_loop._with_promotion_witness`, which restores the prior "
+        "value in a `finally`."
+    )
+
+
+def test_the_promotion_gate_still_refuses_at_the_end_of_the_suite(monkeypatch):
+    """The behavioural half: ask the real tool after everything else has run.
+
+    Asserts the error KIND, not merely `ok is False`. With a leaked witness the gate opens,
+    the call reaches the control plane and returns `ok: False` with `upstream_error` — so
+    the loose version of this assertion passed while the gate was disabled.
+
+    The control plane is stubbed, deliberately. Positive-controlling this test surfaced a
+    real hazard in an earlier draft: with a leaked witness the gate opens and the handler
+    issues a genuine `CreateHarnessEndpoint`. It was rejected
+    (`UnrecognizedClientException`) only because the ambient credentials were invalid — on
+    a machine with working credentials, a test in a suite that advertises ZERO AWS would
+    have created a production endpoint. Stubbing makes that impossible rather than
+    unlikely.
+    """
+    import importlib.util
+    import pathlib
+
+    handler_path = (pathlib.Path(__file__).resolve().parent.parent
+                    / "tools" / "harness_ops" / "handler.py")
+    spec = importlib.util.spec_from_file_location("_isolation_harness_ops", handler_path)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+
+    class _RefuseToCall:
+        """Any control-plane call is a test failure, not a network round trip."""
+
+        def __getattr__(self, name):
+            def _boom(**_kwargs):
+                raise AssertionError(
+                    f"the promotion gate let a real control-plane call through: "
+                    f"{name}(...). The witness leaked."
+                )
+            return _boom
+
+    from sentinel_harness import core
+    monkeypatch.setattr(core, "_control", _RefuseToCall())
+
+    out = module.handler(
+        {"action": "promote_endpoint",
+         "params": {"harness_id": "h-isolation", "endpoint_name": "prod"}},
+        None,
+    )
+    assert out["ok"] is False, f"promotion was ALLOWED at end of suite: {out}"
+    assert out.get("error") == "validation_error", (
+        f"the promotion was refused, but not BY THE GATE — error={out.get('error')!r}, "
+        f"message={str(out.get('message'))[:160]!r}. An `upstream_error` here means the "
+        "gate OPENED and the call reached the control plane, i.e. the witness leaked and "
+        "only the absence of credentials stopped it."
+    )
+    assert "human-approval gate" in str(out.get("message")), out.get("message")
+
+
+def test_the_metric_handler_did_not_stack():
+    """`logutil.get_metric_sink` is idempotent by design (one tagged handler). A stacking
+    regression would duplicate every metric line, which reads as double the token spend."""
+    logger = logging.getLogger("sentinel_harness.metric")
+    tagged = [h for h in logger.handlers
+              if getattr(h, "_sentinel_metric_handler", False)]
+    assert len(tagged) <= 1, (
+        f"{len(tagged)} tagged metric handlers are attached: {tagged}. "
+        "get_metric_sink must reuse the one handler, not append another."
+    )
+
+
+def test_no_test_left_the_repo_registry_path_redirected():
+    """`SENTINEL_REGISTRY_PATH` overrides the governance registry. Left set, it points every
+    later caller — including the MCP server's approved-set load — at whatever file the last
+    test used."""
+    value = os.environ.get("SENTINEL_REGISTRY_PATH")
+    assert value is None, (
+        f"SENTINEL_REGISTRY_PATH leaked as {value!r}; the registry every later test reads "
+        "is whatever that path holds. Set it with monkeypatch.setenv, never os.environ."
+    )