Add NOOA and OpenShell delegated-authority reference - #61
Draft
chuks wants to merge 8 commits into
Draft
Conversation
A working reference for NOOA and NVIDIA's open agent-security stack. A NOOA agent presents a Ratify delegation over MCP, through an OpenShell policy, to an independent receiver that makes the only authorization decision. The agent's own process contains no authorization logic, and a test asserts its absence. Contents: * demos/nvidia-nooa-delegated-authority: the receiving service, the MCP server and clients, the NOOA presentation adapter, and a live OpenShell profile that drives 52 cases in seven groups and writes a machine-readable artifact. Each case declares its expected outcome and every boundary delta it may produce, and is judged against control-plane snapshots taken either side of it. * Hermetic suites covering receiver security, MCP transport carriage, the profile's adjudicator, and the released NOOA middleware API. * scripts/nvidia-reference-check.sh, the authoritative gate. It owns the environment and the pins, requires the optional integrations rather than permitting them to skip, and fails on any skip or unexpected count. An ordinary pytest run on Python 3.11 reported "91 passed, 1 skipped" and exited zero while an entire module had not run. * sandbox-requirements.lock, a hash-pinned dependency set for the sandbox image, so its staged dependency tree is byte-identical between clean builds. Pinning only the top-level packages left the transitive set floating, and one transitive version moved between two runs an hour apart. * A CI job that runs the authoritative gate on Python 3.12. The live profile needs a container runtime and is deliberately not part of the hermetic claim. The accompanying documentation states what has been executed and what has not. Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
Three gaps in the reference's own honesty, and the tests that keep them closed. Driver-level failure accounting. A run in which the entire unified group produced no result failed its four case gates and still reported no driver errors, so driver_reported_no_errors was a weaker claim than its name. The driver now records a structured error for a non-zero or unlaunchable operation, for a missing, unparseable, incomplete or misattributed result, for a partial suite, a missing subcase, an unknown group, and a stale or absent snapshot pair. Case verdicts and driver errors are complementary rather than alternatives. Error text carries no captured subprocess output, so nothing from a presentation leaves in an artifact. Parser coverage. The parser matrix tolerates a policy denial by design, because a policy layer is allowed to be stricter than the server. That makes the safety invariant, admitted as X and never dispatched as Y, satisfiable by a policy that refuses everything: removing one allow rule left all fifteen parser verdicts passing while nothing had been parsed. A separate coverage gate now requires each probe to take the branch it is expected to take, stated per case so that two probes swapping branches cannot cancel out. Presenter clock safety margin. The sandbox-side client is the only presenter that signs inside the sandbox, so its challenge_at comes from the container's clock while the receiver verifies against the host's. SPEC section 10 requires a non-negative challenge age, correctly, since a clock-skew attacker is in the threat model. A container clock that leads the host's is therefore refused as stale_challenge with a negative age, and one concurrent run was refused exactly that way. The client now backdates its own timestamp by two seconds, chosen from a measured container lead of at most 0.228s plus headroom, which is under one percent of the 300 second freshness window. The verifier is unchanged, the timestamp is still generated by the presenter and covered by the agent's signature, and no refusal is retried. A preflight measures the offset through the same interpreter the client uses, records it in the artifact, and fails fast with a clock-discipline error if the lead exceeds the margin. The NOOA import count is now measured by an import audit hook rather than declared as a constant, alongside a count of the processes able to import it. Tests: 177, no skips (54 receiver, 39 MCP transport, 80 adjudicator, 4 NOOA). Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
The reference has been validated against the SDK built from this checkout. That is the right source during development, and the wrong one for evidence a reader will compare against the package they install. Until now nothing in a run said which of the two it was. Both the authoritative gate and the profile now take RATIFY_SDK=local|published, defaulting to local, so behaviour is unchanged until the switch is taken deliberately: * The gate prints the resolved path of the imported ratify_protocol and asserts it. A published run fails if the module resolves inside the repository, and a local run fails if it resolves outside, so a run cannot prove the checkout while reporting the release, or silently import a package that shadows the checkout. * In published mode the profile does not mount the checkout at all. The package comes from the hash-verified lock, and the run stops if the lock lacks the pin or if the version in the built image is not the expected one. * The image check reports the directory ratify_protocol was imported from, not just its version. * Every artifact carries an evidence_status field stating whether it was produced from the published package or from the checkout, so an artifact cannot be mistaken for final evidence. sandbox-requirements.in carries the published pin commented out, with the regeneration step, so the lock cannot claim a distribution that does not exist yet. The inventory's status note is replaced. The old one said the unified path was not a stable gate and that one sandbox per case was the unimplemented remedy; both are obsolete, since the suite imports nooa once in a single process, that count is measured, and the path passes sequentially and concurrently. The single remaining blocker is release provenance, and the transition steps are written down. Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
…it found v1.0.0-alpha.16 is published. sandbox-requirements.in now pins ratify-protocol==1.0.0a16 unconditionally rather than commented out, and the lock is regenerated against it: verified installable and importable outside any repository checkout before this pin was uncommitted. The authoritative gate and the profile were run with RATIFY_SDK=published, and both confirmed ratify_protocol resolves from the installed package rather than the checkout, inside the built sandbox image itself. Two real bugs surfaced by actually exercising published mode rather than by review. macOS ships bash 3.2, which raises "unbound variable" under `set -u` when an array with zero elements is expanded, even though the array is set. Published mode sets SDK_MOUNT=() because no checkout is mounted, and the very first published-mode run failed in 17 seconds on exactly that line, before any container started. Fixed with the standard portable idiom. Under a concurrent pair of published-mode profiles, `sandbox download` was twice observed to exit 1 once and succeed on an immediate second attempt, distinct from every exec and upload, which never did across the whole engagement. A download only re-reads a file the sandbox already finished writing, so retrying it cannot touch the presentation or authorization boundary the way retrying an exec or upload could. Op.run now accepts a bounded retry count, given only to download, and every attempt is still recorded rather than only the last: a self-healed download shows up in the artifact's retried_operations rather than disappearing into a single clean-looking record, and an exhausted retry still reaches driver_errors exactly as an unretried failure would. Tests: 181, no skips (54 receiver, 39 MCP transport, 84 adjudicator, 4 NOOA). Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
…mary Every count and status claim in these documents now comes from the runs against published ratify-protocol==1.0.0a16, not the pre-publication local checkout: 181 tests (54 receiver, 39 MCP transport, 84 adjudicator, 4 NOOA), 52 required cases, 64 live gates, twice sequential and twice concurrent, zero skips, zero driver errors, zero residue, the intentional-failure profile exiting nonzero with parser safety passing and parser coverage correctly failing. The unified NOOA path's status note is replaced throughout: it was previously described as executed but not yet stable, with one-sandbox-per-case named as the unimplemented remedy. Both are obsolete. The suite imports nooa exactly once in a single process, that count is measured by an import audit hook, and the path passes 64/64 sequentially and concurrently. The README's group table gains the nooa_full_path row it was missing, and the ratify_semantic_denials count is corrected from a rounded "fourteen denials" to the actual fifteen cases, thirteen denials plus two authorize controls. docs/evidence/ is new: a redacted summary in both Markdown and JSON, containing counts, hashes, versions, digests, and known limitations, and explicitly nothing that touched the presentation or authorization boundary. Every artifact behind it was audited first: no private keys, JWTs, credentials, or proof bodies anywhere in the underlying evidence, checked by pattern and by decoding rather than assumed. The public-key-sized base64 blobs the scanner flagged were verified by decoding them (32 and 1952 bytes, the ed25519 and ML-DSA-65 public key sizes, with no private or secret field alongside them). Raw artifacts stay outside this repository; only their SHA-256 is recorded here. The platform qualifier is now explicit everywhere a result is claimed: arm64 macOS with Docker is what was executed, and linux/amd64 and Podman are compatibility targets, not results. Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
…mits Distinguishes the exact commit every run executed against (37b378b) from the one-commit-later prose-only commit these files themselves live in (141f429), so neither reads as evidence gone stale relative to the other. Confirmed by diff: 141f429 touches only README/docs prose, zero changes to the driver, adjudicator, tests, Dockerfile, or profile script. Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
Relative markdown links (evidence/nvidia-reference-evidence.md, ../../docs/...) resolve correctly on GitHub but break the moment any of these documents is extracted as a standalone attachment: a PDF render or a bare .md file emailed on its own has no accompanying file for the relative path to find. Converted the six cross-document links to absolute github.com/identities-ai/ratify-protocol/blob/main URLs, which resolve identically inside the repo and outside it. They will 404 until this branch is pushed and merged to main; that is expected and not something this commit can fix. Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
Signed-off-by: chuks <891251+chuks@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This independent reference contribution demonstrates a NOOA agent carrying a Ratify Protocol delegated-authority proof over MCP, through an OpenShell policy gateway, to an independent receiver.
What it demonstrates
_metaVerification
ratify-protocol==1.0.0a16, not a local checkoutdocs/evidence/Review scope
This does not modify NVIDIA code, claim ODIS conformance, or imply NVIDIA endorsement. It is proposed as an executable interoperability profile for review against the Secure Agent Workspace and ODIS direction. Linux amd64, Podman, TLS, OIDC, and mTLS were not exercised.
This is a draft PR seeking maintainer and technical feedback on scope, architecture, and the appropriate upstream path.