Skip to content

Proposal: Parmana refund scenario as a reproducible test package, mapped to TRACE #286

Description

@pavancharak

Summary

Parmana is a server between an AI agent and the systems it acts on. It evaluates each proposed action against a
versioned policy, requires a signed human approval where the policy says so, releases the action only through a
gateway that holds the credentials, and signs a record of every action and every refusal.
This issue proposes one scenario as a reproducible test package, and asks how its decision and execution evidence
should map to TRACE before any integration is written.

Scope proposed

One scenario: a customer refund (paytm:refund) under the policy customer-refund 1.2.0, in three cases:
Case Expected Connector invocations
Valid approval: a signed manager approval for the order, up to the amount 200, APPROVED, signed Execution Trust Record 1 (mock result)
Refusal: every caller fact true, no approval 403 POLICY_DENIED, signed Refusal Record 0
Replay: the same approval sent again with a new request 403 POLICY_DENIED, signed Refusal Record 0

Pinned source and commands

Source: https://github.com/pavancharak/parmana at commit b4deee66fa8f0aad29a7ddc916f7fb33eef0ff50
Package README: https://github.com/pavancharak/parmana/blob/b4deee66fa8f0aad29a7ddc916f7fb33eef0ff50/docs/evaluation/agentrust-refund/README.md
Policy: https://github.com/pavancharak/parmana/blob/b4deee66fa8f0aad29a7ddc916f7fb33eef0ff50/policies/customer-refund/1.2.0/policy.json

git clone https://github.com/pavancharak/parmana.git
cd parmana
git checkout b4deee66fa8f0aad29a7ddc916f7fb33eef0ff50
npm ci
npm run build
npm run evaluate:agentrust-refund

Node.js 24. No network, database, keys or accounts: storage is in memory, the approver key is generated per run, and
the connector is a hermetic mock. The command writes evaluations/agentrust-refund/report.json with the commit,
whether the working tree was clean, the Node.js version and, for each case, the HTTP status, decision, reason,
connector invocation count and the signed record (id, hash, algorithm, policy content hash).
Existing run record: https://github.com/pavancharak/parmana/blob/main/evaluations/agentrust-refund/report.json (a run on a clean checkout of b4deee66; the record names that commit)

Limitations stated up front

Mock results only. The connector is MockPaytmConnectorServer. Nothing in the package confirms a refund at a downstream system; the report labels each connector result as a mock result.

Caller supplied facts. The policy treats refundEligible and fraudCheckPassed as declared by the caller,
not checked against an order or fraud system. They can only refuse a refund; the signed manager approval is what
authorizes one. The report keeps this limitation.
Enforcement versus evidence. The refusals are enforced by the server (policy, approval verification, single use). The signed records are evidence that can be verified afterwards, offline, with the public key; they do not enforce anything by themselves.

Proposed TRACE mapping (for discussion)
Against TRACE v0.2 (agentrust-trace 0.11.0, TrustRecord):
TRACE field From Parmana Status
policy.bundle_hash, policy.version, policy.enforcement_mode Policy content hash, customer-refund@1.2.0, enforce Available (approved requests; Refusal Records do not carry the hash today)
tool_transcript.call_count Connector invocations: 1, 0, 0 Available
tool_transcript.hash Hash over the canonical list of connector calls To define
origin third-party-control-plane, producer parmana, the record id Available
references The Parmana record id, resolver, record hash Available
iat Record creation time Available
runtime software-only; measurement to agree Partly
build_provenance SLSA level 0 for the server today; digest to agree Partly
appraisal none (no hardware evidence) To agree
subject, model, data_class Not visible to a governance layer Need guidance
cnf / signature Parmana signs its own canonical form, not the TRACE form Need guidance

Questions for a maintainer

Is one TRACE record per Parmana decision the right unit, including refusals with call_count: 0?
What should an adapter use for subject, model and data_class when the governance layer does not see them?
Should the TRACE record be signed by an adapter key that references the Parmana record, or should Parmana sign the TRACE form itself?
How should a mock connector result be marked so it is never read as a confirmed downstream action?
I will not start the integration until the scope is agreed here.

Permissions

The current licence is evaluation only. For this work I can grant written permission to you and the AgenTrust reviewers to run the evaluation and publish fixtures and results in agentrust-io/integrations. I will send that permission separately.

Activity

  1. pavancharak commented on Oct 6, 2026

    @pavancharak
    Author

    Update: two gaps from the mapping table are closed on main.

    • policy.bundle_hash: Refusal Records now carry policyContentHash inside the signed record, the same value an Execution Trust Record carries. The refund evaluation checks that the refusal and replay cases have the same hash as the approved case.
    • build_provenance: each GitHub release now publishes the server image ghcr.io/pavancharak/parmana-api: with SLSA build provenance attached in the registry (slsa-verifier verify-image). The hosted deployment is built by Vercel and does not have it.

    The run record pinned in this issue is still b4deee66, which predates both changes. I can re-pin it to a newer commit if that helps.

  2. imran-siddique commented on Oct 6, 2026

    @imran-siddique
    Member

    @pavancharak thanks for asking before writing the integration. Answers in order, checked against trace-spec at db95328.

    Subject, model, data_class. All three are required in schema/trace-claim.json, and a governance layer that cannot see them cannot make the claim a Trust Record carries, so do not fill them with placeholders. That makes Parmana an external-evidence-source in integration.yaml, the role integrations#170 settled for aeoess-aps, with no conformance level of its own.

    Signing. Parmana keeps signing its own canonical form. The agent runtime's Trust Record points at it under references (§3.1.2): rel: authorized-intent for the decision and approval-outcome for the signed manager approval, with Parmana as resolver. No adapter key and no re-signing.

    Unit. One Parmana record per decision, refusals included, works as the referenced unit. A refusal is an authorization decision too.

    Mocks. A mock connector result must never be referenced as observed-effect, which is a state change seen by an observer outside the agent. Keep the mock label in the fixtures and in report.json as you have it.

    Licence. This repository is Apache-2.0, so fixtures merged here are redistributed under it. The permission has to be in the PR as a licence on those files rather than sent separately.

    Please re-pin to the commit carrying the refusal-record hash and the SLSA provenance before opening the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions