Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions skills/deliverability-judge/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# deliverability-judge

A native runx skill: a `SKILL.md` contract, an `X.yaml` execution profile, and a
`run.mjs` script. No build step and no dependencies.

## Develop

```bash
runx harness . --json # run the harness cases in X.yaml
runx skill . --input message=hello --json # run the skill once
runx history # inspect the signed receipt
```

Edit `run.mjs` to do the real work, and keep both harness classes in `X.yaml`:
one happy path and one stop, error, or refusal case.

## Publish

```bash
runx login --provider github --for publish
runx registry publish . # the registry runs the harness as the publish gate
```
59 changes: 59 additions & 0 deletions skills/deliverability-judge/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
name: deliverability-judge
description: Read-only deliverability posture judge that fuses sealed provider evidence into a continue/throttle/pause recommendation or a human-escalation refusal.
source:
type: cli-tool
command: node
args:
- run.mjs
timeout_seconds: 30
sandbox:
profile: readonly
cwd_policy: skill-directory
inputs:
evidence:
type: object
required: true
description: Sealed deliverability evidence with postmaster_report, bounce_metrics, complaint_metrics, and placement_probe.
policy:
type: object
required: true
description: Operator thresholds with min_reputation_score, max_bounce_pct, and max_complaint_pct.
runx:
category: ops
input_resolution:
required:
- evidence
- policy
---

# deliverability-judge

Use this skill when an operator needs a read-only judgment on whether a sending
posture is healthy enough to continue sending.

The skill fuses four sealed provider signals:

- `postmaster_report.reputation_score`
- `bounce_metrics.bounce_pct`
- `complaint_metrics.complaint_pct`
- `placement_probe.passed`

It compares those signals against an operator policy:

- `min_reputation_score`
- `max_bounce_pct`
- `max_complaint_pct`

The output is a JSON verdict. When all signals are sealed, present, and
non-contradictory, the verdict includes a read-only recommendation:

- `continue` for healthy posture
- `throttle` for moderate aligned risk
- `pause` for clearly unhealthy aligned risk

When signals are missing, unsealed, or contradictory, the skill refuses to emit
a recommendation and returns an escalation record instead.

This skill is read-only. It mints no authority, holds no state, emits no Effect,
and never performs a send, throttle, payment, or operational handoff.
83 changes: 83 additions & 0 deletions skills/deliverability-judge/X.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
skill: deliverability-judge
version: "0.1.0"

catalog:
kind: skill
audience: public
visibility: public
role: canonical

harness:
cases:
- name: sealed_healthy_signals_continue
runner: default
inputs:
evidence:
postmaster_report:
source: google-postmaster
timestamp: "2026-07-01T12:00:00Z"
reputation_score: 0.94
bounce_metrics:
source: esp-bounce-export
timestamp: "2026-07-01T12:00:00Z"
bounce_pct: 0.7
complaint_metrics:
source: feedback-loop
timestamp: "2026-07-01T12:00:00Z"
complaint_pct: 0.02
placement_probe:
source: seed-inbox-probe
timestamp: "2026-07-01T12:00:00Z"
passed: true
policy:
min_reputation_score: 0.8
max_bounce_pct: 2
max_complaint_pct: 0.1
expect:
status: sealed
- name: contradictory_signals_escalate
runner: default
inputs:
evidence:
postmaster_report:
source: google-postmaster
timestamp: "2026-07-01T13:00:00Z"
reputation_score: 0.96
bounce_metrics:
source: esp-bounce-export
timestamp: "2026-07-01T13:00:00Z"
bounce_pct: 9.4
complaint_metrics:
source: feedback-loop
timestamp: "2026-07-01T13:00:00Z"
complaint_pct: 0.03
placement_probe:
source: seed-inbox-probe
timestamp: "2026-07-01T13:00:00Z"
passed: true
policy:
min_reputation_score: 0.8
max_bounce_pct: 2
max_complaint_pct: 0.1
expect:
status: sealed

runners:
default:
default: true
type: cli-tool
command: node
args:
- run.mjs
sandbox:
profile: readonly
cwd_policy: skill-directory
inputs:
evidence:
type: object
required: true
description: Sealed deliverability evidence.
policy:
type: object
required: true
description: Deliverability policy thresholds.
57 changes: 57 additions & 0 deletions skills/deliverability-judge/evidence.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
{
"summary": "deliverability-judge 0.1.0 is a read-only runx skill that fuses sealed deliverability evidence into a continue, throttle, pause, or escalation verdict.",
"observations": [
{
"name": "runx_version",
"value": "runx-cli 0.6.14",
"meets_requirement": true
},
{
"name": "package_name",
"value": "deliverability-judge",
"meets_requirement": true
},
{
"name": "github_star",
"value": "pay4love1 currently stars https://github.com/runxhq/runx, verified through the GitHub API before submission preparation.",
"meets_requirement": true
},
{
"name": "source_pr",
"value": "https://github.com/runxhq/runx/pull/204",
"meets_requirement": true
},
{
"name": "local_direct_runner_healthy_case",
"value": "node run.mjs returned verdict.state=healthy and recommendation.action=continue for sealed healthy signals.",
"meets_requirement": true
},
{
"name": "local_direct_runner_contradictory_case",
"value": "node run.mjs returned verdict.state=escalate and no recommendation for contradictory sealed signals.",
"meets_requirement": true
},
{
"name": "windows_runx_harness_blocker",
"value": "runx harness and registry publish were attempted locally on Windows, but the CLI failed in receipt/sandbox runtime paths before skill logic execution. Linux hosted harness/publish is still required for final delivery.",
"meets_requirement": false
},
{
"name": "runx_connect_auth_blocker",
"value": "runx login --provider github --for publish generated connect.runx.ai sessions, but the browser flow returned 404 Not Found after GitHub sign-in. The same result was reproduced by the operator manually, so registry publish could not be completed from this environment.",
"meets_requirement": false
}
],
"commands": [
"runx --version",
"node run.mjs",
"runx login --provider github --for publish",
"runx registry publish ./skills/deliverability-judge/SKILL.md --registry https://api.runx.ai"
],
"artifacts": {
"source_url": "https://github.com/runxhq/runx/pull/204",
"pr_url": "https://github.com/runxhq/runx/pull/204",
"local_output": "skills/deliverability-judge/harness-evidence/local-output.md"
},
"version": "0.1.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"evidence": {
"postmaster_report": {
"source": "google-postmaster",
"timestamp": "2026-07-01T13:00:00Z",
"reputation_score": 0.96
},
"bounce_metrics": {
"source": "esp-bounce-export",
"timestamp": "2026-07-01T13:00:00Z",
"bounce_pct": 9.4
},
"complaint_metrics": {
"source": "feedback-loop",
"timestamp": "2026-07-01T13:00:00Z",
"complaint_pct": 0.03
},
"placement_probe": {
"source": "seed-inbox-probe",
"timestamp": "2026-07-01T13:00:00Z",
"passed": true
}
},
"policy": {
"min_reputation_score": 0.8,
"max_bounce_pct": 2,
"max_complaint_pct": 0.1
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"evidence": {
"postmaster_report": {
"source": "google-postmaster",
"timestamp": "2026-07-01T12:00:00Z",
"reputation_score": 0.94
},
"bounce_metrics": {
"source": "esp-bounce-export",
"timestamp": "2026-07-01T12:00:00Z",
"bounce_pct": 0.7
},
"complaint_metrics": {
"source": "feedback-loop",
"timestamp": "2026-07-01T12:00:00Z",
"complaint_pct": 0.02
},
"placement_probe": {
"source": "seed-inbox-probe",
"timestamp": "2026-07-01T12:00:00Z",
"passed": true
}
},
"policy": {
"min_reputation_score": 0.8,
"max_bounce_pct": 2,
"max_complaint_pct": 0.1
}
}
66 changes: 66 additions & 0 deletions skills/deliverability-judge/harness-evidence/local-output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Local Evidence

Toolchain:

```text
runx-cli 0.6.14
node v22.17.1
```

Direct runner verification was executed against the same inputs used by the
inline harness cases.

## sealed_healthy_signals_continue

Expected: sealed evidence fuses into a healthy verdict and read-only
`continue` recommendation.

Observed:

```json
{
"verdict": {
"state": "healthy",
"confidence_window": "high",
"reason": "all_sealed_signals_pass_policy"
},
"recommendation": {
"action": "continue",
"signal_bindings": {
"postmaster_report": "google-postmaster",
"bounce_metrics": "esp-bounce-export",
"complaint_metrics": "feedback-loop",
"placement_probe": "seed-inbox-probe"
},
"evidence_hash": "de4d14b9583c678f096fecbc0a147a20b9f86ca17b8d297420ba9365bf718c5a"
}
}
```

## contradictory_signals_escalate

Expected: contradictory sealed signals refuse a recommendation and escalate.

Observed:

```json
{
"verdict": {
"state": "escalate",
"confidence_window": "low",
"reason": "contradictory_signals"
},
"escalation": {
"reason": "Signals disagree, so no read-only recommendation is emitted.",
"contradictions": [
"high_reputation_conflicts_with_high_bounce"
],
"signal_names": [
"postmaster_report",
"bounce_metrics",
"complaint_metrics",
"placement_probe"
]
}
}
```
13 changes: 13 additions & 0 deletions skills/deliverability-judge/report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# deliverability-judge 0.1.0 report

- Package: `pay4love1/deliverability-judge@0.1.0`
- Pull request: https://github.com/runxhq/runx/pull/204
- Bounty: https://gofrantic.com/bounties/65
- runx CLI: `runx-cli 0.6.14`
- Purpose: read sealed provider evidence for postmaster reputation, bounce rate, complaint rate, and placement probe status, then emit a read-only deliverability verdict.
- Healthy sealed evidence returns `verdict.state=healthy` with `recommendation.action=continue`.
- Contradictory sealed evidence returns `verdict.state=escalate` and refuses to emit a recommendation.
- The implementation is read-only: no send, throttle, payment, state write, Effect, or operational handoff is performed.
- Local direct runner evidence is included in `harness-evidence/local-output.md`.
- Current blocker: `runx login --provider github --for publish` and manual `runx connect github` both reached the browser/GitHub sign-in flow, then returned 404 Not Found after sign-in.
- Remaining finalization: recover runx publish auth, complete registry publish, hosted harness, clean install, dogfood run, and receipt verification.
Loading