Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/ADRs/0090-runtime-neutral-sandbox-hooks-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ feature:
- The contract is versioned (v1 = the scripts' current stdin/stdout fields).
Verifying it against Claude Code surfaced two pre-existing gaps the
runtimes.md matrix now records instead of a blanket ✓: the runner's
`settings.json` is not loaded from where it is written (#6358), and the
hook wiring was not loaded from where it was written (#6358 — since fixed
via `--settings`), and the
PostToolUse payload differs (`tool_response`,
`hookSpecificOutput.updatedToolOutput`, parallel execution — #6357).
- Alongside this decision the runner's remaining Claude-specific branches
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ repo baseline and overrides)

**Open questions:**

- Does the harness live inside the sandbox (configuring the agent from within its isolation boundary) or outside it (preparing the environment before the agent starts)? (Tool permissions are injected as a host-managed `.claude/settings.json` — configured outside, enforced inside; see [ADR 0027](ADRs/0027-allowed-and-disallowed-tools-for-agents.md). General harness placement remains open.)
- Does the harness live inside the sandbox (configuring the agent from within its isolation boundary) or outside it (preparing the environment before the agent starts)? (Security hooks are injected as a runner-owned `hooks.json` loaded via `--settings`; see [ADR 0027](ADRs/0027-allowed-and-disallowed-tools-for-agents.md). General harness placement remains open.)
- How is codebase context assembled? (See [codebase-context.md](problems/codebase-context.md).)
- How do we version and test harness configurations? (See [testing-agents.md](problems/testing-agents.md).) (Functional tests now test the full pipeline including harness-assembled configuration — [ADR 0052](ADRs/0052-functional-tests-for-agent-pipelines.md). Harness versioning remains open.)

Expand Down
3 changes: 2 additions & 1 deletion docs/guides/dev/cli-internals.md
Original file line number Diff line number Diff line change
Expand Up @@ -429,11 +429,12 @@ Vendoring commit messages use title + body (upload and stale delete). `github st
│ ┌──────────────────────────────────────────┐ │
│ │ Exec() — Run agent in sandbox │ │
│ │ │ │
│ │ Command built by buildClaudeCommand(): │ │
│ │ Command built by buildRunCommand(): │ │
│ │ cd {repoDir} && │ │
│ │ . {envFile} && │ │
│ │ claude --print --verbose │ │
│ │ --output-format stream-json │ │
│ │ [--settings {hooksSettingsPath}] │ │
│ │ --model {model} │ │
│ │ --effort {effort} │ │
│ │ --agent {agent} │ │
Expand Down
33 changes: 16 additions & 17 deletions docs/runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,15 @@ When adding a runtime, fill in the security matrix below and register it in `run
|---------|---------------|-------------|-----------------|---------------------------|
| **Host-side context injection scan** (DeBERTa / LLM Guard, unicode, SSRF patterns on repo context files) | Host + sandbox `scan context` | ✓ | N/A — stub | Requires sandbox image with ML models; harness `security.host_scanners` |
| **Host-side runtime content scan** (agent def, SKILL.md, plugin JSON before upload) | Host (`scanRuntimeContent`) | ✓ | N/A — stub | Uses `security.InputPipeline()`; not part of `Runtime` interface — runner responsibility |
| **Tirith** (Bash command scanning) | Sandbox PreToolUse hook | wired; **not loaded by Claude Code today** (#6358) | N/A — stub | `tirith_check.py`; harness `security.sandbox_hooks.tirith`; fails open on missing binary/timeout unless `TIRITH_REQUIRED=1` |
| **SSRF pre-tool** | Sandbox PreToolUse hook | wired; not loaded (#6358) | N/A — stub | `ssrf_pretool.py`; default on |
| **Canary token detection** | Sandbox Pre/PostToolUse hooks | wired; not loaded (#6358); post-tool field mismatch (#6357) | N/A — stub | `canary_pretool.py` / `canary_posttool.py`; both inert unless `FULLSEND_CANARY_TOKEN` is set |
| **Secret redaction** | Sandbox PostToolUse hook | wired; not loaded (#6358); not effective even when loaded (#6357) | N/A — stub | `secret_redact_posttool.py` |
| **Unicode normalization** | Sandbox PostToolUse hook | wired; not loaded (#6358); not effective even when loaded (#6357) | N/A — stub | `unicode_posttool.py` |
| **Context suppression** | Sandbox PostToolUse hook | wired; not loaded (#6358); not effective even when loaded (#6357) | N/A — stub | `context_suppress_posttool.py` |
| **Tool allowlist** | Sandbox PreToolUse hook | opt-in; wired; not loaded (#6358) | N/A — stub | `tool_allowlist_pretool.py`; requires `FULLSEND_TOOL_ALLOWLIST` (fail-closed when unset) |
| **Tirith** (Bash command scanning) | Sandbox PreToolUse hook | ✓ (loaded via `--settings`, #6358) | N/A — stub | `tirith_check.py`; harness `security.sandbox_hooks.tirith`; fails open on missing binary/timeout unless `TIRITH_REQUIRED=1` |
| **SSRF pre-tool** | Sandbox PreToolUse hook | ✓ (e2e-guarded by `hooks-loaded.feature`) | N/A — stub | `ssrf_pretool.py`; default on |
| **Canary token detection** | Sandbox Pre/PostToolUse hooks | pre ✓; post-tool field mismatch (#6357) | N/A — stub | `canary_pretool.py` / `canary_posttool.py`; both inert unless `FULLSEND_CANARY_TOKEN` is set |
| **Secret redaction** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `secret_redact_posttool.py` |
| **Unicode normalization** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `unicode_posttool.py` |
| **Context suppression** | Sandbox PostToolUse hook | wired; not effective under Claude Code (#6357) | N/A — stub | `context_suppress_posttool.py` |
| **Tool allowlist** | Sandbox PreToolUse hook | opt-in; ✓ when enabled | N/A — stub | `tool_allowlist_pretool.py`; requires `FULLSEND_TOOL_ALLOWLIST` (fail-closed when unset) |
| **Prompt injection (DeBERTa)** | Host Path A + sandbox Path B | ✓ | N/A — stub | Same scanner stack as context files when enabled in harness |
| **Sandbox tool hooks wiring** | `SandboxHooksBootstrap` type assert in `Bootstrap` | ✓ scripts + settings.json installed — but written to `/sandbox/workspace/.claude/`, which the CLI (cwd `/sandbox/workspace/<repo>`) does not read (#6358) | ✗ — `Bootstrap` is a stub; must wire `security.HookPlan` via OpenCode plugin hooks | Hook scripts and wiring plan are runtime-neutral (see [Sandbox hook contract](#sandbox-hook-contract)); a runtime that ignores `SandboxHooksBootstrap` installs **no** sandbox tool hooks — say so explicitly here |
| **Sandbox tool hooks wiring** | `SandboxHooksBootstrap` type assert in `Bootstrap` | ✓ scripts at `claude-config/hooks/`, wiring at `claude-config/hooks.json` via `--settings` (#6358) | ✗ — `Bootstrap` is a stub; must wire `security.HookPlan` via OpenCode plugin hooks | Hook scripts and wiring plan are runtime-neutral (see [Sandbox hook contract](#sandbox-hook-contract)); a runtime that ignores `SandboxHooksBootstrap` installs **no** sandbox tool hooks — say so explicitly here |
| **Transcript / debug artifacts** | `TranscriptHandler` (+ optional `DebugLogNamer`) | ✓ (stream-json, `claude-debug.log`) | No-op — see #1935 | Format-specific; not shared across runtimes. Debug-log filename defaults to `agent-debug.log` unless the runtime implements `DebugLogNamer` |

### Fail modes
Expand All @@ -52,12 +52,12 @@ A runtime whose `Bootstrap` does not type-assert `SandboxHooksBootstrap` will **

The hook scripts in `internal/security/hooks/*.py` are plain programs with no Claude Code dependency; Claude Code invokes them through `settings.json`. Any runtime can call them from its own tool-call interception point (OpenCode `tool.execute.before/after`, pi `tool_call`/`tool_result`, Cursor hooks, …).

- **Files:** `security.HookFiles(cfg)` returns `filename → script bytes` for the enabled hooks; `runtime.installHookScripts(sandbox, dir, cfg)` creates `dir` in the sandbox and uploads them there (executable) — any directory works. Claude uses `/sandbox/workspace/.claude/hooks/` (`security.SandboxHooksDir`).
- **Wiring:** `security.HookPlan(cfg)` returns ordered `HookGroup{Phase, Tools, Scripts}` entries. `Phase` is `PreToolUse` or `PostToolUse`; `Tools` are Claude Code tool names (`Bash`, `Read`, `WebFetch`, `*` = all) — runtimes with other names translate before matching (see #608). **Adapters must run the `Scripts` of one group sequentially in the listed order, feeding each script's modified result to the next** (the PostToolUse order suppress → unicode → redact is a security invariant). `GenerateClaudeSettings` is rendered from `HookPlan`, so the two cannot diverge.
- **Files:** `security.HookFiles(cfg)` returns `filename → script bytes` for the enabled hooks; `runtime.installHookScripts(sandbox, dir, cfg)` creates `dir` in the sandbox and uploads them there (executable) — any directory works. Claude uses `/sandbox/claude-config/hooks/` (`security.SandboxHooksDir`), with the wiring at `/sandbox/claude-config/hooks.json` (`security.SandboxHooksSettings`) loaded via `--settings`.
- **Wiring:** `security.HookPlan(cfg)` returns ordered `HookGroup{Phase, Tools, Scripts}` entries. `Phase` is `PreToolUse` or `PostToolUse`; `Tools` are Claude Code tool names (`Bash`, `Read`, `WebFetch`, `*` = all) — runtimes with other names translate before matching (see #608). **Adapters must run the `Scripts` of one group sequentially in the listed order, feeding each script's modified result to the next** (the PostToolUse order suppress → unicode → redact is a security invariant). `GenerateHooksConfig` is rendered from `HookPlan`, so the two cannot diverge.
- **Wire protocol (per script):** JSON on stdin — `{"tool_name": ..., "tool_input": {...}}` for PreToolUse, plus `"tool_result"` for PostToolUse. Exit `0` = allow. *Blocking* scripts (all PreToolUse scripts, and `canary_posttool.py`) exit `1` and print `{"decision":"block","reason":"..."}` on stdout; the adapter must stop the tool call (or, post-tool, drop the result) and surface the reason. *Sanitizing* PostToolUse scripts (`context_suppress`, `unicode`, `secret_redact`) always exit `0` and print `{"tool_result": <modified>}` when they changed something; empty stdout = unchanged.
- **Fail modes:** blocking scripts fail **closed** on malformed JSON or oversized input (> 10 × 1024 × 1024 characters, read from text-mode stdin) — they block. Empty/whitespace-only stdin is treated as "no tool call" and allowed by every script; a payload without `tool_name` blocks only in the allowlist hook. `tirith_check.py` fails **open** when the `tirith` binary is missing, times out or errors, unless `TIRITH_REQUIRED=1` (which `appendHookEnv` writes when Tirith is enabled — adapters must make sure it reaches the script). Sanitizing scripts fail **open** — malformed or oversized input is passed through unchanged (exit 0, empty stdout; the unicode hook logs an `input_truncated` finding). Adapters must not treat a sanitizer's empty stdout as an error.
- **Environment:** `runtime.appendHookEnv` writes `TIRITH_FAIL_ON` / `TIRITH_REQUIRED` into `/sandbox/workspace/.env`; the runtime must launch the scripts with that file sourced (Claude's run command does). Scripts also read `FULLSEND_TRACE_ID`, `FULLSEND_TOOL_ALLOWLIST` (allowlist hook, fail-closed when unset) and `FULLSEND_CANARY_TOKEN` (both canary hooks are no-ops when it is empty; supply it via harness `env.sandbox`/`host_files`), and write findings to `/sandbox/workspace/.security/findings.jsonl`.
- **Claude Code caveats (#6358, #6357):** (1) *Loading* — the runner writes `settings.json` to `/sandbox/workspace/.claude/` but starts the CLI from `/sandbox/workspace/<repo>`; Claude Code reads project settings only from `<cwd>/.claude/` (plus `$CLAUDE_CONFIG_DIR/settings.json`, managed settings and `--settings`), so **today no sandbox tool hook is loaded at all** — verified from a `--debug` run (`Hooks: Found 0 total hooks in registry`); tracked in #6358. (2) *Payload* — Claude Code's PostToolUse input carries the output as `tool_response` (the scripts read `tool_result`), replacing output requires `hookSpecificOutput.updatedToolOutput` (the scripts print a bare `tool_result`), and all matching hooks run in parallel with no output chaining; tracked in #6357. (3) *Blocking* — Claude Code keys on the stdout JSON on any exit code (`decision:"block"` is deprecated for PreToolUse but still maps to `deny`) and treats a bare exit `1` as non-blocking (exit `2` is its own blocking code); a local control run confirmed the scripts' "exit 1 + `{"decision":"block"}`" convention does block once the settings are loaded. Net: after #6358, the PreToolUse half of the contract is effective under Claude Code; the PostToolUse scripts additionally need #6357.
- **Claude Code caveats (#6357):** (1) *Loading* — fixed by #6358: the hook wiring is written to the runner-owned `/sandbox/claude-config/hooks.json` and passed explicitly via `--settings`, so it loads regardless of the CLI's working directory (previously it sat unread in `/sandbox/workspace/.claude/`); the `hooks-loaded.feature` behaviour scenario guards the "silently not loaded" regression class. Note Claude Code still auto-loads a target repo's own `<repo>/.claude/settings.json` hooks from `<cwd>` — a separate exposure to assess. (2) *Payload* — Claude Code's PostToolUse input carries the output as `tool_response` (the scripts read `tool_result`), replacing output requires `hookSpecificOutput.updatedToolOutput` (the scripts print a bare `tool_result`), and all matching hooks run in parallel with no output chaining; tracked in #6357. (3) *Blocking* — Claude Code keys on the stdout JSON on any exit code (`decision:"block"` is deprecated for PreToolUse but still maps to `deny`) and treats a bare exit `1` as non-blocking (exit `2` is its own blocking code); a local control run confirmed the scripts' "exit 1 + `{"decision":"block"}`" convention does block once the settings are loaded. Net: the PreToolUse half of the contract is effective under Claude Code; the PostToolUse scripts additionally need #6357.

### Runtime-specific config key support

Expand All @@ -70,7 +70,7 @@ Harness keys are runtime-neutral in the YAML but each runtime owns their transla
| `plugins` | Claude plugin marketplace layout (`bootstrapPlugins`) | — | ignored | Claude-specific format; warn and skip if unsupported |
| Agent frontmatter `tools:` (`Bash(gh,jq)` syntax, ADR 0027) | Native Claude permission syntax | — | ignored | Enforce via `--tools`/allowlist plus a hook adapter; Claude tool names differ in case from most runtimes (#608) |
| `skills` | `CLAUDE_CONFIG_DIR/skills/` | — | ignored | Agent Skills spec (`SKILL.md`) is portable; destination is `rt.ConfigDir() + "/skills"` (also used by the runtime fetch service) |
| `security.sandbox_hooks` | `SandboxHooksBootstrap` → settings.json | ✗ (stub) | ignored | See [Sandbox hook contract](#sandbox-hook-contract) |
| `security.sandbox_hooks` | `SandboxHooksBootstrap` → hooks.json via `--settings` | ✗ (stub) | ignored | See [Sandbox hook contract](#sandbox-hook-contract) |
| `--debug` (CLI flag) | `--debug-file`, artifact `claude-debug.log` | — | no-op | Implement `DebugLogNamer` to name the artifact |

## Sandbox workspace layout
Expand All @@ -86,15 +86,14 @@ The sandbox has two key directories that map to Claude Code's config levels:
│ │ ├── code-review/SKILL.md Built-in skills (personal level — wins on collision)
│ │ ├── pr-review/SKILL.md
│ │ └── ...
│ └── plugins/
│ └── ... Plugin state (simplified; see bootstrapPlugins())
│ ├── plugins/
│ │ └── ... Plugin state (simplified; see bootstrapPlugins())
│ ├── hooks/ Security hook scripts (PreToolUse, PostToolUse)
│ └── hooks.json Hook wiring (loaded via --settings in buildRunCommand)
└── workspace/ ← SandboxWorkspace
├── .env Environment variables (sourced before claude)
├── .env.d/ Additional env files (host_files expand)
├── .claude/
│ ├── hooks/ Security hooks (PreToolUse, PostToolUse)
│ └── settings.json Hook wiring (separate from plugin config)
└── <repo-name>/ ← Claude Code's working directory (cd target)
├── CLAUDE.md Project instructions (repo's own or injected bridge)
Expand Down
32 changes: 32 additions & 0 deletions e2e/behaviour/features/dispatch/hooks-loaded.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
Feature: Sandbox security hooks are loaded via --settings
Comment thread
maruiz93 marked this conversation as resolved.
Comment thread
maruiz93 marked this conversation as resolved.

Security hooks (SSRF, canary, secret redaction, etc.) are installed under
the runner-owned claude-config/ directory and wired via the --settings flag
so Claude Code loads them regardless of its working directory. This scenario
verifies that at least one blocking PreToolUse hook fires end-to-end —
catching the "silently not loaded" class of regression where hook wiring
exists but the CLI never reads it.

Scenario: SSRF PreToolUse hook blocks a disallowed URL
Given the enrolled test repository
And a custom harness "hooks-smoke" with:
"""
agent: agents/triage.md
role: triage
slug: fullsend-ai-hooks-smoke
model: opus
image: ghcr.io/fullsend-ai/fullsend-sandbox:latest
trigger: |
event.entity.kind == "work_item"
&& event.transition.kind == "label_changed"
&& event.transition.label.name == "ready-for-hooks-smoke"
"""
And a dummy agent that would:
| description | op | args |
| Fetch metadata endpoint | url_get | http://169.254.169.254/latest/meta-data/ |
| Emit triage JSON | write_fixture | output/agent-result.json, fixtures/triage/sufficient.json |
And an issue
When the issue is labeled "ready-for-hooks-smoke"
Then the harness "hooks-smoke" workflow completes successfully
And the agent will fail to Fetch metadata endpoint
And the agent will succeed to Emit triage JSON
Loading
Loading