Add optional Tirith pre-exec scanning for sandbox bash tool - #14
Open
sheeki03 wants to merge 2 commits into
Open
Add optional Tirith pre-exec scanning for sandbox bash tool#14sheeki03 wants to merge 2 commits into
sheeki03 wants to merge 2 commits into
Conversation
The embedded agent runtime executes bash commands via subprocess.run() with shell=True and no pre-execution security checks. The sandbox provides containment but no intent validation — the LLM can run curl|bash, exfiltrate data, or hit homograph URLs without any gate. This adds an optional Tirith scanner integration that intercepts bash commands before execution. When enabled, tirith check runs against the command and returns structured findings to the model so it can reformulate, rather than silently executing something dangerous. Key design decisions: - Opt-in via TIRITH_ENABLED (default: false, zero impact when off) - Exit code is source of truth (0=allow, 1=block, 2=warn) - JSON enriches findings but never overrides the exit code verdict - Fail-open by default (configurable to fail-closed) - Distinguishes real verdicts from scanner failures in the response - One-time warnings for operational failures (no log spam) - Bash only — other tools are out of scope for this change
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughAdds optional Tirith pre-exec security scanning: new env vars and Settings fields, sandbox env plumbing, runtime scanner integration that can allow/warn/block tool execution, and comprehensive unit/integration tests validating behaviors. Changes
Sequence DiagramsequenceDiagram
participant Client as Bash Tool Caller
participant Gate as Execution Gate
participant Tirith as Tirith Scanner
participant Sandbox as Sandbox Environment
Client->>Gate: execute_tool("bash", command)
alt TIRITH_ENABLED = true
Gate->>Tirith: tirith_check_command(command)
Tirith->>Tirith: run binary, parse JSON findings
alt exit code == 0 (allow)
Tirith-->>Gate: None
Gate->>Sandbox: Execute command
else exit code != 0 (block/warn)
Tirith-->>Gate: {error_type, tirith_action, message, findings}
alt tirith_action == "block"
Gate-->>Client: Return error (status="error", error_type="security_blocked")
else tirith_action == "warn"
Gate->>Sandbox: Execute command, emit warning
end
end
else TIRITH_ENABLED = false
Gate->>Sandbox: Execute command directly
end
Sandbox-->>Client: Command result
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
core-api/api/services/agents/dispatch.py (1)
141-144: Document or enforce a sandbox recycle when Tirith config changes.These vars are only applied in
create_sandbox(). The reconnect paths reuse existing sandboxes unchanged, so togglingTIRITH_ENABLEDorTIRITH_FAIL_MODEonly affects newly created sandboxes. If live policy changes are expected, compare a config hash on reconnect and recreate on drift; otherwise make the recycle requirement explicit.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core-api/api/services/agents/dispatch.py` around lines 141 - 144, The current Tirith-related env vars (TIRITH_ENABLED, TIRITH_PATH, TIRITH_TIMEOUT, TIRITH_FAIL_MODE) are only applied when create_sandbox() runs, so reconnect logic reuses old sandboxes and ignores config changes; to fix, compute a stable config hash from the Tirith-relevant settings (e.g., tirith_enabled, tirith_path, tirith_timeout, tirith_fail_mode) and store it on the sandbox metadata when create_sandbox() makes the sandbox, then update the reconnect flow (the code path that reuses existing sandboxes on agent reconnect) to compare the stored sandbox config hash with the current settings hash and force a sandbox recycle/recreate when they differ (or alternatively throw/require explicit recycle); ensure you reference create_sandbox(), the reconnect handler, and the Tirith env names (TIRITH_ENABLED/TIRITH_FAIL_MODE) so maintainers can locate and apply the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@core-api/api/config.py`:
- Around line 186-190: Validate Tirith config right after loading the settings:
ensure tirith_timeout is a positive integer (>0) and tirith_fail_mode is one of
the allowed values ("open" or "closed"); if tirith_enabled is true reject
invalid timeout or fail_mode values with a clear error/exception instead of
silently defaulting; also treat an explicitly provided TIRITH_FAIL_MODE (or
tirith_path) that is not recognized as invalid and fail fast. Update the config
initialization/validation code around the tirith_* fields (tirith_enabled,
tirith_path, tirith_timeout, tirith_fail_mode) to enforce these checks and
raise/exit on invalid values so unsafe or mistyped settings are not injected
into the sandbox.
---
Nitpick comments:
In `@core-api/api/services/agents/dispatch.py`:
- Around line 141-144: The current Tirith-related env vars (TIRITH_ENABLED,
TIRITH_PATH, TIRITH_TIMEOUT, TIRITH_FAIL_MODE) are only applied when
create_sandbox() runs, so reconnect logic reuses old sandboxes and ignores
config changes; to fix, compute a stable config hash from the Tirith-relevant
settings (e.g., tirith_enabled, tirith_path, tirith_timeout, tirith_fail_mode)
and store it on the sandbox metadata when create_sandbox() makes the sandbox,
then update the reconnect flow (the code path that reuses existing sandboxes on
agent reconnect) to compare the stored sandbox config hash with the current
settings hash and force a sandbox recycle/recreate when they differ (or
alternatively throw/require explicit recycle); ensure you reference
create_sandbox(), the reconnect handler, and the Tirith env names
(TIRITH_ENABLED/TIRITH_FAIL_MODE) so maintainers can locate and apply the
change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 771f7506-f836-439e-aeb4-17b606a0c1fd
📒 Files selected for processing (5)
core-api/.env.examplecore-api/api/config.pycore-api/api/services/agents/dispatch.pycore-api/api/services/agents/runtime_bundle.pycore-api/tests/test_tirith_sandbox.py
Reject TIRITH_TIMEOUT <= 0 and unknown TIRITH_FAIL_MODE values during settings initialization rather than letting them silently degrade scanning behavior inside the sandbox. Normalize fail_mode and trim tirith_path on load. The embedded runtime normalization stays as a second layer of defense for direct env var injection.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
The AI agent sandbox runs bash commands with
subprocess.run(shell=True)and no pre-execution checks. The E2B sandbox provides process isolation, but the LLM inside can stillcurl | bashfrom a homograph URL, exfiltrate credentials, or pipe through a decode chain — and nothing gates that today.This adds Tirith as an optional pre-exec scanner for the sandbox bash tool. When enabled, every bash command the LLM requests is scanned before execution. If Tirith flags the command, it's blocked and structured findings are returned to the model so it can reformulate — the LLM gets told what was wrong and why, which is more useful than a generic rejection.
Tirith has already been integrated into Hermes Agent where it gates all terminal command execution.
What Tirith catches
80+ detection rules across 15 categories. The ones most relevant to sandbox bash:
curl | bash,wget | sh, every source-to-sink patternbase64 -d | bash,python -c "exec(b64decode(...))", PowerShell encoded commandscurl -d @/etc/passwd, env var uploads, command substitution exfilSub-millisecond on clean commands. The LLM never notices it's there until something is actually wrong.
How it works
TIRITH_ENABLED=true, the runtime runstirith check --json --non-interactive -- <command>0= allow,1= block,2= warn (exit code is authoritative — JSON enriches but never overrides)If Tirith is not installed, times out, or crashes, the default behavior is fail-open (configurable to fail-closed via
TIRITH_FAIL_MODE).What the model sees when a command is blocked
{ "status": "error", "error_type": "security_blocked", "tirith_action": "block", "message": "Command blocked by security scan: [CRITICAL] Curl pipe to shell. Review the findings and reformulate the command.", "findings": [ { "rule_id": "curl_pipe_shell", "severity": "CRITICAL", "title": "Curl pipe to shell", "description": "Piping curl output to a shell interpreter allows arbitrary code execution." } ] }The model gets enough context to reformulate (e.g., download first, then inspect, then execute) rather than retrying the same dangerous command.
Configuration
Four env vars, all optional:
TIRITH_ENABLEDfalsetrueto enable scanningTIRITH_PATHTIRITH_TIMEOUT10TIRITH_FAIL_MODEopenopen= allow if scanner fails;closed= blockFor production, prebake the tirith binary into the E2B template at
/usr/local/bin/tirith. For development, install tirith in the sandbox and setTIRITH_PATH.Files changed
core-api/api/config.pycore-api/.env.example[OPTIONAL] Tirithsectioncore-api/api/services/agents/dispatch.pycore-api/api/services/agents/runtime_bundle.pycore-api/tests/test_tirith_sandbox.pyTests
19 tests covering:
TIRITH_TIMEOUT=abcdoesn't crash importAll tests are self-contained — they load
runtime_bundle.pyby file path to avoidapi.servicesimport side effects and work without Supabase env vars.Scope
This PR is intentionally narrow:
read_file,write_file, oredit_fileHow to try it
Summary by CodeRabbit
New Features
Tests