Skip to content

Add plugin-with-script example - #23

Merged
jpshackelford merged 2 commits into
mainfrom
plugin-with-script
Oct 9, 2026
Merged

jpshackelford merged 2 commits into
mainfrom
plugin-with-script

Conversation

@jpshackelford

Copy link
Copy Markdown
Member

Why

None of the cookbook's plugins has a command or skill that runs a script shipped inside the plugin. The scripts they do have are hook scripts, pasted inline into hooks.json. The usual question is how the instructions can refer to the script when the plugin lands at a path nobody knows in advance.

This example answers that with a self-contained city-weather plugin. Its commands/now.md (the /city-weather:now entry command) and its keyword-triggered skill both tell the agent to run skills/city-weather/scripts/weather.py. That script uses only the Python standard library and the free Open-Meteo API, which needs no key.

It works because OpenHands gives the agent a Skill location: line with the absolute path of the command or skill file. Each file writes the script's path relative to itself. The README explains this, shows where Claude Code's plugin docs put scripts, and explains why ${CLAUDE_SKILL_DIR} and bin/ don't apply in OpenHands.

The example is published to the Plugins, skills & MCP category with example.yaml, and its launch badges were generated with build_launch_url.py.

Validation

  • Real conversations on an OpenHands staging deployment, plugin loaded from this branch with load_plugin.py --ref plugin-with-script — For /city-weather:now Tokyo and for "What's the weather in Paris right now?", the agent took one action: it ran python3 ~/.openhands/cache/plugins/…/city-weather/skills/city-weather/scripts/weather.py "<city>", then finished with the script's report word for word. The first message triggered only the command and the second only the skill.
  • Rendered docs page — The MDX generated for plugin-with-script compiles with @mdx-js/mdx, including the ${CLAUDE_SKILL_DIR} accordion title and the nested code fence in the commands/now.md block.
  • weather.py under pyright — Clean when checked directly (the repo's pyright config only covers conversation-metrics). An unknown city or an unreachable API prints an error and exits with status 1.

This PR was drafted by an AI agent on behalf of the user.

@jpshackelford can click here to continue refining the PR

Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py50100% 
cli.py65395%59–60, 100
client.py1051784%56, 109, 117, 150, 160–162, 173, 203, 206–208, 210, 217–219, 221
metrics.py80298%102, 182
v0.py48394%88, 104, 141
v1.py83989%86, 90, 94, 131, 135, 155, 178, 206, 214
TOTAL3863491% 

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown

Docs preview

✅ Docs preview is ready: https://allhandsai-cookbook-preview-pr-23.mintlify.site

Page Change
/cookbook updated
/cookbook/plugin-with-script added

Docs PR: OpenHands/docs#919 (draft preview; never merged, closes with this PR).
Rendered from ce05666. After merge, a separate sync PR updates the docs.

@jpshackelford

jpshackelford commented Oct 9, 2026 •

Copy link
Copy Markdown
Member Author

Test evidence: the agent runs the bundled script

I started two conversations on our OpenHands Enterprise Beta instance, loading the plugin from this branch:

python load-plugin/load_plugin.py --ref plugin-with-script \
  --repo-path plugin-with-script/city-weather --message "<message>"

They ran at f0287c2. The plugin there matches the PR head ce05666, except that weather.py now passes a literal argparse description instead of reading __doc__.

Message What fired Agent's actions Script exit Result
/city-weather:now Tokyo Command city-weather:now only 1 terminal command, then finish 0 Finished with the script's report, word for word
What's the weather in Paris right now? Skill city-weather only (keyword weather in) 1 terminal command, then finish 0 Finished with the script's report, word for word

In both, the agent built the script's absolute path from the Skill location: line it was given. It didn't call Open-Meteo itself.

Event trace: /city-weather:now Tokyo

What the agent received with the user message:

The following information has been included based on a keyword match for "/city-weather:now".
It may or may not be relevant to the user's request.

Skill location: /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/commands/now.md
(Use this path to resolve relative file references in the skill content below)

The only terminal action:

python3 /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/scripts/weather.py "Tokyo"

Output (exit code 0), which was also the finish message:

City Weather Report for Tokyo, Japan

- Current Time: 2026-10-10 01:30 GMT+9
- Temperature: 62°F / 16°C
- Current Precipitation: 0.0 mm

Precipitation Forecast (Next 4 Hours):
| Time  | Probability |
|-------|-------------|
| 01:00 | 0% |
| 02:00 | 0% |
| 03:00 | 0% |
| 04:00 | 0% |
Event trace: What's the weather in Paris right now?

What the agent received with the user message:

The following information has been included based on a keyword match for "weather in".
It may or may not be relevant to the user's request.

Skill location: /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/SKILL.md
(Use this path to resolve relative file references in the skill content below)

The only terminal action:

python3 /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/scripts/weather.py "Paris"

Output (exit code 0), which was also the finish message:

City Weather Report for Paris, France

- Current Time: 2026-10-09 18:30 GMT+2
- Temperature: 61°F / 16°C
- Current Precipitation: 0.0 mm

Precipitation Forecast (Next 4 Hours):
| Time  | Probability |
|-------|-------------|
| 18:00 | 8% |
| 19:00 | 8% |
| 20:00 | 28% |
| 21:00 | 25% |

The launch badges point at main, so they can only be tried once this is merged.

This comment was drafted by an AI agent on behalf of the user.

@jpshackelford

jpshackelford commented Oct 9, 2026 •

Copy link
Copy Markdown
Member Author

Test results on OpenHands Enterprise Beta, with the plugin pinned to this PR's head

I ran three conversations on our OpenHands Enterprise Beta instance through its public REST API. Each loaded the plugin with ref set to the PR head commit ce056661b25d3b79ec9e1d6d2a6ccc4f2fbf7403. The instance's OpenAPI spec says ref accepts a branch, tag, or commit, so the plugin code under test is exactly this PR's.

Versions: OpenHands app 1.70.1 (from /openapi.json) and agent-server v1.50.1 in each sandbox (from /server_info).

Request (POST /api/v1/app-conversations):

{
  "plugins": [{
    "source": "github:OpenHands/enterprise-cookbook",
    "ref": "ce056661b25d3b79ec9e1d6d2a6ccc4f2fbf7403",
    "repo_path": "plugin-with-script/city-weather",
    "parameters": {"city": "Tokyo"}
  }],
  "initial_message": {"role": "user", "content": [{"type": "text", "text": "/city-weather:now Tokyo"}]}
}

The first case is the "Get the weather in Tokyo" badge's plugin spec and message, with ref changed from main to the commit. The other two cases leave out parameters and change the message.

All three cases behaved as expected.

Case Message Activated Expected Observed Pass
Badge payload, command /city-weather:now Tokyo city-weather:now Run the script, show its report 1 terminal command (exit 0), then finished with the script's report, word for word ✅
Skill What's the weather in Paris right now? city-weather (keyword weather in) Run the script, show its report 1 terminal command (exit 0), then finished with the script's report, word for word ✅
Error path /city-weather:now Xyzzyqqq city-weather:now Run the script; it exits 1 with "No city found", and the agent shows that error and stops 1 terminal command (exit 1), then finished with the script's error and no made-up report ✅

The error path is a deliberate negative test. For an unknown city, weather.py prints No city found named '…' and exits with status 1. Step 4 of commands/now.md (and step 3 of SKILL.md) says: "If it exits with an error, show the error and stop." The pass condition is that the agent reports the failure rather than falling back to calling the API itself or inventing a forecast, and that's what it did.

Every case ran the script at its absolute path in the plugin cache. The path was built from the Skill location: line, and no run called Open-Meteo directly:

python3 /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/scripts/weather.py "Tokyo"

Event-log verification: the script really ran and the agent didn't invent the answer

I pulled each conversation's full event log from the agent server (GET /api/conversations/{id}/events/search) and checked it. Every check passed in all three conversations.

Check Badge payload Skill Error path
Tool calls in the whole conversation terminal, then finish terminal, then finish terminal, then finish
Command matches python3 <plugin cache>/…/scripts/weather.py "<city>" ✅ "Tokyo" ✅ "Paris" ✅ "Xyzzyqqq"
The result event came from source: environment (the sandbox, not the model), with the same tool_call_id and an action_id pointing to the terminal action ✅ ✅ ✅
Exit code recorded by the sandbox, and how long the command took 0, 1.92 s 0, 1.43 s 1, 1.44 s
Report or error text in any agent event (thoughts, reasoning, tool arguments) before the script's result none none none
finish comes from a separate model response, and its parent_id is the script's result event ✅ ✅ ✅
finish message compared with the script's output byte-for-byte identical byte-for-byte identical contains the script's error message unchanged, plus a suggestion to retry
Plugin files in the sandbox's plugin cache: checkout HEAD, and SHA-256 of weather.py, now.md, SKILL.md and plugin.json compared with this PR ce05666…, all 4 match ce05666…, all 4 match ce05666…, all 4 match

All three logs have the same event order: user message → terminal action → terminal result → finish action → finish result. The model issued the script command before any weather data existed in the conversation. The data first appears in the sandbox's result event, and the next model response only repeats it. So the reports came from the bundled script, not from the model.

Trimmed event log: badge payload (/city-weather:now Tokyo)
[
  {"kind": "ActionEvent", "id": "4f1f5aeb-…", "source": "agent", "timestamp": "2026-10-09T17:28:35.603913",
   "tool_name": "terminal", "tool_call_id": "toolu_01BqTX2EmJAdJp8j1NMrvhmw",
   "llm_response_id": "chatcmpl-db02ddc3-…",
   "action": {"kind": "TerminalAction",
              "command": "python3 /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/scripts/weather.py \"Tokyo\""}},
  {"kind": "ObservationEvent", "id": "1d80fe9b-…", "source": "environment", "timestamp": "2026-10-09T17:28:37.519376",
   "parent_id": "4f1f5aeb-…", "action_id": "4f1f5aeb-…", "tool_call_id": "toolu_01BqTX2EmJAdJp8j1NMrvhmw",
   "observation": {"kind": "TerminalObservation", "exit_code": 0, "is_error": false,
                   "metadata": {"exit_code": 0, "username": "openhands", "working_dir": "/workspace/project"},
                   "content": [{"type": "text", "text": "City Weather Report for Tokyo, Japan\n\n- Current Time: 2026-10-10 02:15 GMT+9\n…"}]}},
  {"kind": "ActionEvent", "id": "0d5419dd-…", "source": "agent", "timestamp": "2026-10-09T17:28:39.792277",
   "parent_id": "1d80fe9b-…", "tool_name": "finish", "llm_response_id": "chatcmpl-ca0ce8d8-…",
   "action": {"kind": "FinishAction", "message": "City Weather Report for Tokyo, Japan\n\n- Current Time: 2026-10-10 02:15 GMT+9\n…"}}
]
Output: badge payload (/city-weather:now Tokyo)

The script's output, which was also the agent's finish message:

City Weather Report for Tokyo, Japan

- Current Time: 2026-10-10 02:15 GMT+9
- Temperature: 61°F / 16°C
- Current Precipitation: 0.0 mm

Precipitation Forecast (Next 4 Hours):
| Time  | Probability |
|-------|-------------|
| 02:00 | 0% |
| 03:00 | 0% |
| 04:00 | 0% |
| 05:00 | 0% |
Output: skill (What's the weather in Paris right now?)
Skill location: /home/openhands/.openhands/cache/plugins/enterprise-cookbook-3c5c88fa6e51c50a/plugin-with-script/city-weather/skills/city-weather/SKILL.md

The script's output, which was also the agent's finish message:

City Weather Report for Paris, France

- Current Time: 2026-10-09 19:15 GMT+2
- Temperature: 60°F / 16°C
- Current Precipitation: 0.0 mm

Precipitation Forecast (Next 4 Hours):
| Time  | Probability |
|-------|-------------|
| 19:00 | 8% |
| 20:00 | 28% |
| 21:00 | 25% |
| 22:00 | 33% |
Output: error path (/city-weather:now Xyzzyqqq)

Script output (exit code 1):

No city found named 'Xyzzyqqq'.

The agent's finish message:

I ran the weather script for "Xyzzyqqq" and it failed (exit code 1) with this message:

No city found named 'Xyzzyqqq'.

So I don't have a report for it. If you meant a different city, send the correct name, for example `/city-weather:now Tokyo`.

This tests the API call that the /launch badge route makes after its confirmation dialog. The badges themselves load main, so they can only be clicked through once this is merged.

This comment was drafted by an AI agent on behalf of the user.

@jpshackelford
jpshackelford marked this pull request as ready for review October 9, 2026 17:39

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a well-executed example that fills a real documentation gap. The plugin demonstrates how to bundle and reference scripts in a way that works across Claude Code and OpenHands plugin systems.

Structure & Documentation

  • README follows STYLEGUIDE.md precisely: proper sections, Title Case headings, tabs for alternatives, cards for Related
  • Embedded code block (line 139) matches city-weather/commands/now.md exactly
  • Launch badges correctly use source: github:OpenHands/enterprise-cookbook, ref: main
  • Added to root README in the correct category position

Code Quality

  • weather.py is clean: standard library only, proper error handling, Python 3.10+ syntax
  • No security concerns: uses free public API, no credentials required
  • Plugin structure follows Claude Code layout with OpenHands-specific path resolution explained

Educational Value

  • Clearly explains the Skill location: mechanism and why ${CLAUDE_SKILL_DIR} doesn't work in OpenHands
  • Shows both command and skill patterns referencing the same script
  • Documents the hook limitation (can't reference bundled scripts) with rationale

Validation Evidence (from PR description)

  • Tested on staging deployment with both command and skill triggers
  • MDX rendering passes
  • Script verified with pyright

[RISK ASSESSMENT]
Risk Level: LOW
Reasoning: Pure additive documentation example with no impact on existing functionality. Script is self-contained and uses only public APIs.


Was this automated review useful? React with 👍 or 👎 to this review to help us measure review quality.
Workflow run: https://github.com/OpenHands/enterprise-cookbook/actions/runs/37967778981

@jpshackelford
jpshackelford merged commit 25bf9b7 into main Oct 9, 2026
9 checks passed
@jpshackelford
jpshackelford deleted the plugin-with-script branch October 9, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants