Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Create, attach to, and tear down sandboxes (runtimes).
| [start-sandbox](./start-sandbox/) | Start a sandbox (no conversation) and run commands via the agent-server REST API |
| [clone-and-attach](./clone-and-attach/) | Clone a repo + run `.openhands/setup.sh` in a sandbox, then attach a conversation to it |
| [archive-sandbox](./archive-sandbox/) | Archive/delete a conversation to release its **Persistent Volume Claim (PVC)** and free up storage resources |
| [warm-sandbox-pool](./warm-sandbox-pool/) | Keep a pool of pre-initialized "warm" sandboxes that late-bind to conversations on demand, taking sandbox startup and initialization out of the end user's wait |

### Conversation monitoring & reacting

Expand Down
107 changes: 54 additions & 53 deletions warm-sandbox-pool/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,25 @@

Demonstrates maintaining a pool of pre-initialized "warm" sandboxes that late-bind to conversations on demand, taking sandbox startup and initialization out of the end user's wait.

**This example demonstrates a technique for deploying Ruby-based applications** using OpenHands Cloud APIs, showing that custom images are not the only viable approach for handling initialization that takes more than a few seconds.
**This example demonstrates a technique for any situation where expensive setup must finish before an agent can start working**, using OpenHands Cloud APIs. It shows that custom images are not the only viable approach for handling initialization that takes more than a few seconds, and it can also be combined with custom images.

## Context: Alternative to Custom Images
<details>
<summary>Context: Alternative to Custom Images</summary>

When applications have components that run outside the agent control loop and must be available on the system where the agent is running, a custom image is not the only mechanism for packaging these dependencies.

Although OpenHands Enterprise has robust support for [custom sandbox images](https://docs.openhands.dev/enterprise/custom-sandbox-images), custom images are not yet supported on OpenHands SaaS. Where they are supported, a custom image may still not fully address the situation:

- The required setup may change over a shorter period than is practical for rebuilding the image.
- A significant part of the setup may vary so much by task that an image for each variation is impractical.

Even when using custom images in OpenHands Enterprise, some scenarios require additional tasks to be completed on the running sandbox to make it ready for use. **If these tasks take more than a few seconds, the Warm Sandbox Pool technique removes that delay from what an end-user waits for** by preparing a pool of pre-initialized sandboxes that late-bind to conversations when an end-user begins to interact with the agent.

This same approach can be used to install and prepare application services in sandboxes via API calls available in the OpenHands SaaS/Cloud platform, providing a viable alternative to custom images for your deployment needs.
This same approach can be used to install and prepare application services in sandboxes via API calls available in OpenHands SaaS, providing a viable alternative to custom images for your deployment needs.

</details>

## Concept
## How It Works

Instead of waiting for sandbox provisioning and initialization every time a user starts a conversation, this approach:

Expand All @@ -25,54 +33,31 @@ This is particularly valuable when:
- Setup tasks take more than a few seconds (installing Ruby, gems, starting services)
- You want consistent, fast conversation startup times
- You need services running and ready before the agent starts working
- You're using the OpenHands SaaS/Cloud platform without custom images

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│ Web UI │
├──────────────────────────┬──────────────────────────────────┤
│ Pool Visualization │ Conversation Interface │
│ │ │
│ 🟢 Sandbox 1: READY │ [Waiting for pool to be ready] │
│ 🟡 Sandbox 2: PREPARING │ │
│ 🔴 Sandbox 3: STARTING │ [Then: conversation input box] │
└──────────────────────────┴──────────────────────────────────┘
│
▼
┌────────────────────────┐
│ Pool Controller │
│ (Flask Backend) │
└────────────────────────┘
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Sandbox │ │ Sandbox │ │ Sandbox │
│ (READY) │ │ (READY) │ │ (READY) │
└──────────┘ └──────────┘ └──────────┘
│ │ │
└─────────────────┴─────────────────┘
│
Each sandbox has Ruby +
Sinatra gem + demo service
- You're using OpenHands SaaS, where custom images are not yet supported

### Architecture

```mermaid
flowchart TD
subgraph UI["Web UI"]
PV["Pool Visualization<br/>🟢 Sandbox 1: READY<br/>🟡 Sandbox 2: PREPARING<br/>🔴 Sandbox 3: STARTING"]
CI["Conversation Interface<br/>[Waiting for pool to be ready]<br/>[Then: conversation input box]"]
end
UI --> PC["Pool Controller<br/>(Flask Backend)"]
PC --> S1["Sandbox<br/>(READY)"]
PC --> S2["Sandbox<br/>(READY)"]
PC --> S3["Sandbox<br/>(READY)"]
S1 & S2 & S3 --- N["Each sandbox has Ruby +<br/>Sinatra gem + demo service"]
```

## Demo Application: Ruby Sinatra Service

This example installs a Ruby/Sinatra web service in each sandbox to demonstrate the warm pool technique in a realistic scenario. **The Sinatra service demonstrates how Ruby-based systems** can be installed and running before agent conversations begin.

The initialization process (installing Ruby runtime, gems, starting the service) shows how to deploy application services using the same API-driven preparation approach for your specific use case.

## Quick Start

### Prerequisites
## Prerequisites

- Python 3.10+
- OpenHands Cloud API key (`OH_API_KEY`)
- `uv` or standard Python environment

## Run It

### Installation

```bash
Expand All @@ -83,6 +68,8 @@ uv pip install -e .
pip install -r requirements.txt
```

<!-- docs:tabs -->

### Run the Demo

```bash
Expand All @@ -94,6 +81,7 @@ Then open http://localhost:12000 in your browser. The controller prints a random
**access code** when it starts (look for `ACCESS CODE:` in its output). Paste it into
the page to sign in. Nothing is served without it, and a restart issues a new code.

> [!WARNING]
> **Heads up: this creates real sandboxes.** The controller immediately starts
> `POOL_SIZE` sandboxes (default 3) and keeps topping the pool up as you use them.
> Keep the defaults (pool of 3, threshold 2): a smaller pool leaves nothing to watch,
Expand Down Expand Up @@ -137,6 +125,7 @@ Steps for the agent (or for you, in a sandbox terminal):
This was verified through a real work URL: sign-in, the page, the live stream, and
starting a conversation all work through the proxy.

> [!WARNING]
> **Security: what the access code does and does not protect.** A work URL is reachable
> by anyone who has it, so the controller gates **everything** (the page, the status
> API, the live stream, and Start Conversation) behind a random one-time access code.
Expand All @@ -159,6 +148,8 @@ starting a conversation all work through the proxy.
> rate limiting on guesses. For anything longer-lived, put real authentication in
> front of it.

<!-- /docs:tabs -->

### What to Expect (and what the pool does not speed up)

The pool removes **sandbox boot plus your init script** from the user's wait. It does
Expand Down Expand Up @@ -214,10 +205,16 @@ Once a sandbox is in READY state, the Sinatra service is listening on port 4567
**inside** the sandbox. That port is not exposed publicly, so reach it from the
sandbox itself. The easiest way is to start a conversation and ask the agent:

```
```text
"Call the quote service running on localhost:4567 and show me today's quote"
```

## Demo Application: Ruby Sinatra Service

This example installs a Ruby/Sinatra web service in each sandbox to demonstrate the warm pool technique in a realistic scenario. **The Ruby service is only an example:** any long-running startup job can be installed and running before agent conversations begin, such as cloning a very large monorepo or downloading the Maven dependencies of a Java monorepo with many projects.

The initialization process (installing Ruby runtime, gems, starting the service) shows how to deploy application services using the same API-driven preparation approach for your specific use case.

## Implementation Details

### Sandbox Initialization Process
Expand Down Expand Up @@ -259,7 +256,7 @@ failed, deleted, refilling halted.

## Files

```
```text
warm-sandbox-pool/
├── README.md # This file
├── QUICKSTART.md # Short run-it-now guide
Expand Down Expand Up @@ -302,7 +299,7 @@ redis-server --daemonize yes

### 3. Large Codebases

Clone and prepare large repositories with dependencies:
Clone and prepare large repositories with dependencies, such as a very large monorepo or a Java monorepo with heavy Maven dependencies:

```bash
# In init script
Expand Down Expand Up @@ -353,7 +350,7 @@ By pre-warming sandboxes with your application already running, agents can use y
| **Just-in-Time Init** | Simplest code, minimal resources | Slow user experience, wait time on every conversation |

**Warm Sandbox Pool is ideal when**:
- You're on OpenHands SaaS/Cloud (custom images not available)
- You're on OpenHands SaaS (custom images not yet supported)
- Setup time is 10-60 seconds (too slow for UX, too fast to justify custom image complexity)
- You want flexibility to change initialization without rebuilding images
- You have predictable conversation volume
Expand Down Expand Up @@ -444,11 +441,15 @@ Reduce `POOL_SIZE` or implement smarter pool management:
4. **Dynamic Scaling**: Adjust pool size based on demand
5. **Cost Optimization**: Implement sandbox recycling (reset instead of destroy)

## Related Examples
## Related

<!-- docs:cards -->

- [`start-sandbox`](../start-sandbox/) - Basic sandbox provisioning
- [`clone-and-attach`](../clone-and-attach/) - Conversation attachment patterns
- [`upload-skills`](../upload-skills/) - Pre-loading agent skills

- `start-sandbox/` - Basic sandbox provisioning
- `clone-and-attach/` - Conversation attachment patterns
- `upload-skills/` - Pre-loading agent skills
<!-- /docs:cards -->

## License

Expand Down
5 changes: 5 additions & 0 deletions warm-sandbox-pool/example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: Warm Sandbox Pool
description: Keep a pool of pre-initialized sandboxes warm and attach conversations to them on demand.
category: Sandbox lifecycle
icon: fire
Loading