From 1ee59a38b7c873071d8a06adbceb2cb674118543 Mon Sep 17 00:00:00 2001 From: openhands Date: Mon, 6 Jul 2026 08:10:48 +0000 Subject: [PATCH 1/9] Add warm-sandbox-pool example Demonstrates maintaining a pool of pre-initialized warm sandboxes for Ruby-based applications, eliminating startup delays by late-binding conversations to prepared sandboxes. Features: - Pre-initialized sandboxes with Ruby/Sinatra already running - Automatic pool refill when sandboxes are consumed - Web UI for monitoring pool status and starting conversations - Alternative to custom images using OpenHands Cloud APIs Recovered from conversation 4c1947abfc4f4ed6ba76d9b4c7324d8f Co-authored-by: openhands --- warm-sandbox-pool/.gitignore | 14 + warm-sandbox-pool/QUICKSTART.md | 226 ++++++++ warm-sandbox-pool/README.md | 342 ++++++++++++ warm-sandbox-pool/pool_controller.py | 515 ++++++++++++++++++ warm-sandbox-pool/pyproject.toml | 18 + warm-sandbox-pool/requirements.txt | 3 + .../sandbox_prep/init_ruby_service.sh | 139 +++++ .../sandbox_prep/quote_service.rb | 80 +++ warm-sandbox-pool/static/app.js | 253 +++++++++ warm-sandbox-pool/static/styles.css | 458 ++++++++++++++++ warm-sandbox-pool/templates/index.html | 135 +++++ warm-sandbox-pool/test_structure.py | 80 +++ 12 files changed, 2263 insertions(+) create mode 100644 warm-sandbox-pool/.gitignore create mode 100644 warm-sandbox-pool/QUICKSTART.md create mode 100644 warm-sandbox-pool/README.md create mode 100644 warm-sandbox-pool/pool_controller.py create mode 100644 warm-sandbox-pool/pyproject.toml create mode 100644 warm-sandbox-pool/requirements.txt create mode 100644 warm-sandbox-pool/sandbox_prep/init_ruby_service.sh create mode 100644 warm-sandbox-pool/sandbox_prep/quote_service.rb create mode 100644 warm-sandbox-pool/static/app.js create mode 100644 warm-sandbox-pool/static/styles.css create mode 100644 warm-sandbox-pool/templates/index.html create mode 100644 warm-sandbox-pool/test_structure.py diff --git a/warm-sandbox-pool/.gitignore b/warm-sandbox-pool/.gitignore new file mode 100644 index 0000000..bd53561 --- /dev/null +++ b/warm-sandbox-pool/.gitignore @@ -0,0 +1,14 @@ +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +env/ +venv/ +.venv/ +*.egg-info/ +dist/ +build/ +.DS_Store +*.log +.env diff --git a/warm-sandbox-pool/QUICKSTART.md b/warm-sandbox-pool/QUICKSTART.md new file mode 100644 index 0000000..1cc45c6 --- /dev/null +++ b/warm-sandbox-pool/QUICKSTART.md @@ -0,0 +1,226 @@ +# Quick Start Guide + +Get the Warm Sandbox Pool demo running in 5 minutes. + +## Prerequisites + +- **OpenHands Cloud API Key** - Get one from https://app.all-hands.dev +- **Python 3.10+** - Check with `python --version` +- **Git** - To clone this repository + +## Installation (30 seconds) + +```bash +# Clone the repository +git clone https://github.com/jpshackelford/oh-examples.git +cd oh-examples/warm-sandbox-pool + +# Install Python dependencies (use uv or pip) +pip install -r requirements.txt + +# Or with uv (faster) +uv pip install -e . +``` + +## Run the Demo (10 seconds) + +```bash +# Set your API key +export OH_API_KEY=your_api_key_here + +# Start the pool controller +python pool_controller.py + +# You should see: +# INFO pool_controller: Pool controller initialized: size=3, threshold=2 +# INFO pool_controller: Pool manager thread started +# INFO pool_controller: Starting web server on port 5000 +# INFO pool_controller: Open http://localhost:5000 in your browser +``` + +## Use the Demo (2-3 minutes) + +1. **Open http://localhost:5000** in your browser + +2. **Watch the pool initialize**: + - You'll see "Preparing pool..." message + - 3 sandboxes will start initializing (takes ~30-60 seconds each) + - Watch them progress: πŸ”΄ STARTING β†’ 🟑 PREPARING β†’ 🟒 READY + +3. **Start a conversation**: + - Once at least one sandbox is READY, the conversation UI appears + - Type a message or use the default: + ``` + Check if the quote service is running on localhost:4567 and fetch me a random quote. + ``` + - Click "πŸš€ Start Conversation" + +4. **Observe the magic**: + - A ready sandbox is instantly pulled from the pool + - OpenHands conversation starts immediately (no wait!) + - Click the conversation link to see your agent working + - Watch the pool automatically provision a new sandbox to refill + +## What's Happening? + +``` +Before (Without Warm Pool): +User clicks "Start" β†’ Wait 30-60s for sandbox β†’ Agent starts + ⏳ User sees delay + +With Warm Pool: +User clicks "Start" β†’ Instant sandbox β†’ Agent starts + ⚑ No apparent delay + (Pool refills in background) +``` + +## Try These Commands in Your Conversation + +Once your conversation is attached to a warm sandbox: + +``` +"Show me the Ruby version installed" +β†’ Should show Ruby 3.x already installed + +"Check if the quote service is running on port 4567" +β†’ Should show it's already running + +"Fetch a quote from localhost:4567/quote and show it to me" +β†’ Should return a programming quote from the API + +"Get all quotes from localhost:4567/quotes and count them" +β†’ Should list 8 quotes + +"Find quotes by Martin Fowler using the API" +β†’ Should search the quotes service +``` + +## Next Steps + +### For Demo/Learning +- Experiment with different pool sizes: `POOL_SIZE=5 python pool_controller.py` +- Watch the auto-refill behavior by starting multiple conversations +- Check the sandbox init logs in the web UI + +### For Custom Application Deployment +1. **Customize** `sandbox_prep/init_ruby_service.sh` with your application setup +2. **Test** the initialization manually before running the full pool +3. **Adjust** pool size and timing parameters for your needs +4. **Deploy** your adapted pool controller + +## Configuration + +Customize via environment variables: + +```bash +# Basic config +export OH_API_KEY=your_key +export OH_API_BASE=https://app.all-hands.dev # or your instance +export PORT=5000 # Web UI port + +# Pool sizing +export POOL_SIZE=3 # How many ready sandboxes to maintain +export POOL_THRESHOLD=2 # Start refilling when ready count drops below this + +# Sandbox config (optional) +export SANDBOX_SPEC_ID=your_spec_id # Use custom runtime image +``` + +## Troubleshooting + +### "No ready sandboxes available" error +- **Wait longer**: Initial pool fill takes 1-3 minutes +- **Check API key**: Make sure `OH_API_KEY` is valid +- **View logs**: Look at terminal output for error messages +- **Check web UI**: Look for FAILED sandboxes and error messages + +### Sandboxes stuck in PREPARING +- **Check init script**: May be timing out or failing +- **Increase timeout**: Set higher init timeout in code +- **Test manually**: Use `start-sandbox/` example to debug + +### "OH_API_KEY is required" error +```bash +# Set your API key before running +export OH_API_KEY=your_actual_key_here +python pool_controller.py +``` + +### Port already in use +```bash +# Use a different port +PORT=8080 python pool_controller.py +``` + +## Architecture at a Glance + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Browser (You) β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ http://localhost:5000 + β–Ό +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Flask Web Server β”‚ +β”‚ - Serves UI β”‚ +β”‚ - REST API β”‚ +β”‚ - SSE for real-time β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + β–Ό +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Pool Controller β”‚ +β”‚ - Manages 3 sandboxes β”‚ +β”‚ - Auto-refills pool β”‚ +β”‚ - Attaches conversationsβ”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + β”Œβ”€β”€β”€β”€β”΄β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β” + β–Ό β–Ό β–Ό +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚Sandbox1β”‚ β”‚Sandbox2β”‚ β”‚Sandbox3β”‚ +β”‚ Ruby βœ“ β”‚ β”‚ Ruby βœ“ β”‚ β”‚ Ruby βœ“ β”‚ +β”‚ Gems βœ“ β”‚ β”‚ Gems βœ“ β”‚ β”‚ Gems βœ“ β”‚ +β”‚Serviceβœ“β”‚ β”‚Serviceβœ“β”‚ β”‚Serviceβœ“β”‚ +β”‚ READY β”‚ β”‚ READY β”‚ β”‚ READY β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +## Files Overview + +``` +warm-sandbox-pool/ +β”œβ”€β”€ README.md πŸ“– Comprehensive documentation +β”œβ”€β”€ QUICKSTART.md ⚑ This file - get started fast +β”œβ”€β”€ pool_controller.py πŸŽ›οΈ Main backend (Flask + pool logic) +β”œβ”€β”€ requirements.txt πŸ“¦ Python dependencies +β”œβ”€β”€ pyproject.toml πŸ“¦ UV-compatible project config +β”œβ”€β”€ sandbox_prep/ +β”‚ β”œβ”€β”€ init_ruby_service.sh πŸ”§ Demo: Ruby/Sinatra init +β”‚ └── quote_service.rb πŸ’Ž Demo Ruby service +β”œβ”€β”€ static/ +β”‚ β”œβ”€β”€ app.js 🎨 Frontend JavaScript +β”‚ └── styles.css 🎨 UI styling +└── templates/ + └── index.html πŸ“„ Web UI template +``` + +## Getting Help + +- **Documentation**: Read `README.md` for full details +- **Example Code**: All files are heavily commented +- **Related Examples**: Check `../start-sandbox/` and `../clone-and-attach/` + +## Success Criteria + +You've successfully run the demo when: + +βœ… Pool shows 3 sandboxes in READY state +βœ… You can start a conversation and get instant allocation +βœ… The conversation link opens in OpenHands +βœ… The agent can interact with the pre-installed quote service +βœ… Pool automatically provisions a replacement sandbox + +**Total time from zero to working demo: ~5 minutes** + +Now you understand how warm sandbox pools work and can adapt this technique for your own application deployment! diff --git a/warm-sandbox-pool/README.md b/warm-sandbox-pool/README.md new file mode 100644 index 0000000..a638f5f --- /dev/null +++ b/warm-sandbox-pool/README.md @@ -0,0 +1,342 @@ +# Warm Sandbox Pool + +Demonstrates maintaining a pool of pre-initialized "warm" sandboxes that late-bind to conversations on demand, eliminating startup delays for end users. + +**This example demonstrates a technique for deploying Ruby-based applications** using OpenHands Cloud APIs, showing that custom images are not the only viable approach for handling initialization that takes more than a few seconds. + +## Context: Alternative to Custom Images + +When applications have components that run outside the agent control loop and must be available on the system where the agent is running, a custom image is not the only mechanism for packaging these dependencies. + +Even when using custom images in OpenHands Enterprise, some scenarios require additional tasks to be completed on the running sandbox to make it ready for use. **If these tasks take more than a few seconds, the Warm Sandbox Pool technique eliminates the apparent delay an end-user would see** by preparing a pool of pre-initialized sandboxes that late-bind to conversations when an end-user begins to interact with the agent. + +This same approach can be used to install and prepare application services in sandboxes via API calls available in the OpenHands SaaS/Cloud platform, providing a viable alternative to custom images for your deployment needs. + +## Concept + +Instead of waiting for sandbox provisioning and initialization every time a user starts a conversation, this approach: + +1. **Maintains a pool** of pre-initialized sandboxes (e.g., 3 sandboxes) +2. **Pre-installs dependencies** (Ruby, gems, application services) during sandbox preparation +3. **Late-binds conversations** - when a user needs a sandbox, one is pulled from the pool instantly +4. **Auto-refills** - when pool drops below threshold, new sandboxes are automatically provisioned and prepared in the background + +This is particularly valuable when: +- Setup tasks take more than a few seconds (installing Ruby, gems, starting services) +- You want consistent, fast conversation startup times +- You need services running and ready before the agent starts working +- You're using the OpenHands SaaS/Cloud platform without custom images + +## Architecture + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ Web UI β”‚ +β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ +β”‚ Pool Visualization β”‚ Conversation Interface β”‚ +β”‚ β”‚ β”‚ +β”‚ 🟒 Sandbox 1: READY β”‚ [Waiting for pool to be ready] β”‚ +β”‚ 🟑 Sandbox 2: PREPARING β”‚ β”‚ +β”‚ πŸ”΄ Sandbox 3: STARTING β”‚ [Then: conversation input box] β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + β–Ό + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ Pool Controller β”‚ + β”‚ (Flask Backend) β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β–Ό β–Ό β–Ό + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” + β”‚ Sandbox β”‚ β”‚ Sandbox β”‚ β”‚ Sandbox β”‚ + β”‚ (READY) β”‚ β”‚ (READY) β”‚ β”‚ (READY) β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ β”‚ β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ + β”‚ + Each sandbox has Ruby + + Sinatra gem + demo service +``` + +## Demo Application: Ruby Sinatra Service + +This example installs a Ruby/Sinatra web service in each sandbox to demonstrate the warm pool technique in a realistic scenario. **The Sinatra service demonstrates how Ruby-based systems** can be installed and running before agent conversations begin. + +The initialization process (installing Ruby runtime, gems, starting the service) shows how to deploy application services using the same API-driven preparation approach for your specific use case. + +## Quick Start + +### Prerequisites + +- Python 3.10+ +- OpenHands Cloud API key (`OH_API_KEY`) +- `uv` or standard Python environment + +### Installation + +```bash +# Install dependencies +uv pip install -e . + +# Or with pip +pip install -r requirements.txt +``` + +### Run the Demo + +```bash +export OH_API_KEY=your_api_key_here +python pool_controller.py +``` + +Then open http://localhost:5000 in your browser. + +### What You'll See + +1. **Initial State**: "Preparing pool..." message while 3 sandboxes initialize +2. **Pool Visualization**: Real-time status of each sandbox: + - πŸ”΄ **STARTING**: OpenHands is provisioning the sandbox + - 🟑 **PREPARING**: Installing Ruby, gems, starting service + - 🟒 **READY**: Fully initialized and available +3. **Conversation UI**: Once pool is ready, type a message to start a conversation +4. **Sandbox Allocation**: A ready sandbox is pulled from the pool and attached to your conversation +5. **Auto-Refill**: Watch as a new sandbox automatically begins initializing to refill the pool + +### Configuration + +Environment variables: + +- `OH_API_KEY`: OpenHands Cloud API key (required) +- `OH_API_BASE`: API base URL (default: `https://app.all-hands.dev`) +- `POOL_SIZE`: Target pool size (default: `3`) +- `POOL_THRESHOLD`: Trigger refill when pool drops below this (default: `2`) +- `PORT`: Web server port (default: `5000`) + +### Testing the Ruby Service + +Once a sandbox is in READY state, the Sinatra service is running on port 4567. You can test it: + +```bash +# Get the sandbox ID from the web UI, then: +curl https://work-1-.prod-runtime.all-hands.dev:12000/quote +``` + +Or ask the agent to interact with it: +``` +"Call the quote service running on localhost:4567 and show me today's quote" +``` + +## Implementation Details + +### Sandbox Initialization Process + +Each sandbox goes through these preparation steps (see `sandbox_prep/init_ruby_service.sh`): + +1. **Install Ruby**: Uses rbenv to install Ruby 3.2 +2. **Install Sinatra**: Gem install sinatra +3. **Deploy Service**: Copies the quote service code +4. **Start Service**: Launches the Sinatra app in the background +5. **Verify**: Confirms the service responds to health checks + +This simulates a realistic scenario where your agent needs specific tools/services pre-installed. + +### Pool Management + +The `PoolController` class handles: + +- **Provisioning**: Creates sandboxes via OpenHands Cloud API +- **Monitoring**: Polls sandbox status until RUNNING +- **Initialization**: Executes preparation scripts via agent-server API +- **Queue Management**: Thread-safe queue of ready sandboxes +- **Auto-Refill**: Background thread maintains pool size +- **Conversation Binding**: Attaches conversations to pre-warmed sandboxes + +### Real-Time Updates + +The web UI uses Server-Sent Events (SSE) to stream pool state updates in real-time without polling. + +## Files + +``` +warm-sandbox-pool/ +β”œβ”€β”€ README.md # This file +β”œβ”€β”€ pool_controller.py # Flask backend + pool manager +β”œβ”€β”€ requirements.txt # Python dependencies +β”œβ”€β”€ sandbox_prep/ +β”‚ β”œβ”€β”€ init_ruby_service.sh # Bash script to initialize each sandbox +β”‚ └── quote_service.rb # Ruby/Sinatra demo service +β”œβ”€β”€ static/ +β”‚ β”œβ”€β”€ app.js # Frontend JavaScript +β”‚ └── styles.css # UI styling +└── templates/ + └── index.html # Main web page +``` + +## Use Cases + +### 1. Custom Runtime Environments + +Pre-install language runtimes (Ruby, Java, Go) that take time to set up: + +```bash +# In init script +rbenv install 3.2.0 +rbenv global 3.2.0 +gem install rails bundler +``` + +### 2. Service Dependencies + +Start databases, caches, or mock APIs before the agent runs: + +```bash +# In init script +docker run -d -p 5432:5432 postgres +redis-server --daemonize yes +./mock-api-server & +``` + +### 3. Large Codebases + +Clone and prepare large repositories with dependencies: + +```bash +# In init script +git clone --depth 1 https://github.com/your-org/monorepo /workspace/project +cd /workspace/project +bundle install +npm install +make build +``` + +### 4. Custom Application Integration + +Replace the demo Sinatra service with your actual application initialization: + +```bash +# In init script (replace sandbox_prep/init_ruby_service.sh contents) +# Install Ruby (or use a runtime that already has it) +rbenv install 3.2.0 +rbenv global 3.2.0 + +# Install your application gems +gem install your_gem_name + +# Initialize your application +cd /workspace +# ... your app-specific setup commands +bundle install + +# Start your services in daemon mode +rails server -d -p 4567 + +# Wait for service to be ready +until curl -f http://localhost:4567/health; do + echo "Waiting for application..." + sleep 2 +done + +echo "βœ… Application ready" +``` + +By pre-warming sandboxes with your application already running, agents can immediately interact with your services without the 30-60 second initialization delay on every conversation start. + +## Benefits vs. Custom Images + +| Approach | Pros | Cons | +|----------|------|------| +| **Custom Images** | Fastest cold start, baked-in dependencies | Requires OpenHands Enterprise, image build pipeline, version management | +| **Warm Sandbox Pool** | Works on SaaS, flexible initialization, no image builds | Requires pool management code, higher resource usage | +| **Just-in-Time Init** | Simplest code, minimal resources | Slow user experience, wait time on every conversation | + +**Warm Sandbox Pool is ideal when**: +- You're on OpenHands SaaS/Cloud (custom images not available) +- Setup time is 10-60 seconds (too slow for UX, too fast to justify custom image complexity) +- You want flexibility to change initialization without rebuilding images +- You have predictable conversation volume + +## Extending the Example + +### Add More Preparation Steps + +Edit `sandbox_prep/init_ruby_service.sh` to install additional tools: + +```bash +# Install Node.js +curl -fsSL https://deb.nodesource.com/setup_20.x | bash - +apt-get install -y nodejs + +# Install Python packages +pip install pandas numpy jupyter +``` + +### Customize the Demo Service + +Replace `sandbox_prep/quote_service.rb` with your own Ruby application or gem. + +### Adjust Pool Parameters + +```python +# In pool_controller.py +POOL_SIZE = 5 # Maintain 5 ready sandboxes +POOL_THRESHOLD = 3 # Refill when below 3 +INIT_TIMEOUT = 600 # Allow 10 minutes for initialization +``` + +### Add Health Checks + +Extend the initialization to verify services are actually ready: + +```bash +# In init script +until curl -f http://localhost:4567/health; do + echo "Waiting for service..." + sleep 2 +done +``` + +## Troubleshooting + +### Pool Never Reaches Ready State + +Check the initialization logs in the web UI. Common issues: +- Ruby installation timeout (increase `INIT_TIMEOUT`) +- Network issues downloading gems +- Insufficient sandbox resources + +### Sandboxes Get Stuck in PREPARING + +Look at the agent-server command output. The init script may be failing. Test it manually: + +```bash +# Get a sandbox URL from the UI, then: +curl -X POST https:///api/bash/execute_bash_command \ + -H "X-Session-API-Key: " \ + -d '{"command": "bash /workspace/init_ruby_service.sh", "timeout": 300}' +``` + +### High Resource Usage + +Reduce `POOL_SIZE` or implement smarter pool management: +- Scale pool size based on time of day +- Implement idle timeout (destroy sandboxes after 30 min unused) +- Use pool only for peak hours, fall back to JIT otherwise + +## Next Steps + +1. **Production Deployment**: Add error handling, logging, metrics +2. **Persistent Storage**: Save pool state to Redis/database for crash recovery +3. **Multi-Tenant**: Separate pools per user/organization +4. **Dynamic Scaling**: Adjust pool size based on demand +5. **Cost Optimization**: Implement sandbox recycling (reset instead of destroy) + +## Related Examples + +- `start-sandbox/` - Basic sandbox provisioning +- `clone-and-attach/` - Conversation attachment patterns +- `upload-skills/` - Pre-loading agent skills + +## License + +MIT - See repository root LICENSE file diff --git a/warm-sandbox-pool/pool_controller.py b/warm-sandbox-pool/pool_controller.py new file mode 100644 index 0000000..725d1aa --- /dev/null +++ b/warm-sandbox-pool/pool_controller.py @@ -0,0 +1,515 @@ +#!/usr/bin/env python3 +""" +Warm Sandbox Pool Controller + +Maintains a pool of pre-initialized OpenHands sandboxes with Ruby/Sinatra services +already running. When users start conversations, sandboxes are pulled from the pool +and attached instantly, with automatic refill. + +Usage: + export OH_API_KEY=your_key_here + python pool_controller.py + +Then open http://localhost:5000 in your browser. +""" + +import argparse +import json +import logging +import os +import queue +import sys +import threading +import time +from dataclasses import dataclass, field +from datetime import datetime +from enum import Enum +from pathlib import Path +from typing import Any + +import requests +from flask import Flask, jsonify, render_template, request +from flask_cors import CORS + +# Configure logging +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s [%(levelname)s] %(name)s: %(message)s", + datefmt="%Y-%m-%d %H:%M:%S", +) +logger = logging.getLogger("pool_controller") + + +class SandboxState(str, Enum): + """States a sandbox progresses through in the pool.""" + + STARTING = "STARTING" # OpenHands provisioning the sandbox + PREPARING = "PREPARING" # Running initialization script + READY = "READY" # Fully initialized and available + ALLOCATED = "ALLOCATED" # Pulled from pool and attached to conversation + FAILED = "FAILED" # Initialization failed + + +@dataclass +class PooledSandbox: + """Represents a sandbox in the pool with its current state.""" + + id: str + state: SandboxState + agent_url: str | None = None + session_api_key: str | None = None + created_at: datetime = field(default_factory=datetime.now) + ready_at: datetime | None = None + allocated_at: datetime | None = None + conversation_id: str | None = None + error_message: str | None = None + init_log: list[str] = field(default_factory=list) + + def to_dict(self) -> dict: + return { + "id": self.id, + "state": self.state.value, + "agent_url": self.agent_url, + "created_at": self.created_at.isoformat(), + "ready_at": self.ready_at.isoformat() if self.ready_at else None, + "allocated_at": self.allocated_at.isoformat() if self.allocated_at else None, + "conversation_id": self.conversation_id, + "error_message": self.error_message, + "init_log": self.init_log[-10:], # Last 10 log lines + } + + +class PoolController: + """Manages a pool of warm OpenHands sandboxes.""" + + def __init__( + self, + api_key: str, + base_url: str, + pool_size: int = 3, + threshold: int = 2, + sandbox_spec_id: str | None = None, + ): + self.api_key = api_key + self.base_url = base_url.rstrip("/") + self.pool_size = pool_size + self.threshold = threshold + self.sandbox_spec_id = sandbox_spec_id + + self.headers = {"X-Session-API-Key": api_key} + self.ready_queue: queue.Queue[PooledSandbox] = queue.Queue() + self.all_sandboxes: dict[str, PooledSandbox] = {} + self.lock = threading.RLock() + self.running = True + + # Load initialization script + self.init_script = self._load_init_script() + + logger.info( + f"Pool controller initialized: size={pool_size}, threshold={threshold}" + ) + + def _load_init_script(self) -> str: + """Load the sandbox initialization script.""" + script_path = Path(__file__).parent / "sandbox_prep" / "init_ruby_service.sh" + if not script_path.exists(): + raise FileNotFoundError(f"Init script not found: {script_path}") + return script_path.read_text() + + def start(self) -> None: + """Start the pool management background thread.""" + thread = threading.Thread(target=self._pool_manager_loop, daemon=True) + thread.start() + logger.info("Pool manager thread started") + + def _pool_manager_loop(self) -> None: + """Background thread that maintains the pool at the target size.""" + # Initial fill + logger.info(f"Initial pool fill to {self.pool_size} sandboxes") + for _ in range(self.pool_size): + self._provision_sandbox() + + # Maintenance loop + while self.running: + time.sleep(5) + ready_count = self.ready_queue.qsize() + if ready_count < self.threshold: + needed = self.pool_size - self._total_initializing_count() - ready_count + if needed > 0: + logger.info( + f"Pool below threshold ({ready_count} < {self.threshold}), " + f"provisioning {needed} sandbox(es)" + ) + for _ in range(needed): + self._provision_sandbox() + + def _total_initializing_count(self) -> int: + """Count sandboxes currently initializing (STARTING or PREPARING).""" + with self.lock: + return sum( + 1 + for sb in self.all_sandboxes.values() + if sb.state in (SandboxState.STARTING, SandboxState.PREPARING) + ) + + def _provision_sandbox(self) -> None: + """Start provisioning a new sandbox (runs in background thread).""" + thread = threading.Thread(target=self._provision_and_init_sandbox, daemon=True) + thread.start() + + def _provision_and_init_sandbox(self) -> None: + """Provision, initialize, and add a sandbox to the ready queue.""" + sandbox = None + try: + # 1. Create sandbox via Cloud API + logger.info("Creating new sandbox...") + params = {"sandbox_spec_id": self.sandbox_spec_id} if self.sandbox_spec_id else None + resp = requests.post( + f"{self.base_url}/api/v1/sandboxes", headers=self.headers, params=params, timeout=30 + ) + resp.raise_for_status() + sb_data = resp.json() + sandbox_id = sb_data["id"] + + sandbox = PooledSandbox(id=sandbox_id, state=SandboxState.STARTING) + with self.lock: + self.all_sandboxes[sandbox_id] = sandbox + logger.info(f"Sandbox {sandbox_id}: STARTING") + + # 2. Poll until RUNNING + sandbox = self._wait_until_running(sandbox, timeout=180) + + # 3. Run initialization script + sandbox.state = SandboxState.PREPARING + logger.info(f"Sandbox {sandbox_id}: PREPARING (running init script)") + self._run_init_script(sandbox) + + # 4. Mark as ready and add to queue + sandbox.state = SandboxState.READY + sandbox.ready_at = datetime.now() + self.ready_queue.put(sandbox) + logger.info( + f"Sandbox {sandbox_id}: READY " + f"(took {(sandbox.ready_at - sandbox.created_at).seconds}s)" + ) + + except Exception as e: + logger.error(f"Failed to provision sandbox: {e}", exc_info=True) + if sandbox: + sandbox.state = SandboxState.FAILED + sandbox.error_message = str(e) + + def _wait_until_running( + self, sandbox: PooledSandbox, timeout: int = 180 + ) -> PooledSandbox: + """Poll sandbox status until it reaches RUNNING state.""" + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + resp = requests.get( + f"{self.base_url}/api/v1/sandboxes", + headers=self.headers, + params={"id": sandbox.id}, + timeout=10, + ) + resp.raise_for_status() + results = resp.json() + if not results or results[0] is None: + raise ValueError(f"Sandbox {sandbox.id} not found") + + sb_data = results[0] + status = sb_data["status"] + sandbox.init_log.append(f"Status: {status}") + + if status == "RUNNING": + # Extract agent-server URL and session key + sandbox.agent_url = self._extract_agent_url(sb_data) + sandbox.session_api_key = sb_data["session_api_key"] + return sandbox + + time.sleep(3) + + raise TimeoutError( + f"Sandbox {sandbox.id} did not reach RUNNING within {timeout}s" + ) + + def _extract_agent_url(self, sandbox_data: dict) -> str: + """Extract AGENT_SERVER URL from sandbox exposed_urls.""" + url = next( + ( + u["url"] + for u in sandbox_data.get("exposed_urls", []) + if u["name"] == "AGENT_SERVER" + ), + None, + ) + if not url: + raise ValueError( + f"AGENT_SERVER URL not found in sandbox {sandbox_data['id']}" + ) + return url + + def _run_init_script(self, sandbox: PooledSandbox) -> None: + """Execute the initialization script in the sandbox.""" + if not sandbox.agent_url or not sandbox.session_api_key: + raise ValueError(f"Sandbox {sandbox.id} missing agent URL or session key") + + session_headers = {"X-Session-API-Key": sandbox.session_api_key} + + # Upload the init script + sandbox.init_log.append("Uploading init script...") + upload_resp = requests.post( + f"{sandbox.agent_url}/api/upload-files", + headers=session_headers, + files={"file": ("init.sh", self.init_script, "text/plain")}, + data={"destination": "/tmp/init.sh"}, + timeout=30, + ) + upload_resp.raise_for_status() + + # Execute the script + sandbox.init_log.append("Executing init script...") + exec_resp = requests.post( + f"{sandbox.agent_url}/api/bash/execute_bash_command", + headers=session_headers, + json={ + "command": f"chmod +x /tmp/init.sh && SANDBOX_ID={sandbox.id} bash /tmp/init.sh", + "timeout": 300, + }, + timeout=320, + ) + exec_resp.raise_for_status() + result = exec_resp.json() + + # Log output + stdout = (result.get("stdout") or "").strip() + stderr = (result.get("stderr") or "").strip() + if stdout: + sandbox.init_log.extend(stdout.split("\n")[-20:]) # Last 20 lines + if stderr: + sandbox.init_log.append(f"[stderr]: {stderr}") + + exit_code = result.get("exit_code", -1) + if exit_code != 0: + raise RuntimeError( + f"Init script failed with exit code {exit_code}: {stderr or stdout}" + ) + + sandbox.init_log.append("βœ… Initialization complete") + + def get_ready_sandbox(self) -> PooledSandbox | None: + """Get a ready sandbox from the pool (non-blocking).""" + try: + sandbox = self.ready_queue.get_nowait() + sandbox.state = SandboxState.ALLOCATED + sandbox.allocated_at = datetime.now() + return sandbox + except queue.Empty: + return None + + def get_pool_status(self) -> dict: + """Get current pool status for the UI.""" + with self.lock: + sandboxes_list = sorted( + [sb.to_dict() for sb in self.all_sandboxes.values()], + key=lambda x: x["created_at"], + ) + + return { + "pool_size": self.pool_size, + "threshold": self.threshold, + "ready_count": self.ready_queue.qsize(), + "sandboxes": sandboxes_list, + "timestamp": datetime.now().isoformat(), + } + + def attach_conversation( + self, sandbox: PooledSandbox, message: str + ) -> str: + """Attach a new conversation to the given sandbox.""" + payload = { + "sandbox_id": sandbox.id, + "initial_message": { + "role": "user", + "content": [{"type": "text", "text": message}], + }, + "title": f"Warm Pool Demo - {datetime.now().strftime('%H:%M:%S')}", + } + + resp = requests.post( + f"{self.base_url}/api/v1/app-conversations", + headers=self.headers, + json=payload, + timeout=30, + ) + resp.raise_for_status() + task = resp.json() + + # Poll start task until we get conversation ID + task_id = task["id"] + conv_id = task.get("app_conversation_id") + timeout = 60 + deadline = time.monotonic() + timeout + + while not conv_id and time.monotonic() < deadline: + time.sleep(2) + resp = requests.get( + f"{self.base_url}/api/v1/app-conversations/start-tasks", + headers=self.headers, + params={"ids": task_id}, + timeout=10, + ) + resp.raise_for_status() + items = resp.json() + item = items[0] if isinstance(items, list) else items + conv_id = item.get("app_conversation_id") + + if not conv_id: + raise TimeoutError(f"Start task {task_id} did not produce conversation ID") + + sandbox.conversation_id = conv_id + logger.info(f"Conversation {conv_id} attached to sandbox {sandbox.id}") + return conv_id + + def shutdown(self) -> None: + """Graceful shutdown.""" + self.running = False + logger.info("Pool controller shutting down") + + +# Flask Application +app = Flask(__name__) +CORS(app) + +pool_controller: PoolController | None = None + + +@app.route("/") +def index(): + """Serve the main UI.""" + return render_template("index.html") + + +@app.route("/api/pool/status") +def pool_status(): + """Get current pool status.""" + if not pool_controller: + return jsonify({"error": "Pool controller not initialized"}), 500 + return jsonify(pool_controller.get_pool_status()) + + +@app.route("/api/pool/events") +def pool_events(): + """Server-Sent Events stream for real-time pool updates.""" + + def generate(): + while True: + if pool_controller: + status = pool_controller.get_pool_status() + yield f"data: {json.dumps(status)}\n\n" + time.sleep(2) + + return app.response_class(generate(), mimetype="text/event-stream") + + +@app.route("/api/conversation/start", methods=["POST"]) +def start_conversation(): + """Start a new conversation with a sandbox from the pool.""" + if not pool_controller: + return jsonify({"error": "Pool controller not initialized"}), 500 + + data = request.json or {} + message = data.get("message", "Hello! I'm ready to help.") + + # Get a ready sandbox + sandbox = pool_controller.get_ready_sandbox() + if not sandbox: + return ( + jsonify( + { + "error": "No ready sandboxes available", + "ready_count": pool_controller.ready_queue.qsize(), + } + ), + 503, + ) + + # Attach conversation + try: + conv_id = pool_controller.attach_conversation(sandbox, message) + conv_url = f"{pool_controller.base_url}/conversations/{conv_id}" + return jsonify( + { + "conversation_id": conv_id, + "conversation_url": conv_url, + "sandbox_id": sandbox.id, + "message": "Conversation started with pre-warmed sandbox!", + } + ) + except Exception as e: + logger.error(f"Failed to attach conversation: {e}", exc_info=True) + return jsonify({"error": str(e)}), 500 + + +def parse_args(): + p = argparse.ArgumentParser(description="Warm Sandbox Pool Controller") + p.add_argument( + "--api-key", + default=os.environ.get("OH_API_KEY"), + help="OpenHands Cloud API key (env: OH_API_KEY)", + ) + p.add_argument( + "--base-url", + default=os.environ.get("OH_API_BASE", "https://app.all-hands.dev"), + help="OpenHands base URL (env: OH_API_BASE)", + ) + p.add_argument( + "--pool-size", + type=int, + default=int(os.environ.get("POOL_SIZE", "3")), + help="Target pool size (env: POOL_SIZE, default: 3)", + ) + p.add_argument( + "--threshold", + type=int, + default=int(os.environ.get("POOL_THRESHOLD", "2")), + help="Refill when pool drops below this (env: POOL_THRESHOLD, default: 2)", + ) + p.add_argument( + "--sandbox-spec-id", + default=os.environ.get("SANDBOX_SPEC_ID"), + help="Optional sandbox spec ID (env: SANDBOX_SPEC_ID)", + ) + p.add_argument( + "--port", + type=int, + default=int(os.environ.get("PORT", "5000")), + help="Web server port (env: PORT, default: 5000)", + ) + return p.parse_args() + + +def main(): + args = parse_args() + + if not args.api_key: + sys.exit("Error: OH_API_KEY is required. Set via --api-key or environment variable.") + + # Initialize pool controller + global pool_controller + pool_controller = PoolController( + api_key=args.api_key, + base_url=args.base_url, + pool_size=args.pool_size, + threshold=args.threshold, + sandbox_spec_id=args.sandbox_spec_id, + ) + pool_controller.start() + + # Start Flask app + logger.info(f"Starting web server on port {args.port}") + logger.info(f"Open http://localhost:{args.port} in your browser") + app.run(host="0.0.0.0", port=args.port, debug=False, threaded=True) + + +if __name__ == "__main__": + main() diff --git a/warm-sandbox-pool/pyproject.toml b/warm-sandbox-pool/pyproject.toml new file mode 100644 index 0000000..efbe79e --- /dev/null +++ b/warm-sandbox-pool/pyproject.toml @@ -0,0 +1,18 @@ +[project] +name = "warm-sandbox-pool" +version = "0.1.0" +description = "Warm Sandbox Pool demonstration for OpenHands" +readme = "README.md" +requires-python = ">=3.10" +dependencies = [ + "flask>=3.0.0", + "flask-cors>=4.0.0", + "requests>=2.31.0", +] + +[project.scripts] +pool-controller = "pool_controller:main" + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" diff --git a/warm-sandbox-pool/requirements.txt b/warm-sandbox-pool/requirements.txt new file mode 100644 index 0000000..d3ffc4b --- /dev/null +++ b/warm-sandbox-pool/requirements.txt @@ -0,0 +1,3 @@ +flask>=3.0.0 +flask-cors>=4.0.0 +requests>=2.31.0 diff --git a/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh b/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh new file mode 100644 index 0000000..443c7b4 --- /dev/null +++ b/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh @@ -0,0 +1,139 @@ +#!/bin/bash +# Sandbox initialization script for warm pool +# This demonstrates the preparation steps needed to make a sandbox ready for use +# In a production deployment, you would replace this with your actual application setup + +set -e # Exit on error + +echo "πŸš€ Starting sandbox initialization..." +echo "πŸ“¦ Sandbox ID: ${SANDBOX_ID:-unknown}" + +# 1. Install Ruby (using system package manager for speed in this demo) +echo "πŸ“₯ Installing Ruby..." +apt-get update -qq +apt-get install -y -qq ruby-full ruby-dev build-essential > /dev/null 2>&1 +ruby --version + +# 2. Install Sinatra gem +echo "πŸ’Ž Installing Sinatra gem..." +gem install sinatra --no-document > /dev/null 2>&1 + +# 3. Create service directory +echo "πŸ“ Setting up service directory..." +mkdir -p /workspace/services +cd /workspace/services + +# 4. Copy the quote service (in production, this would be your actual app) +echo "πŸ“ Deploying quote service..." +cat > /workspace/services/quote_service.rb << 'RUBY_SERVICE_EOF' +#!/usr/bin/env ruby +# frozen_string_literal: true + +require 'sinatra' +require 'json' + +set :port, 4567 +set :bind, '0.0.0.0' + +QUOTES = [ + { author: 'Alan Kay', text: 'The best way to predict the future is to invent it.' }, + { author: 'Donald Knuth', text: 'Premature optimization is the root of all evil.' }, + { author: 'Rob Pike', text: 'Simplicity is complicated.' }, + { author: 'Ken Thompson', text: 'When in doubt, use brute force.' }, + { author: 'Yukihiro Matsumoto', text: 'Ruby is designed to make programmers happy.' }, + { author: 'David Heinemeier Hansson', text: 'Convention over configuration.' }, + { author: 'Sandi Metz', text: 'Duplication is far cheaper than the wrong abstraction.' }, + { author: 'Martin Fowler', text: 'Any fool can write code that a computer can understand. Good programmers write code that humans can understand.' } +].freeze + +get '/health' do + content_type :json + { status: 'ok', service: 'quote_service', version: '1.0.0' }.to_json +end + +get '/quote' do + content_type :json + quote = QUOTES.sample + { quote: quote[:text], author: quote[:author], timestamp: Time.now.iso8601 }.to_json +end + +get '/quotes' do + content_type :json + { quotes: QUOTES, count: QUOTES.size }.to_json +end + +get '/quotes/:author' do + content_type :json + author_param = params['author'].downcase + matches = QUOTES.select { |q| q[:author].downcase.include?(author_param) } + if matches.empty? + status 404 + { error: 'No quotes found for that author' }.to_json + else + { quotes: matches, count: matches.size }.to_json + end +end + +get '/' do + content_type :json + { + service: 'Quote of the Day API', + version: '1.0.0', + endpoints: { health: 'GET /health', random_quote: 'GET /quote', all_quotes: 'GET /quotes', by_author: 'GET /quotes/:author' }, + sandbox_id: ENV['SANDBOX_ID'] || 'unknown', + message: 'This service demonstrates a pre-initialized Ruby application in a warm sandbox. In production, this would be your actual application service.' + }.to_json +end +RUBY_SERVICE_EOF + +chmod +x /workspace/services/quote_service.rb + +# 5. Start the service in background +echo "🌐 Starting quote service on port 4567..." +cd /workspace/services +nohup ruby quote_service.rb > /tmp/quote_service.log 2>&1 & +SERVICE_PID=$! +echo "Service PID: $SERVICE_PID" + +# 6. Wait for service to be ready (with timeout) +echo "⏳ Waiting for service to respond..." +TIMEOUT=30 +ELAPSED=0 +while [ $ELAPSED -lt $TIMEOUT ]; do + if curl -sf http://localhost:4567/health > /dev/null 2>&1; then + echo "βœ… Service is healthy!" + curl -s http://localhost:4567/health | jq . || true + break + fi + sleep 1 + ELAPSED=$((ELAPSED + 1)) +done + +if [ $ELAPSED -ge $TIMEOUT ]; then + echo "❌ Service failed to start within ${TIMEOUT}s" + echo "Service log:" + cat /tmp/quote_service.log + exit 1 +fi + +# 7. Final verification - test the main endpoint +echo "πŸ§ͺ Testing service endpoint..." +QUOTE_RESPONSE=$(curl -s http://localhost:4567/quote) +echo "Sample quote: $QUOTE_RESPONSE" + +# 8. Create a marker file to indicate successful initialization +echo "βœ… Creating ready marker..." +echo "$(date -Iseconds)" > /workspace/.sandbox_ready +echo "SANDBOX_ID=${SANDBOX_ID:-unknown}" >> /workspace/.sandbox_ready +echo "SERVICE_PID=$SERVICE_PID" >> /workspace/.sandbox_ready +echo "RUBY_VERSION=$(ruby --version)" >> /workspace/.sandbox_ready + +echo "" +echo "πŸŽ‰ Sandbox initialization complete!" +echo "πŸ“Š Summary:" +echo " - Ruby: $(ruby --version | cut -d' ' -f2)" +echo " - Sinatra: installed" +echo " - Quote Service: running on port 4567 (PID $SERVICE_PID)" +echo " - Status: READY" +echo "" +echo "πŸ’‘ In production, replace this initialization with your application-specific setup" diff --git a/warm-sandbox-pool/sandbox_prep/quote_service.rb b/warm-sandbox-pool/sandbox_prep/quote_service.rb new file mode 100644 index 0000000..436deee --- /dev/null +++ b/warm-sandbox-pool/sandbox_prep/quote_service.rb @@ -0,0 +1,80 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Demo Ruby/Sinatra service for warm sandbox pool example +# A simple "quote of the day" API that runs in each initialized sandbox + +require 'sinatra' +require 'json' + +set :port, 4567 +set :bind, '0.0.0.0' + +QUOTES = [ + { author: 'Alan Kay', text: 'The best way to predict the future is to invent it.' }, + { author: 'Donald Knuth', text: 'Premature optimization is the root of all evil.' }, + { author: 'Rob Pike', text: 'Simplicity is complicated.' }, + { author: 'Ken Thompson', text: 'When in doubt, use brute force.' }, + { author: 'Yukihiro Matsumoto', text: 'Ruby is designed to make programmers happy.' }, + { author: 'David Heinemeier Hansson', text: 'Convention over configuration.' }, + { author: 'Sandi Metz', text: 'Duplication is far cheaper than the wrong abstraction.' }, + { author: 'Martin Fowler', text: 'Any fool can write code that a computer can understand. Good programmers write code that humans can understand.' } +].freeze + +# Health check endpoint +get '/health' do + content_type :json + { status: 'ok', service: 'quote_service', version: '1.0.0' }.to_json +end + +# Get a random quote +get '/quote' do + content_type :json + quote = QUOTES.sample + { + quote: quote[:text], + author: quote[:author], + timestamp: Time.now.iso8601 + }.to_json +end + +# Get all quotes +get '/quotes' do + content_type :json + { quotes: QUOTES, count: QUOTES.size }.to_json +end + +# Get quote by author (case-insensitive partial match) +get '/quotes/:author' do + content_type :json + author_param = params['author'].downcase + matches = QUOTES.select { |q| q[:author].downcase.include?(author_param) } + + if matches.empty? + status 404 + { error: 'No quotes found for that author' }.to_json + else + { quotes: matches, count: matches.size }.to_json + end +end + +# Info endpoint +get '/' do + content_type :json + { + service: 'Quote of the Day API', + version: '1.0.0', + endpoints: { + health: 'GET /health', + random_quote: 'GET /quote', + all_quotes: 'GET /quotes', + by_author: 'GET /quotes/:author' + }, + sandbox_id: ENV['SANDBOX_ID'] || 'unknown' + }.to_json +end + +# Startup message +puts "πŸš€ Quote Service starting on port #{settings.port}" +puts "πŸ“š Loaded #{QUOTES.size} quotes" +puts "🎯 Sandbox ID: #{ENV['SANDBOX_ID'] || 'not set'}" diff --git a/warm-sandbox-pool/static/app.js b/warm-sandbox-pool/static/app.js new file mode 100644 index 0000000..34a6948 --- /dev/null +++ b/warm-sandbox-pool/static/app.js @@ -0,0 +1,253 @@ +// Warm Sandbox Pool - Frontend JavaScript + +class PoolUI { + constructor() { + this.poolReady = false; + this.eventSource = null; + this.initialize(); + } + + initialize() { + this.setupEventStream(); + this.setupConversationForm(); + } + + setupEventStream() { + // Connect to Server-Sent Events for real-time pool updates + this.eventSource = new EventSource('/api/pool/events'); + + this.eventSource.onmessage = (event) => { + const data = JSON.parse(event.data); + this.updateUI(data); + }; + + this.eventSource.onerror = (error) => { + console.error('EventSource error:', error); + // Try to reconnect after 5 seconds + setTimeout(() => { + if (this.eventSource.readyState === EventSource.CLOSED) { + this.setupEventStream(); + } + }, 5000); + }; + } + + updateUI(poolStatus) { + // Update pool stats + document.getElementById('pool-size').textContent = poolStatus.pool_size; + document.getElementById('ready-count').textContent = poolStatus.ready_count; + document.getElementById('threshold').textContent = poolStatus.threshold; + + // Update sandbox list + this.renderSandboxes(poolStatus.sandboxes); + + // Check if pool is ready + const wasReady = this.poolReady; + this.poolReady = poolStatus.ready_count > 0; + + // Toggle conversation interface visibility + if (this.poolReady && !wasReady) { + this.showConversationInterface(); + } else if (!this.poolReady && wasReady) { + this.hideConversationInterface(); + } + } + + renderSandboxes(sandboxes) { + const container = document.getElementById('sandbox-list'); + + if (sandboxes.length === 0) { + container.innerHTML = '

No sandboxes yet...

'; + return; + } + + container.innerHTML = sandboxes.map(sb => this.renderSandboxCard(sb)).join(''); + } + + renderSandboxCard(sandbox) { + const statusClass = `status-${sandbox.state.toLowerCase()}`; + const statusEmoji = this.getStatusEmoji(sandbox.state); + + const duration = this.calculateDuration(sandbox.created_at, sandbox.ready_at); + const logs = sandbox.init_log && sandbox.init_log.length > 0 + ? this.renderLogs(sandbox.init_log) + : ''; + + const conversationLink = sandbox.conversation_id + ? ` + πŸ”— View Conversation + ` + : ''; + + const errorMessage = sandbox.error_message + ? `
❌ ${this.escapeHtml(sandbox.error_message)}
` + : ''; + + return ` +
+
+ ${statusEmoji} ${sandbox.state} + ${this.shortId(sandbox.id)} +
+ ${duration ? `
⏱️ ${duration}
` : ''} + ${logs} + ${conversationLink} + ${errorMessage} +
+ `; + } + + getStatusEmoji(state) { + const emojis = { + 'STARTING': 'πŸ”΄', + 'PREPARING': '🟑', + 'READY': '🟒', + 'ALLOCATED': '🟦', + 'FAILED': '⚠️' + }; + return emojis[state] || 'βšͺ'; + } + + renderLogs(logs) { + const recentLogs = logs.slice(-5); // Last 5 lines + return ` +
+ ${recentLogs.map(line => + `
${this.escapeHtml(line)}
` + ).join('')} +
+ `; + } + + calculateDuration(createdAt, readyAt) { + if (!readyAt) return null; + + const created = new Date(createdAt); + const ready = new Date(readyAt); + const seconds = Math.floor((ready - created) / 1000); + + return `Ready in ${seconds}s`; + } + + shortId(id) { + return id.length > 12 ? `${id.substring(0, 8)}...${id.substring(id.length - 4)}` : id; + } + + escapeHtml(text) { + const div = document.createElement('div'); + div.textContent = text; + return div.innerHTML; + } + + showConversationInterface() { + document.getElementById('waiting-state').style.display = 'none'; + document.getElementById('ready-state').style.display = 'block'; + } + + hideConversationInterface() { + document.getElementById('waiting-state').style.display = 'block'; + document.getElementById('ready-state').style.display = 'none'; + } + + setupConversationForm() { + const form = document.getElementById('message-input'); + const button = document.getElementById('start-btn'); + const resultArea = document.getElementById('result-area'); + + // Set default message + form.value = "Check if the quote service is running on localhost:4567 and fetch me a random quote. Show me the result."; + + button.addEventListener('click', async () => { + const message = form.value.trim(); + + if (!message) { + alert('Please enter a message'); + return; + } + + if (!this.poolReady) { + alert('Pool is not ready yet. Please wait for at least one sandbox to be ready.'); + return; + } + + // Disable button during request + button.disabled = true; + button.textContent = '⏳ Starting...'; + + try { + const response = await fetch('/api/conversation/start', { + method: 'POST', + headers: { + 'Content-Type': 'application/json' + }, + body: JSON.stringify({ message }) + }); + + const data = await response.json(); + + if (!response.ok) { + throw new Error(data.error || 'Failed to start conversation'); + } + + // Show success result + this.showResult(data, false); + + // Clear input + form.value = ''; + + } catch (error) { + console.error('Error starting conversation:', error); + this.showResult({ error: error.message }, true); + } finally { + button.disabled = false; + button.textContent = 'πŸš€ Start Conversation'; + } + }); + } + + showResult(data, isError) { + const resultArea = document.getElementById('result-area'); + + if (isError) { + resultArea.className = 'result-area error'; + resultArea.innerHTML = ` +

❌ Error

+

${this.escapeHtml(data.error)}

+ `; + } else { + resultArea.className = 'result-area'; + resultArea.innerHTML = ` +

βœ… ${this.escapeHtml(data.message)}

+

Sandbox ID: ${this.escapeHtml(data.sandbox_id)}

+

Conversation ID: ${this.escapeHtml(data.conversation_id)}

+ + πŸ”— Open Conversation in OpenHands + +

+ The pool will automatically provision a new sandbox to refill the allocated slot. +

+ `; + } + + resultArea.style.display = 'block'; + + // Scroll to result + resultArea.scrollIntoView({ behavior: 'smooth', block: 'nearest' }); + } + + cleanup() { + if (this.eventSource) { + this.eventSource.close(); + } + } +} + +// Initialize when DOM is ready +document.addEventListener('DOMContentLoaded', () => { + const poolUI = new PoolUI(); + + // Cleanup on page unload + window.addEventListener('beforeunload', () => { + poolUI.cleanup(); + }); +}); diff --git a/warm-sandbox-pool/static/styles.css b/warm-sandbox-pool/static/styles.css new file mode 100644 index 0000000..79914f0 --- /dev/null +++ b/warm-sandbox-pool/static/styles.css @@ -0,0 +1,458 @@ +/* Warm Sandbox Pool - UI Styles */ + +:root { + --primary-color: #2563eb; + --success-color: #10b981; + --warning-color: #f59e0b; + --danger-color: #ef4444; + --info-color: #3b82f6; + --bg-light: #f9fafb; + --bg-white: #ffffff; + --text-primary: #111827; + --text-secondary: #6b7280; + --border-color: #e5e7eb; + --shadow: 0 1px 3px 0 rgba(0, 0, 0, 0.1), 0 1px 2px 0 rgba(0, 0, 0, 0.06); + --shadow-lg: 0 10px 15px -3px rgba(0, 0, 0, 0.1), 0 4px 6px -2px rgba(0, 0, 0, 0.05); +} + +* { + margin: 0; + padding: 0; + box-sizing: border-box; +} + +body { + font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', sans-serif; + background-color: var(--bg-light); + color: var(--text-primary); + line-height: 1.6; +} + +.container { + max-width: 1400px; + margin: 0 auto; + padding: 20px; +} + +header { + text-align: center; + margin-bottom: 30px; + padding: 20px; + background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); + color: white; + border-radius: 12px; + box-shadow: var(--shadow-lg); +} + +header h1 { + font-size: 2.5rem; + margin-bottom: 8px; +} + +header .subtitle { + font-size: 1.1rem; + opacity: 0.95; +} + +.main-grid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 20px; + margin-bottom: 20px; +} + +@media (max-width: 1024px) { + .main-grid { + grid-template-columns: 1fr; + } +} + +.panel { + background: var(--bg-white); + border-radius: 12px; + padding: 24px; + box-shadow: var(--shadow); +} + +.panel h2 { + font-size: 1.5rem; + margin-bottom: 16px; + color: var(--text-primary); +} + +/* Pool Panel */ +.pool-stats { + display: flex; + gap: 16px; + margin-bottom: 20px; + padding: 16px; + background: var(--bg-light); + border-radius: 8px; +} + +.stat { + flex: 1; + text-align: center; +} + +.stat-label { + display: block; + font-size: 0.875rem; + color: var(--text-secondary); + margin-bottom: 4px; +} + +.stat-value { + display: block; + font-size: 1.5rem; + font-weight: bold; + color: var(--text-primary); +} + +.stat-value.ready-count { + color: var(--success-color); +} + +/* Sandbox List */ +.sandbox-list { + display: flex; + flex-direction: column; + gap: 12px; + margin-bottom: 20px; + max-height: 500px; + overflow-y: auto; +} + +.sandbox-card { + padding: 16px; + border: 2px solid var(--border-color); + border-radius: 8px; + background: var(--bg-white); + transition: all 0.2s; +} + +.sandbox-card:hover { + border-color: var(--primary-color); + box-shadow: var(--shadow); +} + +.sandbox-header { + display: flex; + align-items: center; + gap: 12px; + margin-bottom: 8px; +} + +.sandbox-id { + font-family: 'Courier New', monospace; + font-size: 0.875rem; + color: var(--text-secondary); + flex: 1; +} + +.status-badge { + padding: 4px 12px; + border-radius: 12px; + font-size: 0.75rem; + font-weight: 600; + text-transform: uppercase; +} + +.status-starting { + background: #fee2e2; + color: #991b1b; +} + +.status-preparing { + background: #fef3c7; + color: #92400e; +} + +.status-ready { + background: #d1fae5; + color: #065f46; +} + +.status-allocated { + background: #dbeafe; + color: #1e40af; +} + +.status-failed { + background: #fecaca; + color: #7f1d1d; +} + +.sandbox-details { + font-size: 0.875rem; + color: var(--text-secondary); + margin-top: 8px; +} + +.sandbox-log { + margin-top: 8px; + padding: 8px; + background: #f9fafb; + border-left: 3px solid var(--info-color); + border-radius: 4px; + font-family: 'Courier New', monospace; + font-size: 0.75rem; + max-height: 100px; + overflow-y: auto; +} + +.sandbox-log-line { + margin: 2px 0; +} + +.conversation-link { + display: inline-block; + margin-top: 8px; + color: var(--primary-color); + text-decoration: none; + font-weight: 500; +} + +.conversation-link:hover { + text-decoration: underline; +} + +/* Pool Legend */ +.pool-legend { + padding: 16px; + background: var(--bg-light); + border-radius: 8px; +} + +.pool-legend h3 { + font-size: 1rem; + margin-bottom: 12px; +} + +.legend-item { + display: flex; + align-items: center; + gap: 8px; + margin: 8px 0; + font-size: 0.875rem; +} + +/* Conversation Panel */ +.waiting-state { + text-align: center; + padding: 60px 20px; +} + +.spinner { + border: 4px solid var(--border-color); + border-top: 4px solid var(--primary-color); + border-radius: 50%; + width: 50px; + height: 50px; + animation: spin 1s linear infinite; + margin: 0 auto 20px; +} + +@keyframes spin { + 0% { transform: rotate(0deg); } + 100% { transform: rotate(360deg); } +} + +.waiting-state p { + font-size: 1.1rem; + color: var(--text-secondary); + margin: 8px 0; +} + +.waiting-subtext { + font-size: 0.875rem !important; + font-style: italic; +} + +.ready-state { + animation: fadeIn 0.5s; +} + +@keyframes fadeIn { + from { opacity: 0; transform: translateY(10px); } + to { opacity: 1; transform: translateY(0); } +} + +.info-box { + padding: 16px; + background: #dbeafe; + border-left: 4px solid var(--info-color); + border-radius: 8px; + margin-bottom: 20px; +} + +.info-box p { + margin: 4px 0; +} + +.conversation-form { + margin-bottom: 20px; +} + +.conversation-form label { + display: block; + font-weight: 500; + margin-bottom: 8px; +} + +.conversation-form textarea { + width: 100%; + padding: 12px; + border: 2px solid var(--border-color); + border-radius: 8px; + font-family: inherit; + font-size: 1rem; + resize: vertical; + transition: border-color 0.2s; +} + +.conversation-form textarea:focus { + outline: none; + border-color: var(--primary-color); +} + +.btn-primary { + width: 100%; + padding: 14px 24px; + background: var(--primary-color); + color: white; + border: none; + border-radius: 8px; + font-size: 1rem; + font-weight: 600; + cursor: pointer; + transition: all 0.2s; + margin-top: 12px; +} + +.btn-primary:hover { + background: #1d4ed8; + transform: translateY(-2px); + box-shadow: var(--shadow-lg); +} + +.btn-primary:active { + transform: translateY(0); +} + +.btn-primary:disabled { + background: var(--text-secondary); + cursor: not-allowed; + transform: none; +} + +.result-area { + margin-top: 20px; + padding: 16px; + background: #d1fae5; + border-left: 4px solid var(--success-color); + border-radius: 8px; + animation: fadeIn 0.5s; +} + +.result-area.error { + background: #fee2e2; + border-left-color: var(--danger-color); +} + +.result-area h3 { + margin-bottom: 12px; +} + +.result-area p { + margin: 8px 0; +} + +.result-area a { + display: inline-block; + margin-top: 12px; + padding: 10px 20px; + background: white; + color: var(--primary-color); + text-decoration: none; + border-radius: 6px; + font-weight: 600; + transition: all 0.2s; +} + +.result-area a:hover { + background: var(--primary-color); + color: white; + transform: translateY(-2px); +} + +/* Info Section */ +.info-section { + margin-top: 30px; + padding-top: 20px; + border-top: 2px solid var(--border-color); +} + +.info-section h3 { + font-size: 1.25rem; + margin: 20px 0 10px; +} + +.info-section h4 { + font-size: 1rem; + margin: 16px 0 8px; + color: var(--text-secondary); +} + +.info-section ul { + margin-left: 20px; + margin-bottom: 12px; +} + +.info-section li { + margin: 6px 0; + color: var(--text-secondary); +} + +.info-section code { + background: var(--bg-light); + padding: 2px 6px; + border-radius: 4px; + font-family: 'Courier New', monospace; + font-size: 0.875rem; +} + +.note { + margin-top: 16px; + padding: 12px; + background: #fef3c7; + border-left: 4px solid var(--warning-color); + border-radius: 8px; + font-size: 0.875rem; +} + +/* Footer */ +footer { + text-align: center; + padding: 20px; + color: var(--text-secondary); + font-size: 0.875rem; +} + +footer a { + color: var(--primary-color); + text-decoration: none; +} + +footer a:hover { + text-decoration: underline; +} + +/* Error State */ +.error-message { + color: var(--danger-color); + background: #fee2e2; + padding: 12px; + border-radius: 8px; + margin-top: 12px; +} diff --git a/warm-sandbox-pool/templates/index.html b/warm-sandbox-pool/templates/index.html new file mode 100644 index 0000000..86ad915 --- /dev/null +++ b/warm-sandbox-pool/templates/index.html @@ -0,0 +1,135 @@ + + + + + + Warm Sandbox Pool Demo + + + +
+
+

πŸ”₯ Warm Sandbox Pool

+

Pre-initialized OpenHands sandboxes with Ruby services ready to go

+
+ +
+ +
+

πŸ“¦ Sandbox Pool

+
+
+ Target Size: + - +
+
+ Ready: + 0 +
+
+ Threshold: + - +
+
+ +
+ +
+ +
+

Status Legend

+
+ πŸ”΄ + STARTING - Provisioning sandbox +
+
+ 🟑 + PREPARING - Installing Ruby & gems +
+
+ 🟒 + READY - Available for conversations +
+
+ 🟦 + ALLOCATED - In use by conversation +
+
+ ⚠️ + FAILED - Initialization error +
+
+
+ + +
+

πŸ’¬ Start a Conversation

+ +
+
+

Preparing sandbox pool...

+

This takes about 30-60 seconds on first start

+
+ + + +
+

About This Demo

+

This demonstrates the Warm Sandbox Pool technique for eliminating initialization delays when using OpenHands agents.

+ +

What's Pre-Installed:

+
    +
  • Ruby 3.x runtime
  • +
  • Sinatra gem
  • +
  • Quote service API running on port 4567
  • +
+ +

Try These Commands:

+
    +
  • "Check if the quote service is running on localhost:4567"
  • +
  • "Fetch a random quote from localhost:4567/quote and show it to me"
  • +
  • "List all quotes available in the service"
  • +
  • "Show me the Ruby version installed in this sandbox"
  • +
+ +
+ πŸ’‘ For Custom Applications: Replace the demo Sinatra service in sandbox_prep/init_ruby_service.sh with your actual application initialization commands. +
+
+
+
+
+ +
+

+ Warm Sandbox Pool - + Alternative to custom images for deploying Ruby-based services | + View on GitHub +

+
+ + + + diff --git a/warm-sandbox-pool/test_structure.py b/warm-sandbox-pool/test_structure.py new file mode 100644 index 0000000..dafacce --- /dev/null +++ b/warm-sandbox-pool/test_structure.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 +""" +Simple test to verify the warm-sandbox-pool structure is complete. +Run this before deploying to catch any missing files. +""" + +from pathlib import Path + +def check_structure(): + """Verify all expected files exist.""" + base = Path(__file__).parent + + required_files = [ + "README.md", + "QUICKSTART.md", + "pool_controller.py", + "requirements.txt", + "pyproject.toml", + "sandbox_prep/init_ruby_service.sh", + "sandbox_prep/quote_service.rb", + "static/app.js", + "static/styles.css", + "templates/index.html", + ] + + missing = [] + for file_path in required_files: + full_path = base / file_path + if not full_path.exists(): + missing.append(file_path) + + if missing: + print("❌ Missing files:") + for f in missing: + print(f" - {f}") + return False + + print("βœ… All required files present") + + # Check that scripts are executable + scripts = [ + base / "pool_controller.py", + base / "sandbox_prep" / "init_ruby_service.sh", + ] + + non_executable = [] + for script in scripts: + if not script.stat().st_mode & 0o111: # Check execute bit + non_executable.append(script.name) + + if non_executable: + print("⚠️ Scripts missing execute permission:") + for s in non_executable: + print(f" - {s}") + print(" Run: chmod +x pool_controller.py sandbox_prep/*.sh") + else: + print("βœ… All scripts are executable") + + # Check README has warm sandbox pool context + readme = (base / "README.md").read_text() + if "Warm Sandbox Pool" in readme: + print("βœ… README mentions Warm Sandbox Pool") + else: + print("⚠️ README doesn't mention Warm Sandbox Pool") + + # Check init script has proper shebang + init_script = (base / "sandbox_prep" / "init_ruby_service.sh").read_text() + if init_script.startswith("#!/bin/bash"): + print("βœ… Init script has proper shebang") + else: + print("⚠️ Init script missing #!/bin/bash shebang") + + print("\nπŸ“Š Structure check complete!") + return len(missing) == 0 + + +if __name__ == "__main__": + import sys + success = check_structure() + sys.exit(0 if success else 1) From 3cfa1907ed4e7e7d7818b1bd7d1b26050b801c1a Mon Sep 17 00:00:00 2001 From: openhands Date: Sat, 3 Oct 2026 16:50:23 +0000 Subject: [PATCH 2/9] Point references at OpenHands/enterprise-cookbook Co-authored-by: openhands --- warm-sandbox-pool/QUICKSTART.md | 4 ++-- warm-sandbox-pool/templates/index.html | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/warm-sandbox-pool/QUICKSTART.md b/warm-sandbox-pool/QUICKSTART.md index 1cc45c6..1418d62 100644 --- a/warm-sandbox-pool/QUICKSTART.md +++ b/warm-sandbox-pool/QUICKSTART.md @@ -12,8 +12,8 @@ Get the Warm Sandbox Pool demo running in 5 minutes. ```bash # Clone the repository -git clone https://github.com/jpshackelford/oh-examples.git -cd oh-examples/warm-sandbox-pool +git clone https://github.com/OpenHands/enterprise-cookbook.git +cd enterprise-cookbook/warm-sandbox-pool # Install Python dependencies (use uv or pip) pip install -r requirements.txt diff --git a/warm-sandbox-pool/templates/index.html b/warm-sandbox-pool/templates/index.html index 86ad915..d89aecc 100644 --- a/warm-sandbox-pool/templates/index.html +++ b/warm-sandbox-pool/templates/index.html @@ -126,7 +126,7 @@

Try These Commands:

Warm Sandbox Pool - Alternative to custom images for deploying Ruby-based services | - View on GitHub + View on GitHub

From 24f7efb8bbbeb7eaa10b780023ecf0c9042d2b94 Mon Sep 17 00:00:00 2001 From: openhands Date: Sat, 3 Oct 2026 17:04:02 +0000 Subject: [PATCH 3/9] Fix pre-commit failures (formatting, whitespace, line length) Co-authored-by: openhands --- warm-sandbox-pool/QUICKSTART.md | 10 +++---- warm-sandbox-pool/README.md | 2 +- warm-sandbox-pool/pool_controller.py | 30 +++++++++++++------ .../sandbox_prep/quote_service.rb | 2 +- warm-sandbox-pool/static/app.js | 20 ++++++------- warm-sandbox-pool/templates/index.html | 14 ++++----- warm-sandbox-pool/test_structure.py | 24 ++++++++------- 7 files changed, 58 insertions(+), 44 deletions(-) diff --git a/warm-sandbox-pool/QUICKSTART.md b/warm-sandbox-pool/QUICKSTART.md index 1418d62..f81b4ac 100644 --- a/warm-sandbox-pool/QUICKSTART.md +++ b/warm-sandbox-pool/QUICKSTART.md @@ -215,11 +215,11 @@ warm-sandbox-pool/ You've successfully run the demo when: -βœ… Pool shows 3 sandboxes in READY state -βœ… You can start a conversation and get instant allocation -βœ… The conversation link opens in OpenHands -βœ… The agent can interact with the pre-installed quote service -βœ… Pool automatically provisions a replacement sandbox +βœ… Pool shows 3 sandboxes in READY state\ +βœ… You can start a conversation and get instant allocation\ +βœ… The conversation link opens in OpenHands\ +βœ… The agent can interact with the pre-installed quote service\ +βœ… Pool automatically provisions a replacement sandbox **Total time from zero to working demo: ~5 minutes** diff --git a/warm-sandbox-pool/README.md b/warm-sandbox-pool/README.md index a638f5f..8dec1c9 100644 --- a/warm-sandbox-pool/README.md +++ b/warm-sandbox-pool/README.md @@ -6,7 +6,7 @@ Demonstrates maintaining a pool of pre-initialized "warm" sandboxes that late-bi ## Context: Alternative to Custom Images -When applications have components that run outside the agent control loop and must be available on the system where the agent is running, a custom image is not the only mechanism for packaging these dependencies. +When applications have components that run outside the agent control loop and must be available on the system where the agent is running, a custom image is not the only mechanism for packaging these dependencies. Even when using custom images in OpenHands Enterprise, some scenarios require additional tasks to be completed on the running sandbox to make it ready for use. **If these tasks take more than a few seconds, the Warm Sandbox Pool technique eliminates the apparent delay an end-user would see** by preparing a pool of pre-initialized sandboxes that late-bind to conversations when an end-user begins to interact with the agent. diff --git a/warm-sandbox-pool/pool_controller.py b/warm-sandbox-pool/pool_controller.py index 725d1aa..5283ac8 100644 --- a/warm-sandbox-pool/pool_controller.py +++ b/warm-sandbox-pool/pool_controller.py @@ -25,12 +25,12 @@ from datetime import datetime from enum import Enum from pathlib import Path -from typing import Any import requests from flask import Flask, jsonify, render_template, request from flask_cors import CORS + # Configure logging logging.basicConfig( level=logging.INFO, @@ -72,7 +72,9 @@ def to_dict(self) -> dict: "agent_url": self.agent_url, "created_at": self.created_at.isoformat(), "ready_at": self.ready_at.isoformat() if self.ready_at else None, - "allocated_at": self.allocated_at.isoformat() if self.allocated_at else None, + "allocated_at": self.allocated_at.isoformat() + if self.allocated_at + else None, "conversation_id": self.conversation_id, "error_message": self.error_message, "init_log": self.init_log[-10:], # Last 10 log lines @@ -163,9 +165,16 @@ def _provision_and_init_sandbox(self) -> None: try: # 1. Create sandbox via Cloud API logger.info("Creating new sandbox...") - params = {"sandbox_spec_id": self.sandbox_spec_id} if self.sandbox_spec_id else None + params = ( + {"sandbox_spec_id": self.sandbox_spec_id} + if self.sandbox_spec_id + else None + ) resp = requests.post( - f"{self.base_url}/api/v1/sandboxes", headers=self.headers, params=params, timeout=30 + f"{self.base_url}/api/v1/sandboxes", + headers=self.headers, + params=params, + timeout=30, ) resp.raise_for_status() sb_data = resp.json() @@ -272,7 +281,10 @@ def _run_init_script(self, sandbox: PooledSandbox) -> None: f"{sandbox.agent_url}/api/bash/execute_bash_command", headers=session_headers, json={ - "command": f"chmod +x /tmp/init.sh && SANDBOX_ID={sandbox.id} bash /tmp/init.sh", + "command": ( + "chmod +x /tmp/init.sh && " + f"SANDBOX_ID={sandbox.id} bash /tmp/init.sh" + ), "timeout": 300, }, timeout=320, @@ -322,9 +334,7 @@ def get_pool_status(self) -> dict: "timestamp": datetime.now().isoformat(), } - def attach_conversation( - self, sandbox: PooledSandbox, message: str - ) -> str: + def attach_conversation(self, sandbox: PooledSandbox, message: str) -> str: """Attach a new conversation to the given sandbox.""" payload = { "sandbox_id": sandbox.id, @@ -492,7 +502,9 @@ def main(): args = parse_args() if not args.api_key: - sys.exit("Error: OH_API_KEY is required. Set via --api-key or environment variable.") + sys.exit( + "Error: OH_API_KEY is required. Set via --api-key or environment variable." + ) # Initialize pool controller global pool_controller diff --git a/warm-sandbox-pool/sandbox_prep/quote_service.rb b/warm-sandbox-pool/sandbox_prep/quote_service.rb index 436deee..9f648eb 100644 --- a/warm-sandbox-pool/sandbox_prep/quote_service.rb +++ b/warm-sandbox-pool/sandbox_prep/quote_service.rb @@ -49,7 +49,7 @@ content_type :json author_param = params['author'].downcase matches = QUOTES.select { |q| q[:author].downcase.include?(author_param) } - + if matches.empty? status 404 { error: 'No quotes found for that author' }.to_json diff --git a/warm-sandbox-pool/static/app.js b/warm-sandbox-pool/static/app.js index 34a6948..3fecc93 100644 --- a/warm-sandbox-pool/static/app.js +++ b/warm-sandbox-pool/static/app.js @@ -15,7 +15,7 @@ class PoolUI { setupEventStream() { // Connect to Server-Sent Events for real-time pool updates this.eventSource = new EventSource('/api/pool/events'); - + this.eventSource.onmessage = (event) => { const data = JSON.parse(event.data); this.updateUI(data); @@ -55,7 +55,7 @@ class PoolUI { renderSandboxes(sandboxes) { const container = document.getElementById('sandbox-list'); - + if (sandboxes.length === 0) { container.innerHTML = '

No sandboxes yet...

'; return; @@ -67,9 +67,9 @@ class PoolUI { renderSandboxCard(sandbox) { const statusClass = `status-${sandbox.state.toLowerCase()}`; const statusEmoji = this.getStatusEmoji(sandbox.state); - + const duration = this.calculateDuration(sandbox.created_at, sandbox.ready_at); - const logs = sandbox.init_log && sandbox.init_log.length > 0 + const logs = sandbox.init_log && sandbox.init_log.length > 0 ? this.renderLogs(sandbox.init_log) : ''; @@ -112,7 +112,7 @@ class PoolUI { const recentLogs = logs.slice(-5); // Last 5 lines return `
- ${recentLogs.map(line => + ${recentLogs.map(line => `
${this.escapeHtml(line)}
` ).join('')}
@@ -121,11 +121,11 @@ class PoolUI { calculateDuration(createdAt, readyAt) { if (!readyAt) return null; - + const created = new Date(createdAt); const ready = new Date(readyAt); const seconds = Math.floor((ready - created) / 1000); - + return `Ready in ${seconds}s`; } @@ -159,7 +159,7 @@ class PoolUI { button.addEventListener('click', async () => { const message = form.value.trim(); - + if (!message) { alert('Please enter a message'); return; @@ -191,7 +191,7 @@ class PoolUI { // Show success result this.showResult(data, false); - + // Clear input form.value = ''; @@ -207,7 +207,7 @@ class PoolUI { showResult(data, isError) { const resultArea = document.getElementById('result-area'); - + if (isError) { resultArea.className = 'result-area error'; resultArea.innerHTML = ` diff --git a/warm-sandbox-pool/templates/index.html b/warm-sandbox-pool/templates/index.html index d89aecc..ec3465d 100644 --- a/warm-sandbox-pool/templates/index.html +++ b/warm-sandbox-pool/templates/index.html @@ -64,7 +64,7 @@

Status Legend

πŸ’¬ Start a Conversation

- +

Preparing sandbox pool...

@@ -79,12 +79,12 @@

πŸ’¬ Start a Conversation

- - + @@ -98,7 +98,7 @@

πŸ’¬ Start a Conversation

About This Demo

This demonstrates the Warm Sandbox Pool technique for eliminating initialization delays when using OpenHands agents.

- +

What's Pre-Installed:

  • Ruby 3.x runtime
  • @@ -124,7 +124,7 @@

    Try These Commands:

    - Warm Sandbox Pool - + Warm Sandbox Pool - Alternative to custom images for deploying Ruby-based services | View on GitHub

    diff --git a/warm-sandbox-pool/test_structure.py b/warm-sandbox-pool/test_structure.py index dafacce..3059308 100644 --- a/warm-sandbox-pool/test_structure.py +++ b/warm-sandbox-pool/test_structure.py @@ -6,13 +6,14 @@ from pathlib import Path + def check_structure(): """Verify all expected files exist.""" base = Path(__file__).parent - + required_files = [ "README.md", - "QUICKSTART.md", + "QUICKSTART.md", "pool_controller.py", "requirements.txt", "pyproject.toml", @@ -22,32 +23,32 @@ def check_structure(): "static/styles.css", "templates/index.html", ] - + missing = [] for file_path in required_files: full_path = base / file_path if not full_path.exists(): missing.append(file_path) - + if missing: print("❌ Missing files:") for f in missing: print(f" - {f}") return False - + print("βœ… All required files present") - + # Check that scripts are executable scripts = [ base / "pool_controller.py", base / "sandbox_prep" / "init_ruby_service.sh", ] - + non_executable = [] for script in scripts: if not script.stat().st_mode & 0o111: # Check execute bit non_executable.append(script.name) - + if non_executable: print("⚠️ Scripts missing execute permission:") for s in non_executable: @@ -55,26 +56,27 @@ def check_structure(): print(" Run: chmod +x pool_controller.py sandbox_prep/*.sh") else: print("βœ… All scripts are executable") - + # Check README has warm sandbox pool context readme = (base / "README.md").read_text() if "Warm Sandbox Pool" in readme: print("βœ… README mentions Warm Sandbox Pool") else: print("⚠️ README doesn't mention Warm Sandbox Pool") - + # Check init script has proper shebang init_script = (base / "sandbox_prep" / "init_ruby_service.sh").read_text() if init_script.startswith("#!/bin/bash"): print("βœ… Init script has proper shebang") else: print("⚠️ Init script missing #!/bin/bash shebang") - + print("\nπŸ“Š Structure check complete!") return len(missing) == 0 if __name__ == "__main__": import sys + success = check_structure() sys.exit(0 if success else 1) From 12e86755f71e0589941db5a23ebffad55e23d910 Mon Sep 17 00:00:00 2001 From: openhands Date: Wed, 7 Oct 2026 19:08:24 +0000 Subject: [PATCH 4/9] warm-sandbox-pool: make the recipe work and stop leaking sandboxes Found by running the recipe end to end against a live OpenHands instance. Broken as written: - Init script upload used a non-existent agent-server endpoint (/api/upload-files); use /api/file/upload?path=... - Init script ran apt-get/gem as the non-root sandbox user; use sudo - Current Sinatra needs rackup and puma to boot Resource safety: - Delete a sandbox as soon as provisioning fails - Stop refilling after MAX_FAILURES consecutive failures (was unbounded) - Delete unused pool sandboxes on shutdown (SIGTERM/Ctrl-C) - Bind the UI to 127.0.0.1 by default and drop the open CORS config Other fixes: - Upload quote_service.rb instead of a divergent copy embedded in the script - UI conversation link pointed at localhost instead of OpenHands - Add INIT_TIMEOUT, MAX_FAILURES, HOST settings that the docs described - README/QUICKSTART: correct Ruby install method, service access, timings, troubleshooting commands and parameter docs - test_structure.py also syntax-checks the init script and controller Co-authored-by: openhands --- warm-sandbox-pool/QUICKSTART.md | 36 ++-- warm-sandbox-pool/README.md | 104 ++++++++---- warm-sandbox-pool/pool_controller.py | 155 ++++++++++++++---- warm-sandbox-pool/pyproject.toml | 1 - warm-sandbox-pool/requirements.txt | 1 - .../sandbox_prep/init_ruby_service.sh | 91 +++------- .../sandbox_prep/quote_service.rb | 1 + warm-sandbox-pool/static/app.js | 7 +- warm-sandbox-pool/templates/index.html | 6 +- warm-sandbox-pool/test_structure.py | 22 ++- 10 files changed, 264 insertions(+), 160 deletions(-) mode change 100644 => 100755 warm-sandbox-pool/pool_controller.py mode change 100644 => 100755 warm-sandbox-pool/sandbox_prep/init_ruby_service.sh diff --git a/warm-sandbox-pool/QUICKSTART.md b/warm-sandbox-pool/QUICKSTART.md index f81b4ac..15f2e04 100644 --- a/warm-sandbox-pool/QUICKSTART.md +++ b/warm-sandbox-pool/QUICKSTART.md @@ -8,7 +8,7 @@ Get the Warm Sandbox Pool demo running in 5 minutes. - **Python 3.10+** - Check with `python --version` - **Git** - To clone this repository -## Installation (30 seconds) +## Installation (about a minute) ```bash # Clone the repository @@ -22,13 +22,18 @@ pip install -r requirements.txt uv pip install -e . ``` -## Run the Demo (10 seconds) +## Run the Demo + +> **This creates real sandboxes.** The controller starts `POOL_SIZE` (default 3) +> sandboxes right away and keeps refilling the pool. For a first try use +> `POOL_SIZE=1 POOL_THRESHOLD=1`. Press Ctrl-C to stop: unused pool sandboxes are +> deleted, sandboxes already attached to a conversation are left running. ```bash # Set your API key export OH_API_KEY=your_api_key_here -# Start the pool controller +# Start the pool controller (add OH_API_BASE=... for a non-default instance) python pool_controller.py # You should see: @@ -44,7 +49,7 @@ python pool_controller.py 2. **Watch the pool initialize**: - You'll see "Preparing pool..." message - - 3 sandboxes will start initializing (takes ~30-60 seconds each) + - 3 sandboxes will start initializing (roughly 10-30 seconds each) - Watch them progress: πŸ”΄ STARTING β†’ 🟑 PREPARING β†’ 🟒 READY 3. **Start a conversation**: @@ -57,7 +62,8 @@ python pool_controller.py 4. **Observe the magic**: - A ready sandbox is instantly pulled from the pool - - OpenHands conversation starts immediately (no wait!) + - The conversation attaches to it after a few seconds, with no sandbox boot or + Ruby install wait - Click the conversation link to see your agent working - Watch the pool automatically provision a new sandbox to refill @@ -69,8 +75,8 @@ User clicks "Start" β†’ Wait 30-60s for sandbox β†’ Agent starts ⏳ User sees delay With Warm Pool: -User clicks "Start" β†’ Instant sandbox β†’ Agent starts - ⚑ No apparent delay +User clicks "Start" β†’ Warm sandbox β†’ Agent starts + ⚑ No sandbox boot or install delay (Pool refills in background) ``` @@ -117,6 +123,7 @@ Customize via environment variables: export OH_API_KEY=your_key export OH_API_BASE=https://app.all-hands.dev # or your instance export PORT=5000 # Web UI port +export HOST=127.0.0.1 # UI has no auth; keep it local # Pool sizing export POOL_SIZE=3 # How many ready sandboxes to maintain @@ -124,20 +131,23 @@ export POOL_THRESHOLD=2 # Start refilling when ready count drops below this # Sandbox config (optional) export SANDBOX_SPEC_ID=your_spec_id # Use custom runtime image +export INIT_TIMEOUT=300 # Seconds the init script may run +export MAX_FAILURES=3 # Stop refilling after N failures in a row ``` ## Troubleshooting ### "No ready sandboxes available" error -- **Wait longer**: Initial pool fill takes 1-3 minutes +- **Wait longer**: Initial pool fill usually takes under a minute, more if the + platform is busy - **Check API key**: Make sure `OH_API_KEY` is valid - **View logs**: Look at terminal output for error messages - **Check web UI**: Look for FAILED sandboxes and error messages ### Sandboxes stuck in PREPARING - **Check init script**: May be timing out or failing -- **Increase timeout**: Set higher init timeout in code -- **Test manually**: Use `start-sandbox/` example to debug +- **Increase timeout**: Set `INIT_TIMEOUT` higher +- **Test manually**: See "Sandboxes Get Stuck in PREPARING" in `README.md` ### "OH_API_KEY is required" error ```bash @@ -195,9 +205,10 @@ warm-sandbox-pool/ β”œβ”€β”€ pool_controller.py πŸŽ›οΈ Main backend (Flask + pool logic) β”œβ”€β”€ requirements.txt πŸ“¦ Python dependencies β”œβ”€β”€ pyproject.toml πŸ“¦ UV-compatible project config +β”œβ”€β”€ test_structure.py βœ… Sanity check of the example's files β”œβ”€β”€ sandbox_prep/ β”‚ β”œβ”€β”€ init_ruby_service.sh πŸ”§ Demo: Ruby/Sinatra init -β”‚ └── quote_service.rb πŸ’Ž Demo Ruby service +β”‚ └── quote_service.rb πŸ’Ž Demo Ruby service (uploaded to each sandbox) β”œβ”€β”€ static/ β”‚ β”œβ”€β”€ app.js 🎨 Frontend JavaScript β”‚ └── styles.css 🎨 UI styling @@ -208,7 +219,6 @@ warm-sandbox-pool/ ## Getting Help - **Documentation**: Read `README.md` for full details -- **Example Code**: All files are heavily commented - **Related Examples**: Check `../start-sandbox/` and `../clone-and-attach/` ## Success Criteria @@ -221,6 +231,6 @@ You've successfully run the demo when: βœ… The agent can interact with the pre-installed quote service\ βœ… Pool automatically provisions a replacement sandbox -**Total time from zero to working demo: ~5 minutes** +**Total time from zero to working demo: a few minutes** Now you understand how warm sandbox pools work and can adapt this technique for your own application deployment! diff --git a/warm-sandbox-pool/README.md b/warm-sandbox-pool/README.md index 8dec1c9..aa7df8b 100644 --- a/warm-sandbox-pool/README.md +++ b/warm-sandbox-pool/README.md @@ -92,9 +92,18 @@ python pool_controller.py Then open http://localhost:5000 in your browser. +> **Heads up: this creates real sandboxes.** The controller immediately starts +> `POOL_SIZE` sandboxes (default 3) and keeps topping the pool up as you use them. +> Try it with `POOL_SIZE=1 POOL_THRESHOLD=1` first. Press Ctrl-C (or send SIGTERM) +> to stop: every sandbox still sitting in the pool is deleted. Sandboxes already +> attached to a conversation are left running, like any other conversation sandbox. +> If the controller is killed with SIGKILL it cannot clean up, so check your sandbox +> list afterwards. + ### What You'll See -1. **Initial State**: "Preparing pool..." message while 3 sandboxes initialize +1. **Initial State**: "Preparing pool..." message while the sandboxes initialize + (roughly 10-30 seconds each when the platform has capacity, longer otherwise) 2. **Pool Visualization**: Real-time status of each sandbox: - πŸ”΄ **STARTING**: OpenHands is provisioning the sandbox - 🟑 **PREPARING**: Installing Ruby, gems, starting service @@ -108,21 +117,26 @@ Then open http://localhost:5000 in your browser. Environment variables: - `OH_API_KEY`: OpenHands Cloud API key (required) -- `OH_API_BASE`: API base URL (default: `https://app.all-hands.dev`) +- `OH_API_BASE`: API base URL (default: `https://app.all-hands.dev`). Set this to + point at a different OpenHands instance. - `POOL_SIZE`: Target pool size (default: `3`) - `POOL_THRESHOLD`: Trigger refill when pool drops below this (default: `2`) +- `SANDBOX_SPEC_ID`: Optional sandbox spec (runtime image) to use for pool sandboxes +- `INIT_TIMEOUT`: Seconds the init script may run in a sandbox (default: `300`) +- `MAX_FAILURES`: Stop refilling after this many provisioning failures in a row + (default: `3`). A failed sandbox is deleted immediately, so a broken init script + cannot silently create sandboxes forever. +- `HOST`: Address the web UI binds to (default: `127.0.0.1`). The UI has no + authentication and can start conversations with your API key, so only widen this + on a trusted network. - `PORT`: Web server port (default: `5000`) ### Testing the Ruby Service -Once a sandbox is in READY state, the Sinatra service is running on port 4567. You can test it: - -```bash -# Get the sandbox ID from the web UI, then: -curl https://work-1-.prod-runtime.all-hands.dev:12000/quote -``` +Once a sandbox is in READY state, the Sinatra service is listening on port 4567 +**inside** the sandbox. That port is not exposed publicly, so reach it from the +sandbox itself. The easiest way is to start a conversation and ask the agent: -Or ask the agent to interact with it: ``` "Call the quote service running on localhost:4567 and show me today's quote" ``` @@ -131,12 +145,15 @@ Or ask the agent to interact with it: ### Sandbox Initialization Process -Each sandbox goes through these preparation steps (see `sandbox_prep/init_ruby_service.sh`): +The controller uploads `sandbox_prep/init_ruby_service.sh` and +`sandbox_prep/quote_service.rb` to the sandbox's agent-server (`POST /api/file/upload`) +and runs the script (`POST /api/bash/execute_bash_command`). The script does the +following (sandboxes run as a non-root user, so it uses `sudo` for installs): -1. **Install Ruby**: Uses rbenv to install Ruby 3.2 -2. **Install Sinatra**: Gem install sinatra -3. **Deploy Service**: Copies the quote service code -4. **Start Service**: Launches the Sinatra app in the background +1. **Install Ruby**: `apt-get install ruby-full` (Ruby 3.3 on current sandbox images) +2. **Install Sinatra**: `gem install sinatra rackup puma` +3. **Deploy Service**: Copies the uploaded `quote_service.rb` into `/workspace/services` +4. **Start Service**: Launches the Sinatra app in the background on port 4567 5. **Verify**: Confirms the service responds to health checks This simulates a realistic scenario where your agent needs specific tools/services pre-installed. @@ -149,7 +166,10 @@ The `PoolController` class handles: - **Monitoring**: Polls sandbox status until RUNNING - **Initialization**: Executes preparation scripts via agent-server API - **Queue Management**: Thread-safe queue of ready sandboxes -- **Auto-Refill**: Background thread maintains pool size +- **Auto-Refill**: Background thread maintains pool size, and stops after + `MAX_FAILURES` consecutive failures +- **Cleanup**: Deletes failed sandboxes immediately and all unused pool sandboxes + on shutdown - **Conversation Binding**: Attaches conversations to pre-warmed sandboxes ### Real-Time Updates @@ -161,11 +181,14 @@ The web UI uses Server-Sent Events (SSE) to stream pool state updates in real-ti ``` warm-sandbox-pool/ β”œβ”€β”€ README.md # This file +β”œβ”€β”€ QUICKSTART.md # Short run-it-now guide β”œβ”€β”€ pool_controller.py # Flask backend + pool manager β”œβ”€β”€ requirements.txt # Python dependencies +β”œβ”€β”€ pyproject.toml # Same dependencies, for uv +β”œβ”€β”€ test_structure.py # Sanity check of the example's files β”œβ”€β”€ sandbox_prep/ β”‚ β”œβ”€β”€ init_ruby_service.sh # Bash script to initialize each sandbox -β”‚ └── quote_service.rb # Ruby/Sinatra demo service +β”‚ └── quote_service.rb # Ruby/Sinatra demo service (uploaded to each sandbox) β”œβ”€β”€ static/ β”‚ β”œβ”€β”€ app.js # Frontend JavaScript β”‚ └── styles.css # UI styling @@ -180,10 +203,9 @@ warm-sandbox-pool/ Pre-install language runtimes (Ruby, Java, Go) that take time to set up: ```bash -# In init script -rbenv install 3.2.0 -rbenv global 3.2.0 -gem install rails bundler +# In init script (sandboxes run as a non-root user; use sudo for system installs) +sudo apt-get install -y ruby-full +sudo gem install rails bundler ``` ### 2. Service Dependencies @@ -217,11 +239,10 @@ Replace the demo Sinatra service with your actual application initialization: ```bash # In init script (replace sandbox_prep/init_ruby_service.sh contents) # Install Ruby (or use a runtime that already has it) -rbenv install 3.2.0 -rbenv global 3.2.0 +sudo apt-get install -y ruby-full # Install your application gems -gem install your_gem_name +sudo gem install your_gem_name # Initialize your application cd /workspace @@ -264,8 +285,8 @@ Edit `sandbox_prep/init_ruby_service.sh` to install additional tools: ```bash # Install Node.js -curl -fsSL https://deb.nodesource.com/setup_20.x | bash - -apt-get install -y nodejs +curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - +sudo apt-get install -y nodejs # Install Python packages pip install pandas numpy jupyter @@ -277,11 +298,9 @@ Replace `sandbox_prep/quote_service.rb` with your own Ruby application or gem. ### Adjust Pool Parameters -```python -# In pool_controller.py -POOL_SIZE = 5 # Maintain 5 ready sandboxes -POOL_THRESHOLD = 3 # Refill when below 3 -INIT_TIMEOUT = 600 # Allow 10 minutes for initialization +```bash +POOL_SIZE=5 POOL_THRESHOLD=3 INIT_TIMEOUT=600 python pool_controller.py +# Maintain 5 ready sandboxes, refill when below 3, allow 10 minutes for init ``` ### Add Health Checks @@ -300,22 +319,35 @@ done ### Pool Never Reaches Ready State -Check the initialization logs in the web UI. Common issues: +Check the controller's terminal output and the init log in the web UI. After +`MAX_FAILURES` failures in a row the controller stops creating sandboxes and the UI +says so. Common issues: - Ruby installation timeout (increase `INIT_TIMEOUT`) - Network issues downloading gems - Insufficient sandbox resources ### Sandboxes Get Stuck in PREPARING -Look at the agent-server command output. The init script may be failing. Test it manually: +Failed sandboxes are deleted right away, so to debug the init script keep a +sandbox alive and run it by hand. Create one with the `start-sandbox/` example (or +`POST /api/v1/sandboxes`), then read its `session_api_key` and `AGENT_SERVER` URL +from `GET /api/v1/sandboxes?id=`: ```bash -# Get a sandbox URL from the UI, then: -curl -X POST https:///api/bash/execute_bash_command \ - -H "X-Session-API-Key: " \ - -d '{"command": "bash /workspace/init_ruby_service.sh", "timeout": 300}' +AGENT=https:// +KEY= + +curl -X POST "$AGENT/api/file/upload?path=/tmp/init.sh" \ + -H "X-Session-API-Key: $KEY" -F file=@sandbox_prep/init_ruby_service.sh +curl -X POST "$AGENT/api/file/upload?path=/tmp/quote_service.rb" \ + -H "X-Session-API-Key: $KEY" -F file=@sandbox_prep/quote_service.rb +curl -X POST "$AGENT/api/bash/execute_bash_command" \ + -H "X-Session-API-Key: $KEY" -H "Content-Type: application/json" \ + -d '{"command": "bash /tmp/init.sh", "timeout": 300}' ``` +Delete the sandbox when you are done (`DELETE /api/v1/sandboxes/?sandbox_id=`). + ### High Resource Usage Reduce `POOL_SIZE` or implement smarter pool management: diff --git a/warm-sandbox-pool/pool_controller.py b/warm-sandbox-pool/pool_controller.py old mode 100644 new mode 100755 index 5283ac8..de9f0e0 --- a/warm-sandbox-pool/pool_controller.py +++ b/warm-sandbox-pool/pool_controller.py @@ -11,6 +11,9 @@ python pool_controller.py Then open http://localhost:5000 in your browser. + +On exit (Ctrl-C / SIGTERM) every sandbox that is still in the pool is deleted. +Sandboxes already handed to a conversation are left alone. """ import argparse @@ -18,6 +21,7 @@ import logging import os import queue +import signal import sys import threading import time @@ -28,7 +32,6 @@ import requests from flask import Flask, jsonify, render_template, request -from flask_cors import CORS # Configure logging @@ -91,32 +94,42 @@ def __init__( pool_size: int = 3, threshold: int = 2, sandbox_spec_id: str | None = None, + init_timeout: int = 300, + max_failures: int = 3, ): self.api_key = api_key self.base_url = base_url.rstrip("/") self.pool_size = pool_size self.threshold = threshold self.sandbox_spec_id = sandbox_spec_id + self.init_timeout = init_timeout + self.max_failures = max_failures self.headers = {"X-Session-API-Key": api_key} self.ready_queue: queue.Queue[PooledSandbox] = queue.Queue() self.all_sandboxes: dict[str, PooledSandbox] = {} self.lock = threading.RLock() self.running = True + self.consecutive_failures = 0 - # Load initialization script - self.init_script = self._load_init_script() + prep_dir = Path(__file__).parent / "sandbox_prep" + self.init_script = self._load_file(prep_dir / "init_ruby_service.sh") + self.service_source = self._load_file(prep_dir / "quote_service.rb") logger.info( f"Pool controller initialized: size={pool_size}, threshold={threshold}" ) - def _load_init_script(self) -> str: - """Load the sandbox initialization script.""" - script_path = Path(__file__).parent / "sandbox_prep" / "init_ruby_service.sh" - if not script_path.exists(): - raise FileNotFoundError(f"Init script not found: {script_path}") - return script_path.read_text() + @staticmethod + def _load_file(path: Path) -> str: + if not path.exists(): + raise FileNotFoundError(f"Sandbox prep file not found: {path}") + return path.read_text() + + @property + def halted(self) -> bool: + """True once provisioning has failed too many times in a row.""" + return self.consecutive_failures >= self.max_failures def start(self) -> None: """Start the pool management background thread.""" @@ -134,6 +147,8 @@ def _pool_manager_loop(self) -> None: # Maintenance loop while self.running: time.sleep(5) + if self.halted: + continue ready_count = self.ready_queue.qsize() if ready_count < self.threshold: needed = self.pool_size - self._total_initializing_count() - ready_count @@ -194,9 +209,16 @@ def _provision_and_init_sandbox(self) -> None: self._run_init_script(sandbox) # 4. Mark as ready and add to queue - sandbox.state = SandboxState.READY - sandbox.ready_at = datetime.now() - self.ready_queue.put(sandbox) + with self.lock: + accepted = self.running + if accepted: + sandbox.state = SandboxState.READY + sandbox.ready_at = datetime.now() + self.ready_queue.put(sandbox) + self.consecutive_failures = 0 + if not accepted: + self._delete_sandbox(sandbox_id) + return logger.info( f"Sandbox {sandbox_id}: READY " f"(took {(sandbox.ready_at - sandbox.created_at).seconds}s)" @@ -204,9 +226,31 @@ def _provision_and_init_sandbox(self) -> None: except Exception as e: logger.error(f"Failed to provision sandbox: {e}", exc_info=True) + with self.lock: + self.consecutive_failures += 1 if sandbox: sandbox.state = SandboxState.FAILED sandbox.error_message = str(e) + self._delete_sandbox(sandbox.id) + if self.halted: + logger.error( + f"{self.consecutive_failures} provisioning failures in a row: " + "refilling is halted. Fix the cause and restart the controller." + ) + + def _delete_sandbox(self, sandbox_id: str) -> None: + """Best-effort delete so failed or unused sandboxes don't leak.""" + try: + resp = requests.delete( + f"{self.base_url}/api/v1/sandboxes/{sandbox_id}", + headers=self.headers, + params={"sandbox_id": sandbox_id}, + timeout=30, + ) + resp.raise_for_status() + logger.info(f"Sandbox {sandbox_id}: deleted") + except Exception as e: + logger.warning(f"Could not delete sandbox {sandbox_id}: {e}") def _wait_until_running( self, sandbox: PooledSandbox, timeout: int = 180 @@ -264,30 +308,29 @@ def _run_init_script(self, sandbox: PooledSandbox) -> None: session_headers = {"X-Session-API-Key": sandbox.session_api_key} - # Upload the init script - sandbox.init_log.append("Uploading init script...") - upload_resp = requests.post( - f"{sandbox.agent_url}/api/upload-files", - headers=session_headers, - files={"file": ("init.sh", self.init_script, "text/plain")}, - data={"destination": "/tmp/init.sh"}, - timeout=30, - ) - upload_resp.raise_for_status() + sandbox.init_log.append("Uploading init script and service...") + for name, content in ( + ("init.sh", self.init_script), + ("quote_service.rb", self.service_source), + ): + upload_resp = requests.post( + f"{sandbox.agent_url}/api/file/upload", + headers=session_headers, + params={"path": f"/tmp/{name}"}, + files={"file": (name, content, "text/plain")}, + timeout=30, + ) + upload_resp.raise_for_status() - # Execute the script sandbox.init_log.append("Executing init script...") exec_resp = requests.post( f"{sandbox.agent_url}/api/bash/execute_bash_command", headers=session_headers, json={ - "command": ( - "chmod +x /tmp/init.sh && " - f"SANDBOX_ID={sandbox.id} bash /tmp/init.sh" - ), - "timeout": 300, + "command": f"SANDBOX_ID={sandbox.id} bash /tmp/init.sh", + "timeout": self.init_timeout, }, - timeout=320, + timeout=self.init_timeout + 20, ) exec_resp.raise_for_status() result = exec_resp.json() @@ -325,11 +368,17 @@ def get_pool_status(self) -> dict: [sb.to_dict() for sb in self.all_sandboxes.values()], key=lambda x: x["created_at"], ) + for sb in sandboxes_list: + if sb["conversation_id"]: + sb["conversation_url"] = ( + f"{self.base_url}/conversations/{sb['conversation_id']}" + ) return { "pool_size": self.pool_size, "threshold": self.threshold, "ready_count": self.ready_queue.qsize(), + "halted": self.halted, "sandboxes": sandboxes_list, "timestamp": datetime.now().isoformat(), } @@ -381,14 +430,23 @@ def attach_conversation(self, sandbox: PooledSandbox, message: str) -> str: return conv_id def shutdown(self) -> None: - """Graceful shutdown.""" - self.running = False - logger.info("Pool controller shutting down") + """Stop refilling and delete every sandbox not handed to a conversation.""" + with self.lock: + self.running = False + leftovers = [ + sb.id + for sb in self.all_sandboxes.values() + if sb.state not in (SandboxState.ALLOCATED, SandboxState.FAILED) + ] + logger.info( + f"Pool controller shutting down, deleting {len(leftovers)} sandbox(es)" + ) + for sandbox_id in leftovers: + self._delete_sandbox(sandbox_id) # Flask Application app = Flask(__name__) -CORS(app) pool_controller: PoolController | None = None @@ -495,6 +553,25 @@ def parse_args(): default=int(os.environ.get("PORT", "5000")), help="Web server port (env: PORT, default: 5000)", ) + p.add_argument( + "--host", + default=os.environ.get("HOST", "127.0.0.1"), + help="Web server bind address (env: HOST, default: 127.0.0.1). " + "The UI has no authentication; only widen this on a trusted network.", + ) + p.add_argument( + "--init-timeout", + type=int, + default=int(os.environ.get("INIT_TIMEOUT", "300")), + help="Seconds the init script may run (env: INIT_TIMEOUT, default: 300)", + ) + p.add_argument( + "--max-failures", + type=int, + default=int(os.environ.get("MAX_FAILURES", "3")), + help="Stop refilling after this many consecutive provisioning failures " + "(env: MAX_FAILURES, default: 3)", + ) return p.parse_args() @@ -514,13 +591,19 @@ def main(): pool_size=args.pool_size, threshold=args.threshold, sandbox_spec_id=args.sandbox_spec_id, + init_timeout=args.init_timeout, + max_failures=args.max_failures, ) pool_controller.start() - # Start Flask app - logger.info(f"Starting web server on port {args.port}") + signal.signal(signal.SIGTERM, lambda *_: sys.exit(0)) + + logger.info(f"Starting web server on {args.host}:{args.port}") logger.info(f"Open http://localhost:{args.port} in your browser") - app.run(host="0.0.0.0", port=args.port, debug=False, threaded=True) + try: + app.run(host=args.host, port=args.port, debug=False, threaded=True) + finally: + pool_controller.shutdown() if __name__ == "__main__": diff --git a/warm-sandbox-pool/pyproject.toml b/warm-sandbox-pool/pyproject.toml index efbe79e..a82676e 100644 --- a/warm-sandbox-pool/pyproject.toml +++ b/warm-sandbox-pool/pyproject.toml @@ -6,7 +6,6 @@ readme = "README.md" requires-python = ">=3.10" dependencies = [ "flask>=3.0.0", - "flask-cors>=4.0.0", "requests>=2.31.0", ] diff --git a/warm-sandbox-pool/requirements.txt b/warm-sandbox-pool/requirements.txt index d3ffc4b..eab2b35 100644 --- a/warm-sandbox-pool/requirements.txt +++ b/warm-sandbox-pool/requirements.txt @@ -1,3 +1,2 @@ flask>=3.0.0 -flask-cors>=4.0.0 requests>=2.31.0 diff --git a/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh b/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh old mode 100644 new mode 100755 index 443c7b4..4abd8c0 --- a/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh +++ b/warm-sandbox-pool/sandbox_prep/init_ruby_service.sh @@ -8,85 +8,38 @@ set -e # Exit on error echo "πŸš€ Starting sandbox initialization..." echo "πŸ“¦ Sandbox ID: ${SANDBOX_ID:-unknown}" -# 1. Install Ruby (using system package manager for speed in this demo) +# Sandboxes run as a non-root user with passwordless sudo +SUDO="" +if [ "$(id -u)" -ne 0 ]; then + SUDO="sudo -n" +fi + +# Run a quiet install step; print its log only if it fails +quiet() { + if ! "$@" > /tmp/init_step.log 2>&1; then + echo "❌ Failed: $*" + tail -n 20 /tmp/init_step.log + exit 1 + fi +} + +# 1. Install Ruby (Debian package, much faster than building with rbenv) echo "πŸ“₯ Installing Ruby..." -apt-get update -qq -apt-get install -y -qq ruby-full ruby-dev build-essential > /dev/null 2>&1 +quiet $SUDO apt-get update -qq +quiet $SUDO apt-get install -y -qq ruby-full ruby-dev build-essential ruby --version -# 2. Install Sinatra gem +# 2. Install Sinatra (4.x needs rackup and puma to boot) echo "πŸ’Ž Installing Sinatra gem..." -gem install sinatra --no-document > /dev/null 2>&1 +quiet $SUDO gem install sinatra rackup puma --no-document # 3. Create service directory echo "πŸ“ Setting up service directory..." mkdir -p /workspace/services -cd /workspace/services -# 4. Copy the quote service (in production, this would be your actual app) +# 4. Deploy the quote service (uploaded next to this script by the controller) echo "πŸ“ Deploying quote service..." -cat > /workspace/services/quote_service.rb << 'RUBY_SERVICE_EOF' -#!/usr/bin/env ruby -# frozen_string_literal: true - -require 'sinatra' -require 'json' - -set :port, 4567 -set :bind, '0.0.0.0' - -QUOTES = [ - { author: 'Alan Kay', text: 'The best way to predict the future is to invent it.' }, - { author: 'Donald Knuth', text: 'Premature optimization is the root of all evil.' }, - { author: 'Rob Pike', text: 'Simplicity is complicated.' }, - { author: 'Ken Thompson', text: 'When in doubt, use brute force.' }, - { author: 'Yukihiro Matsumoto', text: 'Ruby is designed to make programmers happy.' }, - { author: 'David Heinemeier Hansson', text: 'Convention over configuration.' }, - { author: 'Sandi Metz', text: 'Duplication is far cheaper than the wrong abstraction.' }, - { author: 'Martin Fowler', text: 'Any fool can write code that a computer can understand. Good programmers write code that humans can understand.' } -].freeze - -get '/health' do - content_type :json - { status: 'ok', service: 'quote_service', version: '1.0.0' }.to_json -end - -get '/quote' do - content_type :json - quote = QUOTES.sample - { quote: quote[:text], author: quote[:author], timestamp: Time.now.iso8601 }.to_json -end - -get '/quotes' do - content_type :json - { quotes: QUOTES, count: QUOTES.size }.to_json -end - -get '/quotes/:author' do - content_type :json - author_param = params['author'].downcase - matches = QUOTES.select { |q| q[:author].downcase.include?(author_param) } - if matches.empty? - status 404 - { error: 'No quotes found for that author' }.to_json - else - { quotes: matches, count: matches.size }.to_json - end -end - -get '/' do - content_type :json - { - service: 'Quote of the Day API', - version: '1.0.0', - endpoints: { health: 'GET /health', random_quote: 'GET /quote', all_quotes: 'GET /quotes', by_author: 'GET /quotes/:author' }, - sandbox_id: ENV['SANDBOX_ID'] || 'unknown', - message: 'This service demonstrates a pre-initialized Ruby application in a warm sandbox. In production, this would be your actual application service.' - }.to_json -end -RUBY_SERVICE_EOF - -chmod +x /workspace/services/quote_service.rb +cp /tmp/quote_service.rb /workspace/services/quote_service.rb # 5. Start the service in background echo "🌐 Starting quote service on port 4567..." diff --git a/warm-sandbox-pool/sandbox_prep/quote_service.rb b/warm-sandbox-pool/sandbox_prep/quote_service.rb index 9f648eb..6251bbc 100644 --- a/warm-sandbox-pool/sandbox_prep/quote_service.rb +++ b/warm-sandbox-pool/sandbox_prep/quote_service.rb @@ -6,6 +6,7 @@ require 'sinatra' require 'json' +require 'time' set :port, 4567 set :bind, '0.0.0.0' diff --git a/warm-sandbox-pool/static/app.js b/warm-sandbox-pool/static/app.js index 3fecc93..b7d59ee 100644 --- a/warm-sandbox-pool/static/app.js +++ b/warm-sandbox-pool/static/app.js @@ -41,6 +41,9 @@ class PoolUI { // Update sandbox list this.renderSandboxes(poolStatus.sandboxes); + document.getElementById('halted-notice').style.display = + poolStatus.halted ? 'block' : 'none'; + // Check if pool is ready const wasReady = this.poolReady; this.poolReady = poolStatus.ready_count > 0; @@ -73,8 +76,8 @@ class PoolUI { ? this.renderLogs(sandbox.init_log) : ''; - const conversationLink = sandbox.conversation_id - ? ` + const conversationLink = sandbox.conversation_url + ? ` πŸ”— View Conversation ` : ''; diff --git a/warm-sandbox-pool/templates/index.html b/warm-sandbox-pool/templates/index.html index ec3465d..3581a57 100644 --- a/warm-sandbox-pool/templates/index.html +++ b/warm-sandbox-pool/templates/index.html @@ -68,7 +68,11 @@

    πŸ’¬ Start a Conversation

    Preparing sandbox pool...

    -

    This takes about 30-60 seconds on first start

    +

    This usually takes under a minute on first start

    +
+
+
+ Claims: + 0 +
+
+ Avg warm-up: + - +
+
+ Avg attach: + - +
+
+

+ Warm-up is the sandbox boot plus init script that a cold + start would add to each conversation's wait. Attach is the + conversation start itself, which happens either way. +

+ +

In the pool

+

Claimed by conversations

+
+

Nothing claimed yet.

+
+

Status Legend

@@ -65,22 +91,28 @@

Status Legend

πŸ’¬ Start a Conversation

+ +

Preparing sandbox pool...

This usually takes under a minute on first start

-