Skip to content

Commit 8007a54

Browse files
committed
docs: isolate sandbox documentation
1 parent c3f531f commit 8007a54

1 file changed

Lines changed: 251 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,30 @@ rp flash dev main.py # live dev session: edit, re-run, logs stream back
6969
rp flash deploy # deploy production endpoints
7070
```
7171

72+
Functions keep their Python identity: `chat.remote(...)` runs in the cloud, `await chat.remote.aio(...)` is the async form, `chat.local(...)` runs in-process. See [`examples/apps`](examples/apps) for runnable examples and [docs.runpod.io](https://docs.runpod.io) for the full guide.
73+
74+
Queue `.remote()` calls request a synchronous result and poll the same job if the server's wait window expires. Fast results need no client-side polling. Dev sessions use a short sync window to keep worker logs responsive. Use `.spawn()` for an asynchronous job handle. A transport failure before receiving a job ID raises an error without resubmitting the work.
75+
76+
## Storage
77+
78+
`NetworkVolume(name_or_id, size=50, datacenter=None, create=True)` references
79+
datacenter-local storage. Apps resolve names and choose a datacenter compatible
80+
with every resource sharing the volume. `GlobalVolume(name_or_id, create=True)`
81+
references global storage without a datacenter constraint. Both inherit from
82+
the abstract `Volume` base, resolve by name or ID, and create missing storage
83+
when provisioning remote compute. Set `create=False` to require existing storage.
84+
Global-volume lookup and creation use GraphQL; network volumes use REST.
85+
86+
Declare attachments with `mounts={"/path": volume}`. Tasks support one network
87+
and one global volume at distinct, non-overlapping paths. Queue and API resources
88+
support one volume at `/runpod-volume`; global storage requires a GPU endpoint.
89+
90+
Inside worker code, `volume.path` returns the configured mount path. The runtime
91+
binds declared references and resolved IDs before importing user code. Access
92+
raises if the volume is unmounted or has multiple bindings. Evaluate `.path`
93+
inside remote functions, not during local module discovery. Calling `.local()`
94+
does not mount remote storage on the client machine.
95+
7296
## Sandboxes
7397

7498
Sandboxes are an access-controlled preview. An authorized account and a
@@ -222,6 +246,233 @@ You can also test your worker locally before deploying it to Runpod. This is use
222246
python my_worker.py --rp_serve_api
223247
```
224248

249+
### Worker Fitness Checks
250+
251+
Fitness checks allow you to validate your worker environment at startup before processing jobs. If any check fails, the worker exits immediately, allowing your orchestrator to restart it.
252+
253+
```python
254+
# my_worker.py
255+
256+
import runpod
257+
import torch
258+
259+
# Register fitness checks using the decorator
260+
@runpod.serverless.register_fitness_check
261+
def check_gpu_available():
262+
"""Verify GPU is available."""
263+
if not torch.cuda.is_available():
264+
raise RuntimeError("GPU not available")
265+
266+
@runpod.serverless.register_fitness_check
267+
def check_disk_space():
268+
"""Verify sufficient disk space."""
269+
import shutil
270+
stat = shutil.disk_usage("/")
271+
free_gb = stat.free / (1024**3)
272+
if free_gb < 10:
273+
raise RuntimeError(f"Insufficient disk space: {free_gb:.2f}GB free")
274+
275+
def handler(job):
276+
job_input = job["input"]
277+
# Your handler code here
278+
return {"output": "success"}
279+
280+
# Fitness checks run before handler initialization (production only)
281+
runpod.serverless.start({"handler": handler})
282+
```
283+
284+
**Key Features:**
285+
- Supports both synchronous and asynchronous check functions
286+
- Hardware checks run at the first Serverless `import runpod`; processes it launches inherit the result via `RUNPOD_EARLY_FITNESS_CHECKS_DONE` and skip the pass
287+
- Local tests and non-worker imports remain exempt; network readiness and custom checks run at worker start
288+
- Successful early checks are reused at worker start unless their configuration changes
289+
- Runs before handler initialization and job processing begins
290+
- Any check failure exits with code 1 (worker marked unhealthy)
291+
292+
See [Worker Fitness Checks](https://github.com/runpod/runpod-python/blob/main/docs/serverless/worker_fitness_checks.md) documentation for more examples and best practices.
293+
294+
### Network-Volume Warm Cache
295+
296+
When a network volume is attached, `VolumeCache` warms local directories (such as a model cache) across cold starts — hydrating them on startup and syncing new files back on exit — so a repeated multi-GB model download becomes a one-time cost per endpoint. It is stdlib-only and best-effort.
297+
298+
```python
299+
from runpod.serverless import VolumeCache
300+
301+
with VolumeCache(dirs=["/root/.cache/huggingface"]):
302+
model = load_model()
303+
```
304+
305+
See [Network-Volume Warm Cache](https://github.com/runpod/runpod-python/blob/main/docs/serverless/volume_cache.md) documentation for configuration and details.
306+
307+
## 📚 | REST API v2 Wrapper
308+
309+
Use the API wrapper to manage Runpod resources through REST API v2.
310+
311+
```python
312+
import runpod
313+
314+
runpod.api_key = "your_runpod_api_key_found_under_settings"
315+
```
316+
317+
### Endpoints
318+
319+
You can interact with Runpod endpoints via a `run` or `run_sync` method.
320+
321+
#### Basic Usage
322+
323+
```python
324+
endpoint = runpod.Endpoint("ENDPOINT_ID")
325+
326+
run_request = endpoint.run(
327+
{"your_model_input_key": "your_model_input_value"}
328+
)
329+
330+
# Check the status of the endpoint run request
331+
print(run_request.status())
332+
333+
# Get the output of the endpoint run request, blocking until the endpoint run is complete.
334+
print(run_request.output())
335+
```
336+
337+
```python
338+
endpoint = runpod.Endpoint("ENDPOINT_ID")
339+
340+
run_request = endpoint.run_sync(
341+
{"your_model_input_key": "your_model_input_value"}
342+
)
343+
344+
# Returns the job results if completed within 90 seconds, otherwise, returns the job status.
345+
print(run_request )
346+
```
347+
348+
#### API Key Management
349+
350+
The SDK supports multiple ways to set API keys:
351+
352+
**1. Global API Key** (Default)
353+
```python
354+
import runpod
355+
356+
# Set global API key
357+
runpod.api_key = "your_runpod_api_key"
358+
359+
# All endpoints will use this key by default
360+
endpoint = runpod.Endpoint("ENDPOINT_ID")
361+
result = endpoint.run_sync({"input": "data"})
362+
```
363+
364+
**2. Endpoint-Specific API Key**
365+
```python
366+
# Create endpoint with its own API key
367+
endpoint = runpod.Endpoint("ENDPOINT_ID", api_key="specific_api_key")
368+
369+
# This endpoint will always use the provided API key
370+
result = endpoint.run_sync({"input": "data"})
371+
```
372+
373+
#### API Key Precedence
374+
375+
The SDK uses this precedence order (highest to lowest):
376+
1. Endpoint instance API key (if provided to `Endpoint()`)
377+
2. Global API key (set via `runpod.api_key`)
378+
379+
```python
380+
import runpod
381+
382+
# Example showing precedence
383+
runpod.api_key = "GLOBAL_KEY"
384+
385+
# This endpoint uses GLOBAL_KEY
386+
endpoint1 = runpod.Endpoint("ENDPOINT_ID")
387+
388+
# This endpoint uses ENDPOINT_KEY (overrides global)
389+
endpoint2 = runpod.Endpoint("ENDPOINT_ID", api_key="ENDPOINT_KEY")
390+
391+
# All requests from endpoint2 will use ENDPOINT_KEY
392+
result = endpoint2.run_sync({"input": "data"})
393+
```
394+
395+
#### Thread-Safe Operations
396+
397+
Each `Endpoint` instance maintains its own API key, making concurrent operations safe:
398+
399+
```python
400+
import threading
401+
import runpod
402+
403+
def process_request(api_key, endpoint_id, input_data):
404+
# Each thread gets its own Endpoint instance
405+
endpoint = runpod.Endpoint(endpoint_id, api_key=api_key)
406+
return endpoint.run_sync(input_data)
407+
408+
# Safe concurrent usage with different API keys
409+
threads = []
410+
for customer in customers:
411+
t = threading.Thread(
412+
target=process_request,
413+
args=(customer["api_key"], customer["endpoint_id"], customer["input"])
414+
)
415+
threads.append(t)
416+
t.start()
417+
```
418+
419+
### GPU Cloud (Pods)
420+
421+
```python
422+
import runpod
423+
424+
runpod.api_key = "your_runpod_api_key_found_under_settings"
425+
426+
# get all my pods
427+
pods = runpod.get_pods()
428+
429+
# get a specific pod
430+
pod = runpod.get_pod(pods[0]["id"])
431+
432+
# create a pod with a gpu
433+
pod = runpod.create_pod("test", "runpod/stack", "NVIDIA GeForce RTX 4090")
434+
435+
# create a pod with a cpu
436+
pod = runpod.create_pod("test", "runpod/stack", instance_id="cpu3c-2-4")
437+
438+
# stop the pod
439+
runpod.stop_pod(pod["id"])
440+
441+
# resume the pod
442+
runpod.resume_pod(pod["id"])
443+
444+
# terminate the pod
445+
runpod.terminate_pod(pod["id"])
446+
```
447+
448+
### Template and placement options
449+
450+
- With `create_pod(template_id=...)`, omitting `docker_args` inherits the template's
451+
command. Pass `docker_args=""` to clear that inherited command.
452+
- Omitting `volume_mount_path` preserves an inherited GPU template volume's path.
453+
Setting it explicitly changes the path while retaining the inherited volume size.
454+
New persistent volumes and network volume mounts default to `/runpod-volume`.
455+
CPU pods do not inherit template persistent volumes; a CPU `volume_mount_path`
456+
requires `network_volume_id`.
457+
- For GPU pods, `min_memory_in_gb` and `min_vcpu_count` specify minimum host RAM
458+
and vCPUs **per GPU**, not GPU VRAM or totals for the pod.
459+
- CPU `instance_id` must use `<cpu-flavor>-<vcpu-count>-<memory>`, with positive
460+
integer vCPU and memory values, for example `cpu3c-4-8`.
461+
- REST v2 cannot require a public-IP-capable host. `support_public_ip=False` is
462+
the default and does not disable public networking. `True` raises `ValueError`
463+
rather than silently ignoring that requirement.
464+
- `create_template(volume_in_gb=0)` creates a template without a persistent volume.
465+
- `create_endpoint(locations="US-KS-2,EU-RO-1")` accepts comma-separated
466+
datacenter IDs. Country codes such as `US` and `RO` are not supported.
467+
Endpoint creation sends a single REST request without catalog lookups.
468+
- Endpoint `gpu_ids` accepts pool IDs and excluded GPU types, for example
469+
`gpu_ids="ADA_48_PRO,-NVIDIA L40"` selects that pool without NVIDIA L40 GPUs.
470+
- `create_endpoint` accepts only `QUEUE_DELAY` and `REQUEST_COUNT` scaling.
471+
`idle_timeout` applies to `QUEUE_DELAY` (default: 5 seconds); explicitly setting
472+
it with `REQUEST_COUNT` raises `ValueError`.
473+
- `resume_pod(pod_id)` keeps the existing GPU allocation. REST v2 does not support
474+
resizing on resume, so providing `gpu_count` raises `ValueError`.
475+
225476
## 📁 | Directory
226477

227478
```BASH

0 commit comments

Comments
 (0)