Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 67 additions & 21 deletions src/agentcore_rl_toolkit/backends/slime/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,38 +149,84 @@ cp .wandb.env.example .wandb.env # optional; skip to disable wandb

### 3.4 Run training

train.sh defaults target **8 × H100** (NUM_GPUS=8, TP_SIZE=2,
ROLLOUT_GPUS_PER_ENGINE=2). For smaller clusters override via env
(e.g. `NUM_GPUS=1 TP_SIZE=1 ROLLOUT_GPUS_PER_ENGINE=1` for a single
GPU). Defaults also set `NUM_ROLLOUT=1` for smoke testing — bump to
`NUM_ROLLOUT=100` (slime's production value) for a real run.

`SLIME_DIR` /
`MEGATRON_DIR` need to point at the slime + Megatron-LM source trees
(inside the `slimerl/slime:latest` container these are `/root/slime`
and `/root/Megatron-LM`).
Two entry points, same job under the hood:

- **`SlimeRunner` (Python)** — recommended. Write a short `train.py`;
the class handles Ray/SGLang plumbing and slime CLI flags.
- **`train.sh` (bash)** — escape hatch. Use when you need to override
something the class doesn't surface, or to debug the raw slime CLI.

#### Python (`SlimeRunner`)

Defaults target **8 × H100** (`num_gpus=8`, `tp_size=2`,
`rollout_gpus_per_engine=2`). Override kwargs for other cluster sizes.
`.train(num_rollout=…)` defaults to 1 rollout for smoke testing — bump
to 100 for a real run.

`slime_dir` / `megatron_dir` default to the in-container paths
(`/root/slime` and `/root/Megatron-LM`); override for bare-metal
installs.

```python
# train.py — minimal 3B smoke test
from agentcore_rl_toolkit.backends.slime import SlimeRunner

SlimeRunner(
exp_id="gsm8k-3b-smoke",
agent_runtime_arn="arn:aws:bedrock-agentcore:...",
s3_bucket="your-bucket",
model_dir="/path/to/Qwen2.5-3B-Instruct",
data_path="/path/to/gsm8k_tiny.jsonl",
model_type="qwen2.5-3B",
).train(num_rollout=1)
```

```bash
cd /path/to/agentcore-rl-toolkit
python src/agentcore_rl_toolkit/backends/slime/examples/math_agent/train.py
```

32B on 8 GPUs:

```python
SlimeRunner(
exp_id="gsm8k-32b-run",
agent_runtime_arn="arn:aws:bedrock-agentcore:...",
s3_bucket="your-bucket",
model_dir="/path/to/Qwen2.5-32B-Instruct",
data_path="/path/to/gsm8k_tiny.jsonl",
model_type="qwen2.5-32B",
tp_size=8,
rollout_gpus_per_engine=8,
).train(num_rollout=5)
```

Any slime/Megatron-LM/SGLang CLI flag that isn't surfaced as a named
kwarg can be passed through `extra_flags`:

```python
SlimeRunner(..., extra_flags=["--num-epoch", "3"]).train(num_rollout=50)
```

# Qwen2.5-3B, 8 GPUs, 1 rollout (smoke test — train.sh defaults)
If you prefer a YAML config, `SlimeRunner.from_yaml("config.yaml").train()`
accepts the same keys.

#### Bash (`train.sh`) — escape hatch

`train.sh` takes the same knobs via env vars. It's kept as the low-level
reference for what the Python class replicates, and as a debugging path
for slime flag experiments.

```bash
# 3B smoke test
export SLIME_DIR=/root/slime \
MEGATRON_DIR=/root/Megatron-LM \
MODEL_DIR=/path/to/Qwen2.5-3B-Instruct \
DATA_PATH=/path/to/gsm8k_tiny.jsonl
bash src/agentcore_rl_toolkit/backends/slime/examples/math_agent/train.sh
```

For 32B on 8 GPUs:

```bash
export MODEL_DIR=/path/to/Qwen2.5-32B-Instruct \
MODEL_TYPE=qwen2.5-32B \
TP_SIZE=8 \
ROLLOUT_GPUS_PER_ENGINE=8 \
NUM_ROLLOUT=5
bash src/agentcore_rl_toolkit/backends/slime/examples/math_agent/train.sh
```
For 32B on 8 GPUs, add `MODEL_TYPE=qwen2.5-32B TP_SIZE=8 ROLLOUT_GPUS_PER_ENGINE=8 NUM_ROLLOUT=5`.

### 3.5 Run evaluation

Expand Down
12 changes: 12 additions & 0 deletions src/agentcore_rl_toolkit/backends/slime/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
"""Slime training backend for agentcore-rl-toolkit.

Primary entry point is :class:`SlimeRunner`. The ``integration/`` subpackage
and ``patches/`` module are implementation detail — users shouldn't import
them directly, but they are load-bearing (slime loads
``agentcore_rl_toolkit.backends.slime.integration.rollout.generate_rollout``
via ``--rollout-function-path`` at job-submit time).
"""

from .runner import SlimeRunner

__all__ = ["SlimeRunner"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
"""Train the strands math agent on GSM8K via slime — Python entry point.

Prerequisites (see ``SETUP.md`` for full instructions):
- Inside a working slime environment (``slimerl/slime:latest`` container or equivalent).
- Agent deployed to ACR; runtime ARN + S3 bucket handy.
- Model checkpoint and training JSONL downloaded locally.

Run:
python train.py
"""

from agentcore_rl_toolkit.backends.slime import SlimeRunner

if __name__ == "__main__":
SlimeRunner(
exp_id="gsm8k-3b-smoke",
agent_runtime_arn="arn:aws:bedrock-agentcore:<region>:<account>:runtime/<runtime-id>",
s3_bucket="your-bucket-name",
model_dir="/workspace/slime_workdir/models/Qwen2.5-3B-Instruct",
data_path="/workspace/slime_workdir/data/gsm8k_tiny.jsonl",
model_type="qwen2.5-3B",
).train(num_rollout=1)
Loading
Loading