rLLM is an open-source framework for post-training language agents via reinforcement learning. With rLLM, you can easily build your custom agents and environments, train them with reinforcement learning, and deploy them for real-world workloads.
[2025/10/16] rLLM v0.2 is now officially released! We introduce AgentWorkflowEngine for training over arbitrary agentic programs. It also comes integrated with the official verl-0.5.0, featuring support for Megatron training. Check out this blog post for more.
[2025/07/01] We release DeepSWE-Preview, a 32B software engineering agent (SWE) trained with purely RL that achieves 59% on SWEBench-Verified with test-time scaling,(42.2% Pass@1), topping the SWEBench leaderboard for open-weight models.
[2025/04/08] We release DeepCoder-14B-Preview, a 14B coding model that achieves an impressive 60.6% Pass@1 accuracy on LiveCodeBench (+8% improvement), matching the performance of o3-mini-2025-01-031 (Low) and o1-2024-12-17.
[2025/02/10] We release DeepScaleR-1.5B-Preview, a 1.5B model that surpasses O1-Preview and achieves 43.1% Pass@1 on AIME. We achieve this by iteratively scaling Deepseek's GRPO algorithm from 8K→16K->24K context length for thinking.
# Clone the repository
git clone --recurse-submodules https://github.com/rllm-org/rllm.git
cd rllm
# Create a conda environment
conda create -n rllm python=3.10 -y
conda activate rllm
# Install verl
bash scripts/install_verl.sh
# Install rLLM
pip install -e .For a containerized setup, you can use Docker:
# Build the Docker image
docker build -t rllm .
# Create and start the container
docker create --runtime=nvidia --gpus all --net=host --shm-size="10g" --cap-add=SYS_ADMIN -v .:/workspace/rllm -v /tmp:/tmp --name rllm-container rllm sleep infinity
docker start rllm-container
# Enter the container
docker exec -it rllm-container bash- DeepScaleR: Surpassing O1-Preview with a 1.5B Model by Scaling RL
- DeepCoder: A Fully Open-Source 14B Coder at O3-mini Level
- DeepSWE: Training a Fully Open-sourced, State-of-the-Art Coding Agent by Scaling RL
- Tongyi DeepResearch: A New Era of Open-Source AI Researchers
- Terminal-Bench-RL: Training Long-Horizon Terminal Agents with Reinforcement Learning
- Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and Planning
- PettingLLMs: Using On-Policy Reinforcement Learning for Stronger Multi-Agent System
Our work is done as part of Berkeley Sky Computing Lab. The rLLM team is generously supported by grants from Laude Institute, AWS, Hyperbolic and Fireworks AI. We pay special thanks to Together AI for the research partnership and compute support.
@misc{rllm2025,
title={rLLM: A Framework for Post-Training Language Agents},
author={Sijun Tan and Michael Luo and Colin Cai and Tarun Venkat and Kyle Montgomery and Aaron Hao and Tianhao Wu and Arnav Balyan and Manan Roongta and Chenguang Wang and Li Erran Li and Raluca Ada Popa and Ion Stoica},
year={2025},
howpublished={\url{https://pretty-radio-b75.notion.site/rLLM-A-Framework-for-Post-Training-Language-Agents-21b81902c146819db63cd98a54ba5f31}},
note={Notion Blog}
year={2025}
}You may also cite our prior work DeepScaleR, DeepCoder, and DeepSWE.
