docs: rewrite README, update AGENTS.md, add 8 examples#87
Open
cagataycali wants to merge 5 commits intostrands-labs:mainfrom
Open
docs: rewrite README, update AGENTS.md, add 8 examples#87cagataycali wants to merge 5 commits intostrands-labs:mainfrom
cagataycali wants to merge 5 commits intostrands-labs:mainfrom
Conversation
- Bump requires-python from >=3.10 to >=3.12 - Move opencv-python-headless and Pillow to [sim] extra - Add [sim] extra: mujoco, robot_descriptions, opencv, Pillow - Add .python-version (3.12) - Add uv.lock for deterministic builds - Switch CI from hatch to uv - Add libosmesa6-dev to CI for MuJoCo headless rendering - Remove hatch env scripts (use uv run directly) - Remove --cov from pytest addopts (run coverage explicitly) - Update ruff target-version and mypy python_version to 3.12 - Trim pyproject.toml comments for readability Why Python 3.12+: match statements, type aliases, ExceptionGroup patterns used in upcoming simulation code. MuJoCo 3.x works best with 3.12+. No users have reported needing 3.10/3.11 support. Why move opencv/Pillow to [sim]: core strands-robots (hardware control only) shouldn't require heavy image processing deps.
…ssets Add the lightweight simulation framework abstractions: - simulation/models.py: Dataclasses (SimWorld, SimRobot, SimObject, SimCamera, TrajectoryStep, SimStatus) — pure data, no physics deps - simulation/base.py: SimulationBackend ABC defining the contract all backends (MuJoCo, Isaac, Newton) must implement - simulation/factory.py: create_simulation() + pluggable backend registration system - simulation/model_registry.py: URDF/MJCF path resolution via robot_descriptions package + search paths + custom registration - assets/: Asset manager for auto-downloading robot meshes from MuJoCo Menagerie on first use - tools/download_assets.py: @tool for asset downloads - tests/test_simulation_foundation.py: 22 tests covering dataclasses, ABC instantiation/contracts, factory registration, model registry No MuJoCo import required — all abstractions are lightweight.
Complete MuJoCo simulation backend composed of focused mixins:
Simulation(AgentTool)
├── PhysicsMixin # raycasting, jacobians, energy, forces,
│ # mass matrix, checkpoints, inverse dynamics
├── PolicyRunnerMixin # run_policy, eval_policy, replay_episode
├── RenderingMixin # RGB/depth offscreen rendering, observations
├── RecordingMixin # LeRobot dataset recording
└── RandomizationMixin # domain randomization (colors, lighting, physics)
Supporting modules:
- backend.py: lazy mujoco import + headless GL auto-config (EGL/OSMesa/GLFW)
- mjcf_builder.py: procedural MJCF XML generation from dataclasses
- scene_ops.py: XML round-trip for runtime object/camera injection
- simulation.py: orchestrator dispatching 35 actions via tool_spec.json
- dataset_recorder.py: LeRobot v3 format recorder (parquet + video)
Key design decisions:
- Simulation extends AgentTool directly: Agent(tools=[Simulation()]) works
- Lazy MuJoCo import via _ensure_mujoco() — only when first needed
- XML round-trip for scene modification (standard: dm_control, robosuite)
- Same Policy ABC for sim and real — zero code changes for transfer
Tests: 47 new tests (12 E2E + 35 physics unit tests)
All use self-contained inline XML robots (no external files needed).
Add unified Robot() factory function that auto-detects sim vs real:
Robot('so100') → auto-detect → MuJoCo sim (default)
Robot('so100', mode='sim') → Simulation AgentTool
Robot('so100', mode='real') → HardwareRobot AgentTool
Auto-detect priority:
1. STRANDS_ROBOT_MODE env var (explicit override)
2. USB probe for servo controllers (Feetech/Dynamixel)
3. Default to sim (safest — never accidentally send to hardware)
Also adds list_robots(mode='all'|'sim'|'real'|'both') for discovery.
Updates __init__.py with lazy imports:
- Robot (→ factory), list_robots
- Simulation, SimWorld, SimRobot, SimObject, SimCamera
- Auto-configures MuJoCo GL backend for headless environments
Tests: 22 new factory tests covering name resolution, aliases,
list_robots filtering, auto-detect mode, Robot() factory, imports.
- README.md: Complete rewrite with 5-line promise, mermaid architecture diagram, installation extras table, simulation quickstart, policy execution examples - AGENTS.md: Updated project structure to include simulation modules, test instructions, and conventions - examples/01_sim_quickstart.py: 5-line sim quickstart - examples/02_sim_agent.py: Agent + simulation integration - examples/03_sim_recording.py: Dataset recording to LeRobot format - examples/04_real_hardware.py: Real hardware robot setup - examples/05_real_groot_policy.py: GR00T policy on real hardware - examples/06_list_robots.py: Robot discovery and listing - examples/act_policy_simulation.py: ACT policy in MuJoCo sim - examples/physics_agent.py: Natural language physics introspection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Complete documentation rewrite and 8 new examples covering simulation, hardware, and policy execution.
What changed
README.mdAGENTS.mdexamples/01_sim_quickstart.pyexamples/02_sim_agent.pyexamples/03_sim_recording.pyexamples/04_real_hardware.pyexamples/05_real_groot_policy.pyexamples/06_list_robots.pyexamples/act_policy_simulation.pyexamples/physics_agent.pyNo code changes
Docs and examples only. All 357 tests still pass.
Part 6 of 6 — final PR in the MuJoCo simulation decomposition
Full PR chain: