GCULab

Overview

GCU Lab - Additions to IsaacLab Core functionality for Pack task.

Pack Task - Packing environment.

---

Key Features

PPO packing agent

Notes: Currently uses multiprocessing for placement calculation, recommended --num_envs 100, This will be adapted to use depth camera (#13)

Command:

# For training
python scripts/rsl_rl/train.py --task=Isaac-Pack-NoArm-v0  --num_envs 20
# For inference
python scripts/rsl_rl/play.py --task=Isaac-Pack-NoArm-v0  --num_envs 20

Heuristic-scoring based search packing agent

Packing agent based on "Stable bin packing of non-convex 3D objects with a robot manipulator" by Fan Wang and Kris Hauser. arXiv:1812.04093

Notes: Uses multiprocessing for placement calculation, recommended --num_envs 100 Defaults to DBLF heuristic

Command:

python scripts/bpp_agents/fanwang_bpp_agent.py --task=Isaac-Pack-NoArm-v0 --num_envs 100

Pack Task

Demo for Amazon Packing Task

Command:

python scripts/bpp_agents/test_placement_agent.py --task=Isaac-Pack-NoArm-v0 --num_envs 5

---

IK Reachability Analysis

Requires curobo (Installation Instructions).

Command:

python scripts/ik_reachability_agent.py --task=Isaac-Pack-UR5-v0 --num_envs 1

---

Installation

Step 1: Install Isaac Lab

Follow the installation guide. We recommend using the conda installation as it simplifies calling Python scripts from the terminal.

Step 2: Clone or Copy This Repository

Ensure this project/repository is separate from the Isaac Lab installation (i.e., outside the IsaacLab directory).

Step 3: Install the Library in Editable Mode

Using a Python interpreter with Isaac Lab installed, run:

# Use 'PATH_TO_isaaclab.sh|bat -p' instead of 'python' if Isaac Lab is not installed in Python venv or conda
python -m pip install -e source/tote_consolidation
python -m pip install -e source/gculab
python -m pip install -e source/gculab_assets
python -m pip install -e source/gculab_rl

---

Running Tasks with RL Agents

Running Tasks with Dummy Agents

These include dummy agents that output zero or random actions. They are useful for verifying environment configurations.

Zero-Action Agent

Command:

# Use 'FULL_PATH_TO_isaaclab.sh|bat -p' instead of 'python' if Isaac Lab is not installed in Python venv or conda
python scripts/zero_agent.py --task=<TASK_NAME>

---

Random-Action Agent

Command:

# Use 'FULL_PATH_TO_isaaclab.sh|bat -p' instead of 'python' if Isaac Lab is not installed in Python venv or conda
python scripts/random_agent.py --task=<TASK_NAME>

---

Test Placement Agent

Command:

# Use 'FULL_PATH_TO_isaaclab.sh|bat -p' instead of 'python' if Isaac Lab is not installed in Python venv or conda
python scripts/test_placement_agent.py --task=Isaac-Pack-NoArm-v0 --num_envs 5

---

Teleoperation and Imitation Learning

Command:

python scripts/teleop_se3_agent.py --task Isaac-Pack-UR5-Teleop-v0 --num_envs 1 --teleop_device mello

Teleop Device

Mello is a teleoperation device designed for intuitive robot control, similar in concept to Gello. It mimics the robot's joint structure, allowing users to control the robot by physically moving the device. Joint positions are sent directly to the robot, eliminating the need for inverse kinematics or physics-based computation. Mello is especially useful for imitation learning, where human demonstrations collected via teleoperation are used to train models. Because Mello closely matches the robot’s kinematics, it enables efficient and accurate data collection for learning from demonstration.

---

Code Formatting

We use a pre-commit template to automatically format your code.

Installation

Install pre-commit with:

pip install pre-commit

Running Pre-Commit

Run pre-commit for all files:

pre-commit run --all-files