SimDAgger

Failure-aware Recovery-DAgger for bimanual robot learning in RoboTwin

Train a policy, recognize when it is going wrong, rewind to a recoverable state, ask the expert to repair the trajectory, and learn from the correction.

SimDAgger is an experimental system for building failure-aware imitation-learning loops on top of RoboTwin 2.0. It extends ordinary behavior cloning with online failure detection, simulator state snapshots, task-aware takeover policies, expert recovery, dataset aggregation, and post-training.

The repository currently supports ACT, Diffusion Policy, DP3, and pi0 experiments. The primary workflow is privileged Recovery-DAgger: task-specific simulator progress trackers detect failure, trigger takeover, restore a recoverable snapshot, collect an expert correction, and post-train the policy. Visual progress/value models are maintained as an auxiliary research direction for non-privileged takeover.

Recovery in action

The clips below are expert recovery segments collected after SimDAgger detected a policy failure and restored a recoverable simulator state. These corrections become targeted supervision for DAgger post-training.

beat_block_hammer realign the hammer and finish the strike pi0 · recovery_ep014 takeover onset: not recorded · rewind: not recorded	place_empty_cup regrasp and place the cup pi0 · recovery_ep010 takeover onset: not recorded · rewind: not recorded

For runs that include a full episode recording, the left column shows the first half of the policy rollout before recovery (played at 4× speed); the right column shows the expert correction collected after rewind. Step annotations come from each run's episode_meta.json; older runs without those fields are marked as not recorded.

Policy rollout before takeover	Expert recovery
stack_blocks_two — regrasp and complete the stack pi0 · recovery_ep005 · takeover onset: step 298 (green_grasp_lost) · rewind: step 282
handover_block — restore ownership and finish placement pi0 · recovery_ep021 · takeover onset: step 264 (right_grasp_lost) · rewind: step 250
click_bell — recover alignment and press the bell pi0 · recovery_ep019 · takeover onset: step 90 (approach_timeout) · rewind: not recorded
place_can_basket — recover the can placement and basket lift pi0 · recovery_ep016 · takeover onset: step 114 (can_grasp_attempt_failed) · rewind: step 66
open_laptop — reacquire the handle and open the lid ACT · recovery_ep011 · takeover onset: step 182 (lid_opening_stalled) · rewind: step 181

Post-training results

The recovery clips show how targeted corrections are collected; the plots below show how policies change after training on those corrections. Each marker is an evaluated post-training checkpoint, the dashed line is the original policy baseline, and legend labels report the clean/recovery episode mixture used by that experiment.

beat_block_hammer · ACT	beat_block_hammer · pi0
click_bell · pi0	handover_block · pi0
open_laptop · ACT	place_empty_cup · ACT
stack_blocks_two · pi0

Best success rates

The results below are summarized from dagger_results_v5.xlsx. For each task and policy, we report the best recorded checkpoint rather than averaging across checkpoints. The three-seed ACT experiment on beat_block_hammer uses the seed mean within each checkpoint. Improvements are measured in percentage points (pp); relative improvement is (Best DAgger - Baseline) / Baseline.

Task	Policy	Baseline	Best Control	Control gain	Best DAgger	DAgger gain	Relative gain	DAgger vs. Control
open_laptop	ACT	66%	86%	+20 pp	85%	+19 pp	+28.8%	-1 pp
handover_block	pi0	26%	55%	+29 pp	75%	+49 pp	+188.5%	+20 pp
stack_blocks_two	pi0	41%	61%	+20 pp	82%	+41 pp	+100.0%	+21 pp
click_bell	pi0	41%	58%	+17 pp	94%	+53 pp	+129.3%	+36 pp
beat_block_hammer	ACT	53%	69.3%	+16.3 pp	88%	+35 pp	+66.0%	+18.7 pp
beat_block_hammer	DP	40%	43%	+3 pp	65%	+25 pp	+62.5%	+22 pp
beat_block_hammer	pi0	57%	—	—	86%	+29 pp	+50.9%	—
place_empty_cup	ACT	44%	52%	+8 pp	85%	+41 pp	+93.2%	+33 pp

Best DAgger exceeds the corresponding baseline for every task-policy pair. It also outperforms the best control wherever a control result is available, except on open_laptop, where the two are effectively tied.

Recovery data yield

The table below measures how efficiently policy rollouts produce usable recovery demonstrations. A valid recovery is an episode with a saved recovery_ep*/data/episode0.hdf5. Collection yield is valid recoveries / all rollouts, while failure conversion is valid recoveries / failed policy rollouts.

Task	Policy	Rollouts	Policy failures	Valid recoveries	Collection yield	Failure conversion	Rollouts per 100 recoveries
beat_block_hammer	pi0	100	44	38	38.0%	86.4%	263
beat_block_hammer (supplementary collection)	pi0	50	27	25	50.0%	92.6%	200
place_empty_cup	ACT	100	58	46	46.0%	79.3%	217
click_bell	pi0	100	71	70	70.0%	98.6%	143
handover_block (round 1)	pi0	100	64	44	44.0%	68.8%	227
stack_blocks_two (round 1)	pi0	150	107	65	43.3%	60.7%	231
stack_blocks_two (round 2)	pi0	150	54	38	25.3%	70.4%	395
open_laptop	ACT	100	33	26	26.0%	78.8%	385
place_can_basket	pi0	20	17	6	30.0%	35.3%	333

These counts describe generated data, not the smaller curated subsets necessarily selected for post-training. Yield can decrease in later rounds as the policy improves and produces fewer failed rollouts, as seen in stack_blocks_two.

Why SimDAgger?

A policy trained only on clean demonstrations rarely sees the states created by its own mistakes. Standard DAgger addresses this distribution shift, but collecting useful corrections for long-horizon manipulation introduces three practical questions:

1. When should the expert take over? Triggering too early wastes data; triggering too late may leave an unrecoverable scene.
2. Where should recovery begin? SimDAgger snapshots the simulator around a detected failure and rewinds to a verified state.
3. What should be recorded? Only successful expert recovery segments are retained as training data.

SimDAgger's main path uses privileged simulator state during data generation and evaluation, while keeping privileged signals out of the learned policy inputs. A complementary exploratory path infers task progress from multi-view RGB observations.

System overview

Primary workflow: task-aware privileged takeover and recovery

The modular path separates physical facts from decisions:

SAPIEN environment
  -> envs/predicates          reusable contact and geometry facts
  -> envs/task_progress       task phase, milestones, and temporal signals
  -> dagger_takeover          failure decision, rewind step, recovery context
  -> smarter_dagger_collect   rollout, snapshot restore, expert dispatch, recording

This separation keeps task semantics testable. A progress tracker reports what is true; a takeover policy decides whether that state is a failure; a recovery handler decides how to continue after rewind.

Auxiliary exploration: visual progress and value models

The visual branch is not required by the primary privileged workflow. It explores whether progress estimated from RGB can eventually replace simulator-only failure signals during online takeover.

The visual path trains a progress/value model from evaluation rollouts recorded with three RGB cameras. Online takeover can then use a persistent drop from the best predicted progress instead of simulator-only object state. Task-specific phased rules are also supported—for example, a grasp deadline followed by a placement progress-drop rule.

The current exploratory model predicts normalized task progress together with auxiliary planar and vertical geometry. Takeover is derived from temporal progress regression rather than a separately trained onset head.

Repository map

Path	Purpose
script/smarter_dagger_collect.py	Main privileged Recovery-DAgger collector; collect, detect-only, and confirm modes
script/non_privileged_dagger_collect.py	RGB progress/value-model takeover collector
envs/predicates/	Reusable geometry, contact, grasp, articulation, and temporal predicates
envs/task_progress/	Task-specific phase and milestone trackers
dagger_takeover/	Failure detection, rewind selection, and recovery context
envs/utils/state_snapshot.py	Save and restore robot, rigid-body, and articulation state
value_model/	RGB cache construction, labels, progress-model training, benchmarks, and visualization
dagger_agent/	Experimental agent for planning and verifying the end-to-end workflow
policy/	ACT, DP, DP3, and pi0 training/deployment code
tests/	Behavioral tests for predicates, progress, takeover, snapshots, and recovery

Agent skills

SimDAgger includes a set of project-specific agent skills under skills/. They capture the operational knowledge behind the experiments: required checks, portable command templates, completion criteria, and common failure modes. The skills are useful both as instructions for coding agents and as concise runbooks for researchers.

Start with robotwin-dagger-workflow. It is the top-level router for a complete Recovery-DAgger round and points to the specialized skill needed at each stage:

robotwin-dagger-workflow
  ├─ huggingface-robotwin-data       download and verify demonstrations
  ├─ robotwin-act-train-eval         preprocess, train, resume, and evaluate ACT
  ├─ robotwin-privileged-dagger-workflow  task-aware privileged Recovery-DAgger
  ├─ robotwin-privileged-takeover         design failure rules and recovery modes
  ├─ robotwin-tracker-takeover-recovery   implement and test task trackers
  └─ value-model-pipeline            build caches, train progress models, benchmark takeover

The task-aware privileged path has three additional guides:

Skill	When to use it
robotwin-privileged-dagger-workflow	Plan the full privileged DAgger extension for a new task
robotwin-privileged-takeover	Design task-specific failure rules, rewind selection, and thresholds
robotwin-tracker-takeover-recovery	Implement predicates, TaskProgress, TakeoverPolicy, recovery dispatch, and behavioral tests

Skills are deliberately evidence-driven: files, logs, checkpoints, episode counts, and evaluation results determine whether a stage is complete. Checkpoint replacement, process termination, and other high-impact actions still require explicit human confirmation. Paths and runtime settings are expressed portably so the workflows can be adapted to a local workstation or an independently managed compute environment.

Recovery-DAgger workflow

1. Prepare expert demonstrations
2. Train an initial ACT or pi0 policy
3. Evaluate and record trajectories/RGB observations
4. Build labels and train a visual progress model (optional)
5. Detect a failure and save the takeover state
6. Confirm failure, rewind, and run task-aware expert recovery
7. Keep only successful recovery demonstrations
8. Merge clean and recovery data, then post-train
9. Re-evaluate and repeat if the failure distribution still matters

The --confirm collector mode is recommended for clean data: the policy continues after the first trigger, and recovery is collected only if the episode ultimately fails.

Quick start

1. Install the RoboTwin base environment

SimDAgger inherits RoboTwin's simulator, assets, and policy dependencies. Follow the upstream RoboTwin installation guide, then install the dependencies required by the policy you plan to train.

2. Collect or download demonstrations

CUDA_VISIBLE_DEVICES=<gpu_id> python script/collect_data.py <task_name> <task_config>

Example:

CUDA_VISIBLE_DEVICES=0 python script/collect_data.py beat_block_hammer demo_clean

3. Train and evaluate a base policy

Policy-specific code lives under policy/. Evaluation uses script/eval_policy.py; enable RGB recording when the result will be used to train a visual progress model.

python script/eval_policy.py --config policy/ACT/deploy_policy.yml --overrides \
  --task_name beat_block_hammer \
  --task_config demo_clean \
  --ckpt_setting demo_clean_s0 \
  --ckpt_dir policy/ACT/act_ckpt/act-beat_block_hammer/demo_clean-50 \
  --ckpt_name policy_last.ckpt \
  --seed 0 --temporal_agg true --test_num 50 --record_rgb true

4. Collect confirmed recovery demonstrations

python script/smarter_dagger_collect.py --confirm \
  --policy ACT \
  --task_name stack_blocks_two \
  --task_config demo_clean \
  --expert_data_num 50 \
  --seed 0 --num_episodes 30 --gpu 0

For non-privileged takeover, provide a trained progress model:

python script/non_privileged_dagger_collect.py --confirm \
  --policy pi0 \
  --task_name beat_block_hammer \
  --task_config demo_clean \
  --expert_data_num 50 \
  --seed 0 --num_episodes 30 --gpu 0 \
  --trigger progress \
  --value_ckpt value_model/runs/<run-name>/best.ckpt

Run python script/smarter_dagger_collect.py --help before launching an experiment; task and policy branches do not all expose identical options.

Collector modes

Mode	Flag	Behavior
Immediate recovery	default	Stop at the first takeover signal and invoke the expert
Detection benchmark	--detect_only	Finish the rollout and report detector precision/recall without recovery
Confirmed recovery	--confirm	Snapshot at trigger, finish rollout, then rewind and recover only confirmed failures

Typical outputs include rollout diagnostics, traj_data.npz, optional per-episode RGB archives, recovery HDF5 files, videos, and a structured results.pkl summary. Generated datasets, checkpoints, videos, caches, and logs are intentionally excluded from Git.

Supported task-aware modules

The modular progress/takeover implementation currently includes:

• stack_blocks_two
• open_laptop
• handover_block
• place_can_basket

Other tasks may still use legacy detectors embedded in the collector. Adding a new task normally requires a progress tracker, takeover policy, recovery handler, registry entries, and behavioral tests.

Testing

Run the lightweight behavioral suite with an environment that provides NumPy:

python -m unittest discover -s tests -p 'test_*.py' -v

Full simulator rollouts require SAPIEN, task assets, policy checkpoints, and a rendering-capable GPU. Unit tests do not replace rollout validation, especially for restored grasps and articulation state.

Current status

• Simulator snapshots and confirmed rollback/recovery collection are implemented.
• Privileged predicates, task progress, and takeover decisions are modularized for several tasks.
• Multi-view visual progress models and offline takeover benchmarks are available.
• ACT and pi0 post-training/evaluation workflows have been exercised on multiple tasks.
• dagger_agent/ is an experimental orchestration layer; some stages still require task-specific engineering and human confirmation.

RoboTwin attribution

SimDAgger is built on RoboTwin 2.0 and retains its simulation tasks, assets, and policy baselines. Please cite RoboTwin when using this repository:

@article{chen2025robotwin,
  title={RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation},
  author={Chen, Tianxing and Chen, Zanxin and Chen, Baijun and Cai, Zijian and Liu, Yibin and others},
  journal={arXiv preprint arXiv:2506.18088},
  year={2025}
}

For the original platform documentation, task definitions, dataset, and licensing context, visit the RoboTwin website and upstream repository.

License

This repository retains the upstream MIT license. See LICENSE.