This repository is based on Assistive Gym which is a physics-based simulation framework for physical human-robot interaction and robotic assistance.
This code accompanies the submission:
"A Multi-Agent Simulation Environment for Human-Robot Collaboration in an Industrial Setting"
Gabriele Ansaldo
The Reaching Task consists of two sub-tasks, a deterministic and subsequently a stochastic one. The first sub-task is called deterministic since the robot’s joint angles and movements are predetermined and hardcoded. The second sub-task is referred to as stochastic since the robot’s joint positions are not predetermined but are the result of a Reinforcement Learning policy learned in simulation using Assistive Gym. For the Reaching Task, a blue and a red marker are placed on a shelf out of reach from a human operator. The position of the markers is constant for all simulations. Firstly, in the deterministic sub-task, the human points at the desired marker, and through computer vision (ZED camera), the robot understands which of the markers was chosen. The chosen marker is then grabbed by the robot using predetermined joint positions. Secondly, the stochastic sub-task consists of the robot passing the marker to the human’s right hand. This sub-task is solely based on the environment designed in Assistive Gym which allows the robot to learn to reach a human’s hand.
An example of the designed task can be seen in the following videos:
Picking the RED marker:
Picking the BLUE marker:
The repository is composed of two main folders. One folder contains the files created for designing the simulation environment, and the other contains files related to transferring a learned policy onto a real Sawyer Robot with the help of a ZED 2 stereo camera.
Here is a side by side picture of the real environment and the designed environment.
The designed environment was trained in simulation for approximately 10,000,000 timesteps. The following video shows how the trained model performs.
The policy learned in simulation was then transfered to the real environment. The following video shows how the trained model performs in real life.
For the design of the environment Assistive Gym was utilized. For details on how to install Assistive Gym please check out the installation guide for Assistive Gym.
Once Assistive Gym is installed replace/add the following files and folders with the ones present in Folder 1 (Designed Environment):
Replace/add the following files at given location:
| File Name | Location | Actions |
|---|---|---|
config.ini |
/assistive-gym/assistive_gym/ |
Replace |
__init__.py |
/assistive-gym/assistive_gym/ |
Replace |
__init__ 2.py |
/assistive-gym/assistive_gym/envs/ |
Rename to __init__.py and Replace |
reaching.py |
/assistive-gym/assistive_gym/envs/ |
Add |
reaching_envs.py |
/assistive-gym/assistive_gym/envs/ |
Add |
furniture.py |
/assistive-gym/assistive_gym/envs/agents/ |
Replace |
sawyer.py |
/assistive-gym/assistive_gym/envs/agents/ |
Replace |
tool.py |
/assistive-gym/assistive_gym/envs/agents/ |
Replace |
Add the following folders at given location:
| Folder Name | Location | Actions |
|---|---|---|
ZED_camera |
/assistive-gym/assistive_gym/envs/assets/ |
Add |
lab_table |
/assistive-gym/assistive_gym/envs/assets/ |
Add |
lab_shelf |
/assistive-gym/assistive_gym/envs/assets/ |
Add |
lab_marker |
/assistive-gym/assistive_gym/envs/assets/ |
Add |
ReachingSawyer-v1 |
/assistive-gym/trained_models/ppo/ |
Add |
It will now be possible to render the designed environment in simuation. The below command will render a Sawyer robot taking random actions within the Reaching Task with a static person.
python3 -m assistive_gym --env "ReachingSawyer-v1"In order to render a single rollout of the trained policy use the following command:
python3 -m assistive_gym.learn --env "ReachingSawyer-v1" --algo ppo --render --seed 0 --load-policy-path ./trained_models/ --render-episodes 10Refer to the ZED 2 installation guide and Sawyer SDK installation guide to set up both the camera and robot.
The second folder (Simulation to Real) contains two main codes:
ReachingTask.pyis the main code used to run the full task on the Sawyer and ZED cameracoordinate_system_calibration.pyis utilized to calibrate the ZED coordinate system with the Sawyer reference frame
Move the entire Folder 2 to/assistive-gym/.
This code contains both the deterministic and stochastic sub-tasks. It is necessary to use both the Sawyer robot and ZED 2 camera in order to run this code. However, if different hardware is used, this code may be used as a template for further modification.
A crucial step in assuring proper functioning of the ReachingTask.py code is to perform an accurate calibration of the ZED and Sawyer coordinate systems. This code is used to perform calibration. The following video shows how calibration is performed:
