ManyTor

This is a project inspired by the gym module, we seeks to bring facilities to researchers and developers who work with robotic manipulators and/or reinforcement learning.

Environment

For general purpose, i decided to create a new environmnet instead use one environment already created. Is this project i use vispy to plot and simulate links, joints and objectives, as well as ground. My base environment consist in one manipulator with 4 joints (3+terminal) with the Denavit-Hartember parameters present here on the right.

Obs: I am currently working to generalize the code with the purpose of adapting the simulation to any manipulator model only by changing its forward kinematic function.

Setup

All of requirements is show in the badgets above, but if you want to install all of them, enter the repository and execute the following line of code:

pip3 install -r requirements.txt

Objectives

The environment objectives consist in random points refreshing when all points is colected. Each point have one fixed reward and one variable reward, but that will be explained soon.

Reward System

As the agent observes the current state of the environment and chooses an action, the environment transitions to a new state, and also returns a reward that indicates the consequences of the action. In this task, rewards are +1 for every time the agent catches a ball and the environment terminates if all the balls were picked up or the manipulator hit the ground. This means that the most performing scenarios will end in timesteps, where means the number of objectives.

If you liked this repository, please don't forget to starred it!