Skip to content

Commit 54b6acd

Browse files
ashishgaurav13ashishgaurav13
committed
update README
1 parent 29a843a commit 54b6acd

File tree

5 files changed

+49
-25
lines changed

5 files changed

+49
-25
lines changed

‎README.md

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,34 @@
11
## Inverse Constraint Learning
2-
This repository contains the code for ICL paper. After you run any command, the
3-
results will be logged to tensorboard.
4-
5-
## Before running anything
6-
* Install OpenMPI.
7-
* Ensure Mujoco libraries (2.1.0) are installed.
8-
* Update lines 410-413 in `tools/environments/exiD_environment.py` to reflect the directory of
9-
ExiD dataset.
2+
3+
Paper: [Learning Soft Constraints From Constrained Expert Demonstrations, Gaurav et al. (2023)](https://openreview.net/forum?id=8sSnD78NqTN)
4+
5+
This repository contains the code for ICL paper. After you run any command, the results will be logged to tensorboard.
6+
7+
## How does it work?
8+
9+
Constrained RL takes in reward and constraint(s) and produces an optimal constrained policy.
10+
11+
<img src="images/crl.png" width=400>
12+
13+
The inverse problem, i.e. Inverse Constrained RL takes in a dataset of trajectories sampled using an optimal expert and produces a set of reward and constraint(s) such that they produce the expert policy when CRL is performed with them.
14+
15+
<img src="images/icrl.png" width=400>
16+
17+
Due to unidentifiability, Inverse Constrained RL is a difficult problem. Hence, we solve a simplified problem - i.e. we assume the reward is known and that we only need to learn a single constraint.
18+
19+
<img src="images/icl.png" width=400>
20+
21+
The idea is inspired from the IRL template, which alternates between policy optimization and reward adjustment. In our case, we alternate between constrained policy optimization and constraint function adjustment.
22+
23+
<img src="images/template.png" width=400>
24+
25+
For further details regarding the optimization and algorithm, please see the [paper](https://openreview.net/forum?id=8sSnD78NqTN).
26+
27+
We conduct several experiments across synthetic environments, robotics environments and real world highway environments. The steps to run these experiments are detailed further in this README.
28+
29+
## Setup
30+
* Install OpenMPI and Mujoco 2.1.0
31+
* Update `tools/__init__.py` constants to have the correct directories for ExiD dataset.
1032
* Install `tools` package by running `pip install .` in the root directory.
1133

1234
## High level workflow
@@ -31,9 +53,9 @@ ExiD dataset.
3153
data in `tools/assets/exiD`, already provided, which was generated using `prepare_exid_data.py`)
3254
* Generate for other environments: `python3 -B expert.py -c configs/ENV.json`
3355
* Run methods
34-
* ICL: `python3 -B 03-icl-mix-improved.py -c configs/ENV.json -seed SEED -beta BETA`
35-
* GAIL-Constraint: `python3 -B 11-gail.py -c configs/gail-ENV.yaml -seed SEED`
36-
* ICRL: `python3 -B 12-icrl.py -c configs/icrl-ENV.yaml -seed SEED`
56+
* ICL: `python3 -B icl.py -c configs/ENV.json -seed SEED -beta BETA`
57+
* GAIL-Constraint: `python3 -B gail_constraint.py -c configs/gail-ENV.yaml -seed SEED`
58+
* ICRL: `python3 -B icrl.py -c configs/icrl-ENV.yaml -seed SEED`
3759

3860
## Credits
3961

@@ -50,7 +72,7 @@ Please check the individual repositories for licenses.
5072
* ExiD dataset
5173
* https://www.exid-dataset.com/
5274
* Free for non-commercial use, but to get the dataset, you must request it.
53-
* You must place this dataset in (any) directory and update `tools/environments/exiD_environment.py` as
75+
* You must place this dataset in (any) directory and update `tools/__init__.py` as
5476
mentioned previously.
5577
* Wise-Move environment
5678
* https://git.uwaterloo.ca/wise-lab/wise-move

‎prepare_exid_data.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,10 @@
44
from tools.exid import load_trajectories2
55
from shapely.geometry import Point, LineString
66
import glob, os, pandas
7+
import tools
78

8-
DATA_DIR = os.path.expanduser(
9-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/data/")
10-
MAPS_DIR = os.path.expanduser(
11-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/maps/lanelet2/")
9+
DATA_DIR = tools.store["DATA_DIR"]
10+
MAPS_DIR = tools.store["MAPS_DIR"]
1211
nums = [item.split("/")[-1].split("_")[0] \
1312
for item in glob.glob(DATA_DIR+"*_tracks.csv")]
1413
for num in nums[:5]:

‎tools/__init__.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,9 @@
1111

1212
import os
1313
# Package level constants
14-
store = data.Configuration({})
14+
store = data.Configuration({
15+
"DATA_DIR": os.path.expanduser(
16+
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/data/"),
17+
"MAPS_DIR": os.path.expanduser(
18+
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/maps/lanelet2/"),
19+
})

‎tools/environments/exiD_environment.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,17 +15,16 @@
1515
find_closest_node, find_segment
1616
from shapely.geometry import Point, LineString
1717
from gym import spaces
18+
import tools
1819

1920

2021
class ExiDSampleEnvironmentLateral(Environment):
2122

2223
def __init__(self):
2324
self.dt = 1/25. # from meta file
2425
self.ers = []
25-
DATA_DIR = os.path.expanduser(
26-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/data/")
27-
MAPS_DIR = os.path.expanduser(
28-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/maps/lanelet2/")
26+
DATA_DIR = tools.store["DATA_DIR"]
27+
MAPS_DIR = tools.store["MAPS_DIR"]
2928
nums = [item.split("/")[-1].split("_")[0] \
3029
for item in glob.glob(DATA_DIR+"*_tracks.csv")]
3130
for num in nums[:5]:

‎tools/exid/exid_gym.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
from .exid_sim import load_trajectories, load_trajectories2, lane_change_start_end, \
1515
find_closest_node, find_segment, find_segment2
1616
from shapely.geometry import Point, LineString
17+
import tools
1718

1819
class ExiDSampleEnvironmentLateral(Environment):
1920

@@ -169,10 +170,8 @@ class ExiDSampleEnvironmentLateral2(Environment):
169170
def __init__(self):
170171
self.dt = 1/25. # from meta file
171172
self.ers = []
172-
DATA_DIR = os.path.expanduser(
173-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/data/")
174-
MAPS_DIR = os.path.expanduser(
175-
"~/Projects/Datasets/exiD/exiD-dataset-v2.0/maps/lanelet2/")
173+
DATA_DIR = tools.store["DATA_DIR"]
174+
MAPS_DIR = tools.store["MAPS_DIR"]
176175
nums = [item.split("/")[-1].split("_")[0] \
177176
for item in glob.glob(DATA_DIR+"*_tracks.csv")]
178177
for num in nums[:5]:

0 commit comments

Comments
 (0)