Skip to content

README.md

Latest commit

 

History

History
181 lines (144 loc) · 11.6 KB

README.md

File metadata and controls

181 lines (144 loc) · 11.6 KB

Scheduling and Path Planning for Multi-Agent Reinforcement Learning using Graph Neural Networks

Task-Agent RElational Graph Encoding for Team-based NAvigation and Execution of Tasks (TARGETNET) is is a framework for multi-agent path planning and scheduling in a dynamic environment. The framework is based on the Graph Neural Networks (GNNs) and Reinforcement Learning (RL) techniques. The framework is designed to be used in a variety of applications, including autonomous vehicles, robotics, and logistics.

Installation

# Clone the repository

Architecture

The TARGETNET framework consists of three main components: the environment, the path planning and the solver models. The environment is a dynamic graph that represents the environment in which the agents operate. The path planning model is a graph neural network that learns to predict the optimal path for the agents. The solver model is a reinforcement learning model that learns to schedule the agents in the environment.

Environment

The environment is a Task Allocation and Scheduling with Path Planning Environment designed to account for Spatio-Temporal Constraints.

Experiments

Problem Generation

Installation

The training and evaluation utilize tmux to run multiple processes. The following is the installation of tmux

sudo apt-get install tmux

The following sets up the environment for the project using environment.yml

# Clone the repository
conda env create -f environment.yml

Problem Generation

The problem generation is done using the following script. The script generates the training and test sets for the Euclidean Distance. The script generates the problems for the following configurations:

  • 10 robot 20 task 10% time window 25% wait time
  • 10 robot 50 task 10% time window 25% wait time
  • 20 robot 100 task 10% time window 25% wait time
  • 40 robot 200 task 10% time window 25% wait time

The script also generates the problems for the following ablation studies:

  • 10 robot 20 task [0-5% time window] 25% wait time
  • 10 robot 20 tasks [5-10% time window] 25% wait time
  • 10 robot 20 task 10% time window [No wait time]
  • 10 robot 20 task 10% time window [50% wait time]

You can run the following command to generate the problems for the Euclidean Distance:

bash generate_problems_neurips2025.sh

Or you can use the following commands to generate the problems individually:

## (NEURIPS 2025) Problem Generation
########## TRAINING SET ###############
# 10 robot 20 task 10% time window 25% wait time
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w25_euc_2000_uni --config_file data/problem_r10_t20_s0_f10_w25_config.json --start_problem 1 --end_problem 2000 --seed 10
########## TEST SET ###############
# 10 robot 20 task 10% time window 25% wait time
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w25_euc_2000_test --config_file data/problem_r10_t20_s0_f10_w25_config.json --start_problem 1 --end_problem 2000 --seed 2000
# 10 robot 50 task 10% time window 25% wait time
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t50_s0_f10_w25_euc_2000_uni --config_file data/problem_r10_t50_s0_f10_w25_config.json --start_problem 1 --end_problem 2000 --seed 10
# 20 robot 100 task 10% time window 25% wait time -> 50 Problems
python scheduling/generate_large_problems.py --shared_folder data/problem_set_r20_t100_s0_f10_w25_euc_200 --config_file data/problem_r10_t50_s0_f10_w25_config.json --number_of_agents 20 --number_of_tasks 100 --tmp_location data/tmp_r20_t100_s0_f10_w25 --start_problem 1 --end_problem 200 --seed 10
# 40 robot 200 task 10% time window 25% wait time -> 50 Problems
python scheduling/generate_large_problems.py --shared_folder data/problem_set_r40_t200_s0_f10_w25_euc_200 --config_file data/problem_r10_t50_s0_f10_w25_config.json --number_of_agents 40 --number_of_tasks 200 --tmp_location data/tmp_r40_t200_s0_f10_w25 --start_problem 1 --end_problem 200 --seed 10
# 80 robot 400 task 10% time window 25% wait time -> 50 Problems (Stretch)
python scheduling/generate_large_problems.py --shared_folder data/problem_set_r80_t400_s0_f10_w25_euc_200 --config_file data/problem_r10_t50_s0_f10_w25_config.json --number_of_agents 40 --number_of_tasks 200 --tmp_location data/tmp_r80_t400_s0_f10_w25 --start_problem 1 --end_problem 200 --seed 10
## Ablation Studies
# 10 robot 20 task [0-5% time window] 25% wait time
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f5_w25_euc_2000 --config_file data/problem_r10_t20_s0_f5_w25_config.json --start_problem 1 --end_problem 200 --seed 10
# 10 robot 20 tasks [5-10% time window] 25% wait time
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s5_f10_w25_euc_2000 --config_file data/problem_r10_t20_s5_f10_w25_config.json --start_problem 1 --end_problem 200 --seed 10

# 10 robot 20 task 10% time window [No wait time]
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w0_euc_2000 --config_file data/problem_r10_t20_s0_f10_w0_config.json --start_problem 1 --end_problem 200 --seed 10
# 10 robot 20 task 10% time window [50% wait time]
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w50_euc_2000 --config_file data/problem_r10_t20_s0_f10_w50_config.json --start_problem 1 --end_problem 200 --seed 10

# 10 robot 20 task 10% time window 25% wait time fast
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w25_euc_200_fast --config_file data/problem_r10_t20_s0_f10_w25_config_fast.json --start_problem 1 --end_problem 200 --seed 10
# 10 robot 20 task 10% time window 25% wait time slow
python scheduling/generate_problems.py --shared_folder data/problem_set_r10_t20_s0_f10_w25_euc_200_slow --config_file data/problem_r10_t20_s0_f10_w25_config_slow.json --start_problem 1 --end_problem 200 --seed 10

Training

# Train the models, sequential:
bash train_all_neurips2025_sequential.sh
# Train the models, simultaneous:
bash train_all_neurips2025_simultaneous.sh

Evaluation

# Evaluate the models, sequential:
bash evaluate_neurips2025_sequential.sh
# Evaluate the models, simultaneous:
bash evaluate_neurips2025_simultaneous.sh

Reproduce the results in the paper

bash evaluate_neurips2025_reproduce.sh
bash evaluate_neurips2025_simultaneous_reproduce.sh

Evaluate time

python evaluation/evaluate_time.py --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test
python evaluation/evaluate_time.py --env_location data/problem_set_r10_t50_s0_f10_w25_euc_2000_uni
python evaluation/evaluate_time.py --env_location data/problem_set_r20_t100_s0_f10_w25_euc_200 --num_problems 5
python evaluation/evaluate_time.py --env_location data/problem_set_r40_t200_s0_f10_w25_euc_200 --num_problems 5
# python evaluation/evaluate_time.py --env_location data/problem_set_r80_t400_s0_f10_w25_euc_2000_uni --num_problems 5


python evaluation/evaluate_time_simultaneous.py --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test
python evaluation/evaluate_time_simultaneous.py --env_location data/problem_set_r10_t50_s0_f10_w25_euc_2000_uni
python evaluation/evaluate_time_simultaneous.py --env_location data/problem_set_r20_t100_s0_f10_w25_euc_200 --num_problems 5
python evaluation/evaluate_time_simultaneous.py --env_location data/problem_set_r40_t200_s0_f10_w25_euc_200 --num_problems 5
# python evaluation/evaluate_time_simultaneous.py --env_location data/problem_set_r80_t400_s0_f10_w25_euc_2000_uni --num_problems 5

Post-Hoc Analysis:

Additional results, such as optimality rate and optimality gap, can be obtained using the following scripts. The results are saved in the final_results folder.

# Sequential Models:
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_200_test --initial_performance
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_200_test
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t50_s0_f10_w25_euc_2000_uni
python evaluation/optimality_gap.py --start_problem 1 --end_problem 30 --env_location data/problem_set_r20_t100_s0_f10_w25_euc_200
python evaluation/optimality_gap.py --start_problem 1 --end_problem 10 --env_location data/problem_set_r40_t200_s0_f10_w25_euc_200
# python evaluation/optimality_gap.py --start_problem 1 --end_problem 10 --env_location data/problem_set_r80_t400_s0_f10_w25_euc_200
# Ablations
## 10 robot 20 task [5% time window] 25% wait time
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f5_w25_euc_2000
## 10 robot 20 tasks [10% time window] 25% wait time
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s5_f10_w25_euc_2000
## 10 robot 20 task 10% time window [No wait time]
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w0_euc_2000
## 10 robot 20 task 10% time window [50% wait time]
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w50_euc_2000

## 10 robot 20 task 10% time window 25% wait time fast
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test_fast
## 10 robot 20 task 10% time window 25% wait time slow
python evaluation/optimality_gap.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test_slow

# Simultaneous Models:
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_200_test --initial_performance
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_200_test
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t50_s0_f10_w25_euc_2000_uni
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 30 --env_location data/problem_set_r20_t100_s0_f10_w25_euc_200
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 10 --env_location data/problem_set_r40_t200_s0_f10_w25_euc_200
# "python evaluation/evaluation.py --start_problem 1 --end_problem 10 --env_location data/problem_set_r80_t400_s0_f10_w25_euc_200"
# Ablations
## 10 robot 20 task [5% time window] 25% wait time
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f5_w25_euc_2000
## 10 robot 20 tasks [10% time window] 25% wait time
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s5_f10_w25_euc_2000
## 10 robot 20 task 10% time window [No wait time]
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w0_euc_2000
## 10 robot 20 task 10% time window [50% wait time]
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w50_euc_2000

## 10 robot 20 task 10% time window 25% wait time fast
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test_fast
## 10 robot 20 task 10% time window 25% wait time slow
python evaluation/optimality_gap_sim.py --start_problem 1 --end_problem 200 --env_location data/problem_set_r10_t20_s0_f10_w25_euc_2000_test_slow