FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors

Website | Arxiv | YouTube

Table of Contents

• Update
• Installation
• Real-Time prediction on GelSight Mini
• Calibration
• Citing this paper

Update

• 01/30/25 Accepted to ICRA25!
• 03/01/25 Data release
• 03/24/25 First release of the code

⚙️ Installation

1️⃣ Clone this repository

To clone this repository you can launch the following command

git clone https://github.com/prgumd/FeelAnyForce.git
cd FeelAnyForce

2️⃣ Create and activate the virtual environment

To create and activate the virtual environment to work with this repository you can launch the following command

conda env create -f env.yml
conda activate anyforce_env

or

virtualenv anyforce_env
source anyforce_env/bin/activate
pip install -r requirements.txt

3️⃣ Download the data

Create dataset folder and download the data with

bash dataset_downloader.sh

4️⃣ Training from scratch

To train the network,

python training_multi_head.py --epochs 100 --labels_train ./dataset/TacForce_train_set.csv.csv --labels_val ./dataset/TacForce_val_set.csv --ckpt_dir $ckpt_dir --data_basedir ./dataset/ --training_name TacForce_v1 

🎬 Real-Time prediction on GelSight Mini

wget -P ckpt/ https://huggingface.co/amirsh1376/FeelAnyForce/resolve/main/checkpoint_v1.pth.tar   # download the checkpoint
python online_test.py --checkpoint ckpt/checkpoint_v1.pth.tar

🩹 Calibration

Stay tuned!

Citing this paper

@misc{shahidzadeh2024feelanyforceestimatingcontactforce,
      title={FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors}, 
      author={Amir-Hossein Shahidzadeh and Gabriele Caddeo and Koushik Alapati and Lorenzo Natale and Cornelia Fermüller and Yiannis Aloimonos},
      year={2024},
      eprint={2410.02048},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2410.02048}, 
}

🧔 Maintainer

This repository is maintained by:

	@AmirShahid
	@gabrielecaddeo

Acknowledegments

Thanks to these great repositories: DinoV2, Dino, Sim2Surf, gsrobotics