T-FAKE: Synthesizing Thermal Images for Facial Landmarking

Downloading the dataset

To download the color images, sparse annotations, and segmentation masks for the dataset, please use the links in the FaceSynthetics repository.

Our dataset has been generated for a warm and for a cold condition. Each dataset can be downloaded separately as

• A small sample with 100 images from here warm and here cold
• A medium sample with 1,000 images from here warm and here cold
• The full dataset with 100,000 images from here warm and here cold
• The dense annotations are available from here

Using the landmarker

Coming soon.

Pre-trained models

The models for the thermalization as well as the landmarkers can be downloaded from here.

Thermalization

Our baseline U-Net translation model is imported from segmentation_models_pytorch library. Specifically, we define the translator as follows:

import segmentation_models_pytorch as smp

translator = smp.Unet(
    encoder_name="resnet34",        
    encoder_weights="imagenet",     
    in_channels=3,                  
    classes=1,                      
    activation="sigmoid"
)

This model is based on a U-Net architecture with a ResNet-34 encoder pre-trained on ImageNet. It takes three-channel RGB input images and outputs a single-channel thermal image with a sigmoid activation function. For training progress of the thermalization model see ThermalizationCode/ThermalizerOutput.ipynb.

Landmarking

Will be added soon.

Running the benchmark

To run the benchmark, you have to download the CHARLOTTE ThermalFace dataset.

License

This dataset and the landmarking methods are licensed under the Attribution-NonCommercial-ShareAlike 4.0 International license as it is derived from the FaceSynthetics dataset.

Citation

If you use this code for your own work, please cite our paper:

P. Flotho, M. Piening, A. Kukleva and G. Steidl, “T-FAKE: Synthesizing Thermal Images for Facial Landmarking,” arxiv, 2024. doi:10.48550/arXiv.2408.15127

BibTeX entry

@article{flotea2024a,
    author = {Flotho, P. and Piening, M. and Kukleva, A. and Steidl, G.},
    title = {T-FAKE: Synthesizing Thermal Images for Facial Landmarking},
    year = {2024},
    journal = {arXiv preprint arXiv:2408.15127},
    doi = {https://doi.org/10.48550/arXiv.2408.15127}
}