face_recognition

🎭 Face Recognition System

This repository contains a complete face recognition system developed as part of a Computer Vision project at FAU (Winter Term 2023/24). The project is based on a skeleton provided by the instructors and is highly inspired by the work in LORD: Leveraging Open-Set Recognition with Unknown Data (ICCV 2023).

We implement real-time face identification and clustering from videos or webcam, featuring robust face detection, tracking, alignment, feature extraction, and open-set recognition.

---

📌 Features

• 🔍 Face Detection: MTCNN-based detector to localize faces.
• 🎯 Face Tracking: Fast template matching to track faces across frames.
• 🧼 Face Alignment: Normalizes faces for consistent feature extraction.
• 🧠 Deep Feature Embedding: Uses a ResNet50-based FaceNet model to extract 128D embeddings.
• 🧮 Cosine Distance Matching: For face recognition using k-NN with cosine similarity.
• 🧊 Open-Set Handling: Recognizes unknown faces using distance and confidence thresholds.
• 📷 Camera-Compatible: Train and test with webcam input.
• 🖥️ CPU-Compatible: Runs smoothly without a GPU.

---

⚙️ Installation

Clone the repo and install dependencies:

git clone https://github.com/ArchitNK/computer_vision.git
cd computer_vision/face_recognition
pip install -r requirements.txt

🧪 Modes of Operation

The system supports two modes:

• ident — Supervised face identification
• cluster — Unsupervised face clustering

---

🔐 Identification Mode (ident)

Train and test the system to recognize labeled individuals using videos or webcam.

➤ Train with a video:

python training.py --mode ident --video data/Nancy_Sinatra_training/Nancy_Sinatra_Video.mp4 --label Nancy_Sinatra

➤ Train with your own face using webcam:

python training.py --mode ident --label Your_Name

Embeddings are stored in: data/recognition_gallery.pkl

➤ Test with a video:

python test.py --mode ident --video data/Nancy_Sinatra_test/Nancy_Sinatra_Video.mp4

➤ Test with a webcam:

python test.py --mode ident

➤ Test with a unknown identity:

python test.py --mode ident --video data/Al_Pacino/Al_Pacino_Video.mp4

For cluster mode just make --mode cluster

🧠 Recognition Details

The recognition pipeline uses deep learning-based face embeddings and distance-based classification:

• Embedding Extraction:

  • Aligned face images are passed through a ResNet-50-based FaceNet model (ONNX format).
  • Output is a 128-dimensional embedding vector per face.

• Distance Metric:

  • Recognition is performed using cosine similarity in the embedding space.
  • A k-Nearest Neighbors (k-NN) approach is used to find the closest known identities.

• Open-Set Handling:

  • The system can identify unknown faces using a decision threshold:
    • Distance threshold (τₑ): Reject if similarity is too low.
    • Posterior probability threshold (τₚ): Reject if classifier confidence is too low.
    • If a test face is too far from all known embeddings or has low confidence, it is labeled as: 'Unknown'

• Storage:

  • All known face embeddings and labels are saved in:

    data/recognition_gallery.pkl
    

This makes the system robust to new and unseen identities, supporting both closed-set and open-set scenarios.

📂 Directory Structure

face_recognition/
├── data/
│   ├── recognition_gallery.pkl         # Stored known face embeddings
│   ├── clustering_gallery.pkl         # Stored known face clusters
│   ├── [Training and Test Videos]      # Video files for training/testing
│   └── resnet50_128.onnx               # Face embedding model (download separately from https://github.com/ox-vgg/vgg_face2)
├── training.py                         # Script for training identification or clustering models
├── test.py                             # Script for testing identification or clustering
├── face_detector.py                   # Face detection using MTCNN and tracking
├── face_recognizer.py                 # Face recognition implementation using k-NN
├── requirements.txt                   # Python dependencies

📸 Demo

Figure 1: Face recognition training

Figure 2: Face recognition testing (known person)

Figure 3: Face recognition testing (Unknown person)

🧾 Acknowledgments

Institution: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Course: Computer Vision - Winter Term 2023/24

Instructors: Thomas Köhler

Inspiration: LORD: Leveraging Open-Set Recognition with Unknown Data – ICCV 2023,
VGGFace2, MTCNN PyPI