BISIG (FSL Intelligence) is a unified platform designed to empower the Filipino deaf and hard-of-hearing community by bridging the communication gap through real-time, bidirectional translation between Filipino Sign Language (FSL) and spoken/written language.
The project consists of three core components working together to provide a seamless translation experience:
A high-performance REST API designed to translate text into sign language video sequences and high-fidelity skeleton datasets.
- Multi-Language Support: ASL and full native support for Filipino Sign Language (FSL).
- Human Reference Preview: Side-by-side human reference video previews for all signs.
- Smart Variant Selection: Automatically detects and selects between linguistic variants.
- High-Fidelity Tracking: Tracks 478 face landmarks and 33 pose landmarks for accurate avatar rendering.
A modern web-native interface that connects to the backend API to provide a user-friendly translation experience.
- Real-time Interface: Accessible via any modern browser.
- Pose Estimation: Uses TensorFlow.js and MediaPipe for on-device pose detection.
- Visual Feedback: Renders translations via 3D Avatars (Three.js), skeletal data, or photorealistic videos.
The core data repository and high-performance media server.
- Unified Architecture: A Go-based server that handles media delivery and intelligent proxying for the entire ecosystem.
- Large Dataset: Managing over 9,900+ FSL video files.
- Authentication: A complete auth system (Node.js/SQLite) for progress tracking and user management.
Backend-API/: Core translation engine (Python/FastAPI/MediaPipe).Frontend/: React-based user interface.FSL-Datasets/: Go-based media server, dataset metadata, and authentication backend.
- Bidirectional Translation: Supports both Sign-to-Text and Text-to-Sign.
- Localized for FSL: Specifically trained and optimized for Filipino Sign Language.
- Hybrid Search: Intelligent fallback mechanisms between languages and fingerspelling.
- High-Resolution Data: Provides raw landmark coordinates for interactive 3D applications.
To run the entire ecosystem locally, please refer to the specific setup guides within each directory:
We would like to express our gratitude to the following organizations for making this project possible:
- FSL Datasets: Special thanks to the Iglesia ni Cristo for providing the comprehensive Filipino Sign Language datasets used in this project.
- ASL Datasets: We credit the Pocket Sign ASL App (pocketsign.org) for the American Sign Language video resources.
This project is licensed under the Apache License 2.0. See the LICENSE and NOTICE files for details.