Personalized Sign Language to Speech for Assistive Communication
Real-time Sri Lankan Sign Language to Sinhala speech translation
Overview
Communication barriers remain a major challenge for people with hearing and speech impairments, especially where localized assistive tools are scarce. This project builds a real-time Sri Lankan Sign Language (SLSL) → Sinhala speech translation system using computer vision and deep learning.
Unlike word-level systems, it targets sentence-level translation and incorporates facial expressions and body movements for more natural communication. A custom SLSL dataset is built by annotating interpreter videos from Sri Lankan parliament broadcasts, and Transformer-based models learn the mapping from sign sequences to spoken output — running efficiently in real time.
Methodology & Workflow
Recognition pipeline
Sign Video Input
Data Preprocessing
Frame extraction
Feature Extraction
Hand / Pose / Face · MediaPipe + OpenCV
Deep Learning Model
CNN + Encoder–Decoder
Text Output
Text-to-Speech
Voice Output
Research Team
Ruwanthi Kalpana Liyanage
