{"id":12602,"date":"2026-06-20T15:52:11","date_gmt":"2026-06-20T10:22:11","guid":{"rendered":"https:\/\/eng.jfn.ac.lk\/mlsp\/?page_id=12602"},"modified":"2026-06-20T16:31:40","modified_gmt":"2026-06-20T11:01:40","slug":"sign-language-to-speech","status":"publish","type":"page","link":"https:\/\/eng.jfn.ac.lk\/mlsp\/?page_id=12602","title":{"rendered":"Personalized Sign Language to Speech for Assistive Communication"},"content":{"rendered":"
Research Project · MLSP<\/span><\/p>\n

Personalized Sign Language to Speech for Assistive Communication<\/h1>\n

Real-time Sri Lankan Sign Language to Sinhala speech translation<\/p>\n<\/div>\n

\n
\n

Overview<\/h2>\n

Communication barriers remain a major challenge for people with hearing and speech impairments, especially where localized assistive tools are scarce. This project builds a real-time Sri Lankan Sign Language (SLSL) → Sinhala speech<\/strong> translation system using computer vision and deep learning.<\/p>\n

Unlike word-level systems, it targets sentence-level translation<\/strong> and incorporates facial expressions and body movements for more natural communication. A custom SLSL dataset is built by annotating interpreter videos from Sri Lankan parliament broadcasts, and Transformer-based<\/strong> models learn the mapping from sign sequences to spoken output — running efficiently in real time.<\/p>\n<\/section>\n

\n

Methodology & Workflow<\/h2>\n
\"Gesture
Gesture Recognition Methodology — from sign-language video to Sinhala voice output.<\/figcaption><\/figure>\n

Recognition pipeline<\/p>\n

\n
\n
1<\/div>\n

Sign Video Input<\/h4>\n<\/div>\n
\n
2<\/div>\n

Data Preprocessing<\/h4>\n

Frame extraction<\/span><\/div>\n

\n
3<\/div>\n

Feature Extraction<\/h4>\n

Hand \/ Pose \/ Face · MediaPipe + OpenCV<\/span><\/div>\n

\n
4<\/div>\n

Deep Learning Model<\/h4>\n

CNN + Encoder–Decoder<\/span><\/div>\n

\n
5<\/div>\n

Text Output<\/h4>\n<\/div>\n
\n
6<\/div>\n

Text-to-Speech<\/h4>\n<\/div>\n
\n
7<\/div>\n

Voice Output<\/h4>\n<\/div>\n<\/div>\n<\/section>\n
\n

Research Team<\/h2>\n
\n
\"Ruwanthi<\/p>\n

Ruwanthi Kalpana Liyanage<\/h4>\n
Undergraduate Researcher<\/div>\n
MLSP Research Group<\/div>\n<\/div>\n
\"Sithara<\/p>\n

Sithara Weerasekara<\/h4>\n
Undergraduate Researcher<\/div>\n
MLSP Research Group<\/div>\n<\/div>\n<\/div>\n<\/section>\n
← Back to Projects<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Research Project · MLSP Personalized Sign Language to Speech for Assistive Communication Real-time Sri Lankan Sign Language to Sinhala speech translation Overview Communication barriers remain a major challenge for people with hearing and speech impairments, especially where localized assistive tools are scarce. This project builds a real-time Sri Lankan Sign Language (SLSL) → Sinhala speech translation system using computer vision and deep learning. Unlike word-level systems, it targets sentence-level translation and incorporates facial expressions and body movements for more natural communication. A custom SLSL dataset is built by annotating interpreter videos from Sri Lankan parliament broadcasts, and Transformer-based models learn the mapping from sign sequences to spoken output — running efficiently in real time. Methodology & Workflow Gesture Recognition Methodology — from sign-language video to Sinhala voice output. Recognition pipeline 1 Sign Video Input 2 Data Preprocessing Frame extraction 3 Feature Extraction Hand \/ Pose \/ Face · MediaPipe + OpenCV 4 Deep Learning Model CNN + Encoder–Decoder 5 Text Output 6 Text-to-Speech 7 Voice Output Research Team Ruwanthi Kalpana Liyanage Undergraduate Researcher MLSP Research Group Sithara Weerasekara Undergraduate Researcher MLSP Research Group ← Back to Projects<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/pages\/12602"}],"collection":[{"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12602"}],"version-history":[{"count":1,"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/pages\/12602\/revisions"}],"predecessor-version":[{"id":12615,"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=\/wp\/v2\/pages\/12602\/revisions\/12615"}],"wp:attachment":[{"href":"https:\/\/eng.jfn.ac.lk\/mlsp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}