Sign Language To Multilingual Speech

Abstract

Communication barriers faced by individuals who rely on sign language often limit their ability to interact with a broader audience. This project, Sign Language to Speech Conversion using Machine Learning, addresses this challenge by developing a system capable of recognizing American Sign Language (ASL) gestures and translating them into spoken language in real time.

The system utilizes a custom dataset comprising gestures for all 26 alphabets (A-Z), 10 digits (0-9), and special gestures for space and full stop, enabling the formation of words and sentences. Key hand landmarks are detected using MediaPipe, and the extracted features are used to train a machine learning model. Among various algorithms tested, the Random Forest Classifier demonstrated high accuracy and robustness, making it the algorithm of choice for this project.

To enhance system usability, a graphical user interface (GUI) was developed to display the recognized text while a text-to-speech (TTS) engine converts it into audible output. The real-time recognition process is further stabilized using buffer techniques to minimize misclassification. Preprocessing techniques such as feature normalization and hyperparameter optimization were employed to improve model performance.

The system's performance was evaluated using metrics like accuracy, precision, and recall, achieving reliable recognition rates for both static gestures and transitions between them. By bridging the gap between sign language users and non-sign language users, this project highlights the  potential. of machine learning in improving accessibility and inclusivity. Future enhancements will include support for dynamic gestures, integration of additional sign languages, and deployment on mobile platforms, expanding the reach and impact of this innovative solution.