Assistive Interaction Using Gestures and Voice Commands

Abstract

Now, thanks to advancements in artificial
intelligence and speech recognition technology,
even those with limited mobility can use
computers, laptops, and cellphones. Users are
able to do a wide variety of tasks, including as
launching apps, sending emails, checking the
time and weather, visiting websites, and
powering down their devices, all with voice
commands. Their conversations are transcribed
verbatim and then searched for relevant terms
to complete the necessary actions. Paralyzed
individuals who are unable to leave their chairs
may still use PCs and laptops by pointing and
clicking with their hands. The mouse may be
controlled by the user's hand motions detected
via the use of a variety of hand gestures. Our
software uses a camera to record the user's
hand movements and gestures, then simulates
mouse clicks in response to those movements.
This is possible with the use of contour analysis,
feature extraction, and hand tracking.