Interactive multimedia solutions for improving accessibility of deaf and disabled people

Interactive multimedia and visual communication systems continue to evolve to meet various needs of users, making the best use of cutting-edge technologies. These technologies have great potential to better include deaf, disabled and functionally illiterate people and provide the support to hard of hearing students in academic settings and social routines. The lack of skilled sign language interpreters and captioners with advanced domain knowledge along with language barriers make this communication challenging. Furthermore, extensive use of textual information on web means that illiterate and functionally illiterate people cannot access services and information provided. Consequently, most of the applications and web pages developed these days pose an accessibility barrier for such people. These applications are increasingly important in one's day to day life whether it’s online employment application, utilizing e-government services, seeking medical information, or simply for awareness purposes. This dissertation describes technological solutions to facilitate the integration of deaf people in the society. Interactive multimedia solutions are designed, implemented, and are evaluated to provide assistance and support to the hard of hearing students in sign language learning. These will help them to communicate and interact with people using other languages. The machine translation system using various methodologies and representation forms are illustrated. The research on text to sign language translation, with the focus on Italian text into Italian sign language translation system developed in the reported work. The sign language signs are represented using virtual character based animations. The analysis of different applications and implementations where human-computer relationships are advantageous to the users is also provided. Human-computer relationships are also important where an increasing number of illiterate and semi-literate people have to interact with the applications for everyday needs. However, they are barred due to accessibility and understandability. The additional support on the web interfaces can make written and multimedia contents accessible to the functionally illiterate people by easing of use functionalities. The usability and accessibility issues of semi literate users and user study on interfaces designed and developed for semi literates are also discussed. The virtual character based interactive interfaces are introduced which concentrates on localization of interfaces by providing verbal help in the regional language through life-like characters. A comparative study of virtual character based interactive interface, audio interface and original (textual) interface has been conducted. The purpose of evaluating different interfaces is to enhance the e-participation of low literate users and motivate them use ICT in the developing countries. The research revealed that the performance and accessibility of functionally illiterate people is increased with the additional support. These animations enhance accessibility and understandability of people to the textual information. The results show that the performance of semi-literate users improved significantly when using the online assistance. Moreover, the percentage of completed tasks increased from 52% to 94%.