Eye tracking and speech driven human-avatar emotion-based communication

Feelings, emotions, and empathy play an important role in many daily activities including verbal and non-verbal communications. Their automatic recognition and interpretation is important in a variety of applications requiring communication skills that are difficult to reproduce in computer-simulated environments, including those involving human-avatar interactions. Our recent work has begun investigating the development of intelligent avatars capable of detecting user (human) emotions to allow for realistic human-avatar interactions within medical-based virtual simulations and serious games. In this paper, we present a system that couples eye tracking and dialogue interpretation to allow for intelligent and realistic human-avatar communication. Although formal testing is required, preliminary results are promising, showing the potential of the system.