A sketch-based interface for facial animation in immersive virtual reality

Creating facial animations using 3D computer graphics represents a very laborious and time-consuming task. Among the numberless approaches for animating faces, the use of blendshapes remains the most common solution because of their simplic- ity and the ability to produce high-quality results. This approach, however, is also characterized by important drawbacks. With the traditional animation suites, in order to select the blendshapes to be activated animators are generally requested to memorize the mapping between the blendshapes and the influenced mesh vertices; alternatively, they need to adopt a trial-and-error search within the whole library of available blendshapes. Moreover, the level of expressiveness that can be reached may be lower than expected; this is due to the fact that the possibility to apply transformations to vertices different than just linear translations and mechanisms for adding, e.g., exaggerations, are typically not integrated into the same anima- tion environment. In order to tackle these issues, this paper proposes an immersive virtual reality-based interface that leverages sketches for the direct manipulation of blendshapes. Animators can draw both linear and curved strokes, which are used to automatically extract information about the blendshape to be activated and its weight, the trajectories that associated vertices have to follow, as well as the timing of the overall animation. A user study was carried out with the aim of evaluating the proposed approach onto several representative animations tasks. Both objective and subjective measurements were collected. Experimental results showed the benefits of the devised interface in terms of task completion time, animation accuracy, and usability.