Improving visual speech synthesis using Decision Tree Models
Improving Visual Speech Synthesis using Decision Tree Models C.F. Rademan Department of Electrical and Electronic Engineering, University of Stellenbosch, Private Bag X1, Matieland 7602, South Africa. Thesis: MScEng (Elec) March 2016 Visual speech synthesis is essential for believable virtual character interaction. Traditionally, animation artists recreate the oral motions expected from speech utterances. In response, we present decision tree based clustering techniques which are employed in automating visual speech animation. This is achieved using a small dataset of phoneticallyannotated audiovisual speech. Our work focuses on extending existing tree based clustering algorithms by improving on the modelling of coarticulation effects. This is accomplished by capturing the motion of natural speech segments, referred to as dynamic visemes, and conserving their parameters during clustering and speech synthesis. Dynamic visemes are defined as the trajectories of oral features segmented by triphone boundaries. By applying simple search and concatenation criteria, our visual speech synthesis system uses decision trees to better predict which dynamic visemes to use. Experimentation guided all design decisions, suggesting which oral features were of greatest importance, identifying an appropriate dynamic viseme length and finding an effective interpolation method for conserving coarticulation. We evaluate the performance of our visual speech synthesis models by computing squared error differences between synthesised and measured feature trajectories. Perceptual tests also asked participants to compare virtual characters animated by model outputs. Both measured and perceptual tests show that our approaches lead to a clear improvement over a comparable baseline. Through our research, we intended on making speech synthesis more accessible. Therefore, the conversational agents are based on the freely available MakeHuman and Blender software components. The customised oral feature motion capture system is also easily reproduced and requires only consumer grade recording equipment. ii Stellenbosch University https:/scholar.sun.ac.za