Prosodic and visual naturalness of dialogs presented by conversational virtual agents

Tag / Zeit: 16.08.2021, 16:00-16:40
Raum: Schubert 3
Typ: Poster
Online-Zugang: Bitte loggen Sie sich ein, damit weitere Inhalte sichtbar werden (bspw. der Zugang zur Onlinesitzung).
Information: Die Poster sind von Montag morgen bis Mittwoch nachmittag in der Mall bzw. hier als PDF im jeweiligen Posterbeitrag einsehbar. Das Posterforum zu diesen Postern findet am Montag von 16:00 - 16:40 Uhr im hier angegebenen Saal statt. Für weiterführende Diskussion verabreden Sie sich bitte mit der/dem jeweiligen Autor(in) am Poster oder nutzen Sie die Chatfunktion im virtuellen Posterausstellungsraum. Dieser steht bis Montag ca. 18:00 Uhr zur Verfügung.
Zusammenfassung: Conversational virtual agents, with and without visual representation, are becoming more present in our daily life, e.g. as intelligent virtual assistants on smart devices. To investigate the naturalness of both the speech and the nonverbal behavior of embodied conversational agents (ECAs), an interdisciplinary research group was initiated, consisting of phoneticians, computer scientists, and acoustic engineers. For a web-based pilot experiment, simple dialogs between a male and a female speaker were created, with three prosodic conditions. For condition 1, the dialog was created synthetically using a text-to-speech engine. In the other two prosodic conditions (2,3) human speakers were recorded with 2) the erroneous accentuation of the text-to-speech synthesis of condition 1, and 3) with a natural accentuation. Face tracking data of the recorded speakers was additionally obtained and applied as input data for the facial animation of the ECAs. Based on the recorded data, auralizations in a virtual acoustic environment were generated and presented as binaural signals to the participants either in combination with the visual representation of the ECAs as short videos or without any visual feedback. A preliminary evaluation of the participants’ responses to questions related to naturalness, presence, and preference is presented in this work.