Source: Kühlem et al., 2025
Whether in digital learning spaces or virtual museum tours, virtual reality (VR) makes an important contribution to immersive learning. In this context, users can interact with virtual teachers. At RWTH Aachen University, research is being conducted into how these conversations can be made as realistic and natural as possible.
Virtual Teachers in Digital Environments
Embodied conversational agents (ECAs), i.e., computer-controlled virtual humans, are an important component of many VR applications. These include, for example, interactive learning environments in which users can communicate with a virtual teacher, or—as in our example—museum tours in which a digital guide provides information about the various exhibits and answers any questions. In these scenarios, it is crucial that a fluid and natural conversation takes place between the ECAs and the VR users. To achieve this, the ECA must be able to understand the users’ questions and statements, think about appropriate answers, and then communicate these both verbally and non-verbally.
Conversations Between Users and Virtual Teachers
In recent years, much has happened in the area of “thinking” through the use of large language models (LLMs) such as ChatGPT. Instead of a limited number of pre-programmed responses, ECAs can now respond individually to users’ questions and needs through the use of AI. The AI analyzes the user’s questions and generates an appropriate and context-related response. Finally, this response is converted into spoken language, appropriate lip movements are generated for the ECA, and the response is played back in VR. In this way, the use of AI makes it possible to respond meaningfully to almost all queries and turn ECAs into plausible conversation partners.
Minimal Response Time for Natural Conversation Flow
For the most natural conversation flow possible, it is important that the response times of ECAs are as short as possible. This means that the ECA should respond to the VR user’s request or statement as quickly as possible and without significant delay. Gestures, emotions, and eye contact can minimize the perceived response time by allowing the ECA to simulate human-like “thinking.” This can be done both verbally, through filler words and sounds, and non-verbally, for example through gestures and body language.
Research at RWTH Aachen University
Source: Kühlem et al., 2025
At RWTH Aachen University, the Virtual Reality and Immersive Visualization research group is investigating communication between humans and ECAs. Its research uses a virtual museum tour as an example, in which a virtual guide accompanies users through the museum and responds to individual questions. In this scenario, it is possible to achieve realistic interaction between humans and ECAs.
Among other things, the aim is to investigate how the response time of ECAs can be reduced technically. An initial demo was already presented at ACM IVA 2025 in Berlin. The results will also be presented at IEEE AIxVR in Osaka, Japan, in January 2026.
Responsible for the content of this article are Andrea Bönsch, Hedda Faber, and Konstantin W. Kühlem.
Leave a Reply