What acoustic properties do rooms have? And how can virtual sound elements be integrated as plausibly as possible into real spaces and merged with each other so that people in different locations experience their virtual communication as realistically as possible? Researchers and students of Media Technology/Research in Media Engineering at the Electronic Media Technology Group at TU Ilmenau, led by Dr. Stephan Werner, are working on these questions relating to mixed reality (MR). Their goal: to offer a listening experience that comes as close as possible to a real scenario. As part of the "Isoperare - Isoperceptive areas in room acoustics" project, a research collaboration with Reality Labs at Meta, they have developed a system that measures room acoustics automatically and with high precision. Among other things, it can be used to further develop future-oriented applications for the Metaverse such as innovative teleconferencing systems or virtual assistance systems.
Authentic acoustics in virtual worlds: Scientists develop mobile robot for high-precision room acoustic measurements
Simplified acoustic simulations are generally used to recreate the acoustics of a real room for mixed reality applications. Machine learning can also be used to estimate the acoustics of a room by evaluating audio data or using optical data such as photos, videos or 3D scans to create a virtual model of the room and its acoustic properties. "However, all of these approaches can only approximate the real room acoustics," explains Lukas Treybig, research assistant and doctoral student at the department:
In order to better understand virtual acoustic realities, we first capture the real room acoustics in high spatial resolution. This then enables us to create better adapted simulations. If the virtual acoustics differ significantly from the real acoustics, the virtual audio objects are not perceived in the room, but in the head.
However, high-precision measurements at many positions in the room are complex to create, generate large amounts of data and are prone to errors. "It is therefore practically impossible to carry out such measurements manually," says Treybig.
Automated acoustic measurements at any position in the room
With the support of MetraLabs GmbH, the Electronic Media Technology Group has therefore developed a system that can autonomously carry out acoustic measurements in different rooms. The research team led by Lukas Treybig, Georg Stolz and Florian Klein has equipped a mobile robot platform with a microphone array: A total of seven microphones make it possible to estimate how the sound is reflected in the room on its way from a sound source to the microphone array. To make this possible for any position in the room, the microphone array is mounted on a robot platform that can navigate independently through the room and carry out measurements.
In further investigations, the research group at the I3TC - Ilmenau Interactive Immersive Technologies Center is now focusing in particular on how the data obtained can be evaluated and simplified in order to reproduce acoustics in various applications in a realistic way. The new measurement system is already being used in the Multiparties project, among others, to model auditory perception and spatial audio reproduction. The aim of the project led by Brandenburg Labs GmbH, which is funded by the federal government as part of the "KMU-innovativ: Interactive Technologies for Health and Quality of Life" program, is to develop innovative, interactive technologies for collaborative telepresence systems in augmented reality. By combining realistic augmented reality avatars with expressive gestures and facial expressions with spatial audio, the aim is to enable realistic online meetings between several people in shared communication spaces. A total of three research groups are involved in the project at TU Ilmenau, including the Virtual Worlds and Digital Games Group and the Audiovisual Technology Group.
More information and publications on the acoustic measuring robot
Kontakt
Dr. Stephan Werner
Head of Electronic Media Technology