In a research cooperation with the US technology company Meta, Technische Universität Ilmenau is investigating acoustic properties of rooms. In the project "Isoperare - Isoperceptive domains in room acoustics", the scientists are investigating how virtual sound elements can be integrated into real rooms without a significant difference to people and real objects being audible on site. With this basic research, TU Ilmenau is making an important contribution to the further development of future-oriented applications for the metaverse, such as novel teleconferencing systems or virtual classrooms.
Whether in teleteaching, video conferencing or social interaction - communication technologies are becoming increasingly important in our everyday digital lives. Already today, virtual elements in the form of avatars can be projected into real space using virtual reality and mixed reality (VR and MR) to facilitate communication between people in different locations. To ensure that users experience these interactions as true-to-life as possible not only visually but also acoustically, modern audio technologies are used, which are researched and developed at the Electronic Media Technology Group. Worldwide, the know-how of Ilmenau's scientists is in great demand, because only a few research institutions are concerned with the focus of the group: spatial audio recording and audio playback for use in VR and MR. The focus of the investigations is spatial auditory perception for the generation of so-called highly plausible auditory illusions in real space - i.e. a lifelike reproduction of spatial sound via headphones.
Hearing tests with test persons
With a joint research cooperation between TU Ilmenau and Meta (formerly Facebook), scientists Dr. Stephan Werner, acting head of the Electronic Media Technology Group, and Dr. Florian Klein, research associate at the same group, want to find out how the characteristics of a room affect the sound quality of an audio object in terms of its authenticity. To do this, they will first investigate the characteristics of rooms in the listening lab at the Institute of Media Technology. With the help of a robot, extensive measurements are taken at various locations in the listening lab. In addition, the researchers conduct listening tests with test persons. Headphones are used to generate three-dimensional sound fields that simulate the acoustics of a room. In this way, the test subjects can evaluate how authentically an audio source sounds when parameters are changed, such as different distances from the listener to the object. Based on this data, the scientists can draw conclusions about how audio objects should be optimally positioned in rooms and how the human brain perceives and processes acoustics in space. The research results form the basis for a variety of applications in VR and MR. In school teaching, for example, they can be used to establish virtual classrooms. The research also makes fundamental contributions to the acoustic realization and further development of the so-called Metaverse, a mixed reality environment for social interactions.
With the Isoperare project, the Electronic Media Technology Group has set itself the goal of being able to reproduce audio elements in virtual spaces as authentically and lifelike as possible, as Dr. Stephan Werner explains:
Our goal is to identify so-called iso-perceptive areas, i.e. acoustic areas in a room that evoke a comparable auditory perception. These areas will be used to predict the audibility of room acoustic changes in order to minimize effects that may expose a virtual acoustic object as virtual.
Research at the I3TC
In addition to the Isoperare project, scientists from various disciplines at TU Ilmenau are currently researching technical solutions to be used in digital communication. In the Co-HUMANICS project, researchers from five disciplines are working together in an interdisciplinary manner to enable older people to contact familiar people in their home environment who are far away from their location. Social co-presence can be realized through augmented and mixed reality techniques and robot-based telepresence, in which spatially distant individuals are virtually present in a person's real-world environment. The five-year project is funded by the Carl Zeiss Foundation.
In the project "Audiovisual Plausibility and Experience in Mixed Reality with Multiple Participants" (APlausE-MR), researchers from TU Ilmenau and Bauhaus Universität Weimar are investigating human perception, cognition and social interaction in distributed mixed reality communication scenarios with multiple participating parties. The goal of the project is to gain deeper insights into the factors that influence plausibility and quality during the shared experience of realistic interactive virtual environments (IVE).
This and further work and projects will be bundled in the newly founded "Ilmenau Interactive Immersive Technologies Center" (I3TC).
Key data
Duration: June 1, 2021 to February 30, 2024
Funded within the framework of a research cooperation with Meta Reality Labs
Project partners: Electronic Media Technology, Meta
Project team members: Dr. Stephan Werner, Dr. Florian Klein
Contact
Dr. Stephan Werner
Acting Head of Electronic Media Technology Group