The Technische Universität Ilmenau has developed an innovative audio technology for augmented and virtual reality that makes it possible to generate virtual sound sources in a natural environment. To a person moving in the room, the sound sounds realistic and convincing at all times: as they walk toward, past or around the sound source, they perceive the sound as if the source were actually in the room. The technology, which is already being developed with Meta Group, formerly Facebook, has huge market potential that extends far beyond entertainment and gaming.
Virtual, augmented and mixed reality applications, so-called immersive media, are increasingly finding their way into our digital world. Users are immersed in digital environments of virtual reality (VR), which they perceive as real thanks to acoustic and visual information, or reality is enhanced by additional acoustic and visual stimuli with the help of augmented reality (AR) or mixed reality. The sense of hearing plays a fundamental role in the three-dimensional experience of virtual spaces. Not only when a person listens motionlessly in them, but also when they walk toward, past or around a virtual sound source, their auditory perception must be realistic in all three dimensions at all times.
To achieve the most lifelike auditory perception of virtual sound sources, acoustics experts have so far made use of commercially available headphones around the world and adjusted the reproduction of sound sources located in the room according to the listener's head movements and rotations. When a listener changes position in a room, he or she not only changes listening position relative to the sound source itself, but also relative to any reflective surface, regardless of its shape, size, surface texture, material, distance, and direction. If all these acoustic details are to be taken into account in sound reproduction, this requires not only a gigantic computing power of computers, but also a huge measurement and modeling effort.
Thus, it is exceedingly time-consuming for a 3D room simulation software and for its operator,to render the acoustic surface properties of a complex room environment, i.e. to generate an image of the environment from the raw data of, say, a winding library with thousands of books, in order to calculate the properties of all the individual wall, shelf and book surfaces that are necessary for sound reproduction.
