The Technische Universität Ilmenau has started the research project "MULTIPARTIES - Multi-Party Augmented Reality Telepresence System" that enables realistic online meetings between several people over distances. The two-and-a-half-year collaborative project, led by Brandenburg Labs GmbH, is funded by the federal government as part of "KMU-innovativ: Interactive Technologies for Health and Quality of Life."
Online meetings are still very different from real meetings. This was demonstrated not least by the Corona pandemic. Important interpersonal aspects of communication, such as body language or the ability to whisper something to the other person, are lost. Online meetings also lead to a higher cognitive load than real meetings. The goal of MULTIPARTIES is to overcome these limitations and make it possible for people who are physically separated to have more natural conversations online.
With the help of augmented reality (AR), i.e. thecomputer-assisted augmentation of reality perception, and spatial audio technologies, people participating in a MULTIPARTIES meeting will perceive, communicate and interact with each other as naturally as possible. To achieve this, realistic 3D avatars of the people with expressive gestures and facial expressions as well as spatially plausible audio are seamlessly integrated into the real environment. This creates the impression of actually meeting with the other conversation partners. Karlheinz Brandenburg, CEO of Brandenburg Labs GmbH, sees AR telepresence systems as the future of online conferencing: "The goal of such systems is by no means to replace real interpersonal contact. Rather, we want to facilitate digital learning as well as work processes, promote togetherness, and create more diverse opportunities for social participation."
In addition to the TU Ilmenau and Brandenburg Labs GmbH, partners in the joint project include plazz AG, a company from Erfurt that developsintegrated solutions for events, and Consensive GmbH from Weimar, which develops software applications that facilitate social exchange.On the TU Ilmenau side, three groups are contributing their expertise to the MULTIPARTIES project. The Virtual Worlds and Digital Games Group is responsible for researching AR technologies and interaction techniques. Head Prof. Wolfgang Broll: "A major challenge lies in capturing and reproducing individual facial expressions and gestures to enable realistic and expressive avatars. In addition, we will explore new interaction techniques for controlling the application as well as for collaboration between participants." Prof. Alexander Raake, head of the Audiovisual Technology Group, is devoting the MULTIPARTIES project to the questions of which factors contribute to communication that is as natural as possible, and how this can be measured. The focus is primarily on hearing, seeing, speaking and communication through facial expressions and gestures. And Dr. Stephan Werner, head of the Electronic Media Technology Group , is conducting fundamental research into the recognition and assessment of relevant and irrelevant acoustic information and developing the audio reproduction of the new online technology.
Brandenburg Labs GmbH, coordinator of the project, is dedicated to spatial audio playback via headphones integrated into the system. They enable a natural reproduction of sound that allows participants in the virtual meeting to talk to each other at the same time, for example, or to "whisper" something in other people's ears. Jürgen Mayer, CEO of plazz AG, expects valuable insights for the event sector: "With our event and community platform Polario, we naturally connect a large number of people in different environments and scenarios. With MULTIPARTIES, we enable users to create deeper connections with each other and we enhance digital work environments. In-house and external events, which are organized virtually or hybrid in a sustainable way, especially due to the Corona pandemic, but also climate-friendly online marketing and sales meetings will benefit from MULTIPARTIES." The Weimar-based start-up consensive GmbH brings its expertise for 3D representation and interaction of participants in mixed reality. An important starting point of consensive is the use of so-called digital twins. This involves digitally recreating people's real environments and even recombining them to create a shared space.
Contact
