Scientists from Technische Universität Ilmenau and RWTH Aachen University offered a glimpse into the future of human-centered communication at the conclusion of the research project “CO-HUMANICS – Co-Presence between People and Interactive Companions for Seniors” in Ilmenau. In the laboratories of the Ilmenau Interactive Immersive Technologies Center (I3TC) and the Thuringian Center for Learning Systems and Robotics, the interdisciplinary research team presented the results of the five-year project: technical foundations for novel, user-centered communication systems that represent geographically distant people so realistically that it feels as if they were in the same room. Such systems could fundamentally change the way people stay in touch with one another in an era of individualization, physical separation, and an aging society.
Social Closeness Despite Distance: Digital Companions for Greater Connection in Old Age
Successful completion of a five-year research project funded by the Carl Zeiss Foundation
Elderly people, in particular, rely on social connections to stay active, healthy, and happy. But when loved ones pass away, move away, or health issues make in-person meetings difficult, direct contact becomes more challenging – for example, with a grandchild studying far away, a friend back in one’s hometown, or a daughter in Australia.
To foster social closeness even across great distances, five research groups at TU Ilmenau pooled their expertise in the CO-HUMANICS project: Audiovisual Technology, Electronic Media Technology, Media Psychology and Media Design, Neuroinformatics and Cognitive Robotics, and Virtual Worlds and Digital Games.
In collaboration with the Institute of Communications Engineering at RWTH Aachen University, the CO-HUMANICS team explored some fundamental questions: How can technology bring people – and seniors in particular – even closer to their families and friends? How can we create the feeling that a person who is physically far away is actually sitting at the table with me, shaking my hand, or playing a game with me? What role can telepresence robots and virtual avatars in augmented reality (AR) play in this? And what technical and psychological conditions ensure that people experience this form of communication via augmented reality (AR) and robotics as natural and trustworthy?
The team was supported by an advisory board of experts from the Federal Association of Senior Citizens’ Organizations (BAGSO), the University of Applied Sciences Northwestern Switzerland in Zurich, the Honda Research Institute Europe in Offenbach, the Bauhaus University Weimar, and the Carl von Ossietzky University of Oldenburg. The research project was funded by the Carl Zeiss Foundation, which creates space for research.
Precise interaction of numerous technical components
Within five years, the research team developed two functional prototypes: an augmented reality system and a robotic platform. Both are designed to support communication and shared interaction between two or more people who are physically separated, giving them the feeling of being close to one another.
With the help of the AR prototype, virtual representations of friends and family members can be superimposed into the home environment of older adults. This creates the impression that the geographically separated individuals are together in one place – a sense of virtual co-presence.
Prof. Wolfgang Broll, head of the Virtual Worlds and Digital Games Group and director of the I3TC, explains:
The method we developed was not just about using augmented reality to integrate distant people directly and seamlessly into the home environment. It was also crucial to ensure that their appearance and movements were as lifelike as possible. Only when the virtual people blend credibly into their surroundings can a sense of closeness and togetherness emerge.
A credible virtual encounter also requires realistic spatial sound reproduction. Scientists from the Electronic Media Technology Group therefore developed and investigated methods for simulating sound sources in space in a way that is both realistic and efficient. They also researched the influence of sound and its spatial localization on the perception of virtual avatars.
For the robot platform developed in the project, numerous technical components also had to work together precisely so that the robot could navigate safely through a living space, recognize people, and follow them. Subproject leader Prof. Horst-Michael Groß from the Neuroinformatics and Cognitive Robotics Group at TU Ilmenau explains:
In the field of robotics, this includes so-called “search and follow” skills based on person tracking, as well as spatial perception and representation through panoptic segmentation and 3D NDT maps. In addition, there is a projection system for visualizing planned paths and detected obstacles or for pointing at objects, a shared web-based interface for communication, and gesture control for simple commands.
"Solutions that really work in everyday life"
In doing so, the researchers also incorporated the wishes and needs of future users directly into the development process and explored the potential for collaborative activities in extended reality.
About 30 seniors aged 68 to 83 from the Ilmenau and Erfurt regions regularly participated in tests. They visited the researchers in their labs, tested AR applications and telepresence robots, provided critical feedback on the quality of communication, and actively helped shape the technologies.
By developing methods to evaluate the quality of virtual communication and directly incorporating these user experiences into our research, we were able to develop technological innovations that are truly intended to work in everyday life in the future
explains Prof. Alexander Raake from the Institute of Communications Engineering at RWTH Aachen University.
All participating seniors reported a strong sense of community generated by the various scenarios, according to project leader Prof. Nicola Döring from the Media Psychology and Media Design Group:
The seniors really enjoyed using the technology. Once they realized, after some initial uncertainty, that they could handle it, they became very interested – and in some cases so enthusiastic that they would have loved to keep going even after the study ended. Many had never experienced virtual reality before. But many also found it fascinating to remotely control a robot and, for example, ask it to find an object in the room. Many could also imagine using similar applications themselves in the future – depending on their life situation. The point is not to use as much technology as possible, but to make it available where it truly provides a benefit.
Expertise in human-robot interaction and human-human communication
The research team presented its findings in more than 30 scientific publications at international conferences in Ireland, Japan, and the United States, among other places. In addition, several doctoral dissertations were completed. Prof. Raake explains:
This expertise will now be further expanded through new collaborations and larger follow-up projects. The goal is to make technological solutions even more user-friendly and emotionally engaging – for example, through even more realistic, personalized avatars.
During their final presentation at TU Ilmenau, the researchers brought their prototypes to life through interactive demonstrations: Two people, located in different labs within the Grace Hopper Building, used a synchronized augmented reality application to build a virtual house together out of LEGO bricks. In addition, Robby the robot established a video connection with a person at a remote location using natural language, helped find a glasses case, and responded to simple control gestures to navigate through the apartment.
“With these innovations, we’re not only opening up new avenues for interaction and community for older adults, but by connecting people and machines, we can also improve social participation overall,” Prof. Groß concludes.
The CO-HUMANICS project was funded by the Carl Zeiss Foundation as part of the “Breakthroughs 2020” grant program.
Contact
Prof. Nicola Döring
Head of Media Psychology and Media Design
Highlights from the "Co-Humanics" Final Presentation
About the Carl-Zeiss-Stiftung
The Carl-Zeiss-Stiftung funds science and teaching in the STEM disciplines of Science, Technology, Engineering, Mathematics. As a partner to outstanding science, it provides funding opportunities for basic and applied research. The foundation was established in 1889 by Ernst Abbe in Jena and is today one of the large private research funding organizations in Germany. It is the sole owner of Carl Zeiss AG and SCHOTT AG. The funding activities are made possible by dividends from the two foundation companies.