Beyond looks: a study on agent movement and audiovisual spatial coherence in augmented reality. - In: IEEE Xplore digital library, ISSN 2473-2001, (2024), S. 502-512
The appearance of virtual humans (avatars and agents) has been widely explored in immersive environments. However, virtual humans’ movements and associated sounds in real-world interactions, particularly in Augmented Reality (AR), are yet to be explored. In this paper, we investigate the influence of three distinct movement patterns (circle, side-to-side, and standing), two rendering styles (realistic and cartoon), and two types of audio (spatial audio and non-spatial audio) on emotional responses, social presence, appearance and behavior plausibility, audiovisual coherence, and auditory plausibility. To enable that, we conducted a study (N=36) where participants observed an agent reciting a short fictional story. Our results indicate an effect of the rendering style and the type of movement on the subjective perception of the agents behaving in an AR environment. Participants reported higher levels of excitement when they observed the realistic agent moving in a circle compared to the cartoon agent or the other two movement patterns. Moreover, we found an influence of agent’s movement pattern on social presence and higher appearance and behavior plausibility for the realistic rendering style. Regarding audiovisual spatial coherence, we found an influence of rendering style and type of audio only for the cartoon agent. Additionally, the spatial audio was perceived as more plausible than non-spatial audio. Our findings suggest that aligning realistic rendering styles with realistic auditory experiences may not be necessary for 1-1 listening experiences with moving sources. However, movement patterns of agents influence excitement and social presence in passive unidirectional communication scenarios.
https://doi.org/10.1109/VR58804.2024.00071
Age and realism of avatars in simulated augmented reality: experimental evaluation of anticipated user experience. - In: IEEE Xplore digital library, ISSN 2473-2001, (2024), S. 83-93
Augmented reality (AR) presents vivid opportunities for interpersonal communication. With the growing diversity of social AR users, understanding their unique needs and perceptions becomes crucial. This study delves into how younger, middle-aged, and older adults perceive avatars with different aging attributes and degrees of realism, focusing on their anticipated user experience within a social AR system. We conducted an online within-subjects experiment involving N=2086 age-diverse participants from Germany who assessed a set of nine gender-matched avatars for their perceived social attractiveness (research question 1 = RQ1) and the likelihood of selecting these avatars for self-representation in social AR (RQ2). The evaluated avatars represented different age groups (younger, middle-aged, and older) and levels of realism (low, medium, and high). We validated both the created avatars and our experimental setup and employed a linear mixed-effects modeling approach to analyze the data. Our findings unveiled a strong preference for younger high-realism avatars as communication partners (RQ1), which was consistent across all participant age groups. Similarly, participants favored younger high-realism avatars for self-representation in social AR (RQ2). However, older adults were more inclined to opt for avatars resembling their actual age. The study highlights the prevalence of age-related stereotypes in avatar-based communication. Similar to face-to-face social interactions, these stereotypes tend to render older avatars less socially attractive than their younger counterparts, irrespective of the avatar’s degree of realism. Our results invite considerations on how to combat these stereotypes through a more thoughtful and inclusive avatar design process that encompasses a broader spectrum of aging attributes.
https://doi.org/10.1109/VR58804.2024.00032
Towards augmented and mixed reality on future mobile networks. - In: Multimedia tools and applications, ISSN 1573-7721, Bd. 83 (2024), 3, S. 9067-9102
Augmented and Mixed Reality (AR/MR) technologies enhance the human perception of the world by combining virtual and real environments. With the increase of mobile devices and the advent of 5G, this technology has the potential to become part of people’s life. This article aims to evaluate the impact of 5G and beyond mobile networks in the future of AR/MR. To attend to this objective, we surveyed four digital libraries to identify articles and reviews concerning AR/MR use based on mobile networks. The results describe the state-of-the-art of mobile AR/MR applications and the benefits and challenges of the technology. Finally, after the review, we propose a roadmap concerning AR/MR hardware and software development to run applications supported by future mobile networks.
https://doi.org/10.1007/s11042-023-15301-4
Digital media in intergenerational communication: status quo and future scenarios for the grandparent-grandchild relationship. - In: Universal access in the information society, ISSN 1615-5297, Bd. 23 (2024), 1, S. 379-394
Communication technologies play an important role in maintaining the grandparent-grandchild (GP-GC) relationship. Based on Media Richness Theory, this study investigates the frequency of use (RQ1) and perceived quality (RQ2) of established media as well as the potential use of selected innovative media (RQ3) in GP-GC relationships with a particular focus on digital media. A cross-sectional online survey and vignette experiment were conducted in February 2021 among N = 286 university students in Germany (mean age 23 years, 57% female) who reported on the direct and mediated communication with their grandparents. In addition to face-to-face interactions, non-digital and digital established media (such as telephone, texting, video conferencing) and innovative digital media, namely augmented reality (AR)-based and social robot-based communication technologies, were covered. Face-to-face and phone communication occurred most frequently in GP-GC relationships: 85% of participants reported them taking place at least a few times per year (RQ1). Non-digital established media were associated with higher perceived communication quality than digital established media (RQ2). Innovative digital media received less favorable quality evaluations than established media. Participants expressed doubts regarding the technology competence of their grandparents, but still met innovative media with high expectations regarding improved communication quality (RQ3). Richer media, such as video conferencing or AR, do not automatically lead to better perceived communication quality, while leaner media, such as letters or text messages, can provide rich communication experiences. More research is needed to fully understand and systematically improve the utility, usability, and joy of use of different digital communication technologies employed in GP-GC relationships.
https://doi.org/10.1007/s10209-022-00957-w
Bridging distance with a collaborative telepresence robot for older adults - report on progress in the CO-HUMANICS project. - In: ISR Europe 2023: 56th International Symposium on Robotics, (2023), S. 346-353
In an aging society, the social needs of older adults, such as regular interactions and independent living, are crucial for their quality of life. However, due to spatial separation from their family and friends, it is difficult to maintain social relationships. Our multidisciplinary project, CO-HUMANICS, aims to meet these needs, even over long distances, through the utilization of innovative technologies, including a robot-based system. This paper presents the first prototype of our system, designed to connect family members or friends virtually present through a mobile robot with an older adult. The system incorporates bi-directional video telephony, remote control capabilities, and enhanced visualization methods. A comparison is made with other state-of-the-art robotic approaches, focusing on remote control capabilities. We provide details about the hard- and software components, e.g., a projector-based pointing unit for collaborative telepresence to assist in everyday tasks. Our comprehensive scene representation is discussed, which utilizes 3D NDT maps, enabling advanced remote navigation features, such as autonomously driving to a specific object. Finally, insights about past and concepts for future evaluation are provided to assess the developed system.
https://ieeexplore.ieee.org/document/10363093
Eye and face tracking in VR: avatar embodiment and enfacement with realistic and cartoon avatars. - In: MUM 2023, (2023), S. 270-278
Previous studies have explored the perception of various types of embodied avatars in immersive environments. However, the impact of eye and face tracking with personalized avatars is yet to be explored. In this paper, we investigate the impact of eye and face tracking on embodiment, enfacement, and the uncanny valley with four types of avatars using a VR-based mirroring task. We conducted a study (N=12) and created self-avatars with two rendering styles: a cartoon avatar (created in an avatar generator using a picture of the user’s face) and a photorealistic scanned avatar (created using a 3D scanner), each with and without eye and face tracking and respective adaptation of the mirror image. Our results indicate that adding eye and face tracking can be beneficial for certain enfacement scales (belonged), and we confirm that compared to a cartoon avatar, a scanned realistic avatar results in higher body ownership and increased enfacement (own face, belonging, mirror) - regardless of eye and face tracking. We critically discuss our experiences and outline the limitations of the applied hardware and software with respect to the provided level of control and the applicability for complex tasks such as displaying emotions. We synthesize these findings into a discussion about potential improvements for facial animation in VR and highlight the need for a better level of control, the integration of additional sensing and processing technologies, and an objective metric for comparing facial animation systems.
https://doi.org/10.1145/3626705.3627793
Exploring the use of social virtual reality conferences in higher education. - In: 2023 IEEE 2nd German Education Conference (GeCon), (2023), insges. 6 S.
Sparked by the recent growth of online and remote teaching formats, social virtual reality (Social VR) applications are being employed in higher education teaching. This eliminates the need for physical presence in one classroom and allows for increased accessibility of classes. More engaging distant virtual classrooms and extracurricular activities are made possible by Social VR applications like Mozilla Hubs (MH). We conducted a virtual conference in web-based MH as a cross-university collaboration for two game development courses for undergraduate and graduate students. We report on our organizational strategy and subsequent online survey evaluation of N = 29 attendees. We present solutions to problems that are frequently encountered, specifically in MH, while organizing virtual conferences.
https://doi.org/10.1109/GECon58119.2023.10295104
Zeitgebers, time judgments, and VR: a constructive replication study. - In: 2023 IEEE International Symposium on Mixed and Augmented Reality adjunct, (2023), S. 1-2
Previous research has attempted to understand the influence of virtual reality (VR) on human perception of time, but neither a comprehensive understanding nor conclusive results have been achieved. To extend and continue research on this topic, we closely replicated a previous study and included new elements in a constructive replication study. To do this, we replicate the original setup and investigate the influence of workload and sun speed on time production. Contrary to previous findings, we did not find significant differences in virtual sun movements on time judgments. However, consistent with the original study, time perception in VR was affected by cognitive workload. In addition, we found that immersion in the virtual environment influenced time perception after VR. The contrasting results highlight the need for further research into the factors contributing to time perception in and after VR to fully explain the phenomenon of altered perception.
https://doi.org/10.1109/ISMAR-Adjunct60411.2023.00007
Cube-SSIM: a metric for evaluating 360-degree images as cube maps. - In: IEEE Xplore digital library, ISSN 2473-2001, (2023), S. 248-251
360-degree image data is a crucial aspect in graphics applications where they are typically used for lighting purposes. Fields like mixed reality generally rely on lighting estimation techniques to estimate the 360-degree environment. To evaluate such approaches, accurate image assessment in this domain is important. However, traditional image evaluation metrics like SSIM, PSNR, and IMED are problematic when analyzing 360-degree image data as they would require two-dimensional representations like equirectangular panoramas or unfolded cubes that introduce image distortions. In this paper, we address this problem by presenting Cube-SSIM, a variant of SSIM designed specifically for cube maps. For this, we modify SSIM to take the solid angles of cube map pixels into account which gives more consistent results than using SSIM for the individual cube faces. The computations can run efficiently on graphics hardware due to their native support for cube maps and no further image conversions are required. We show that our approach allows for more accurate results than other comparison metrics that largely depend on 2D images. While SSIM is especially important due to its wide usage, the modification can also be applied to other image metrics for which we include IMED as an example.
https://doi.org/10.1109/CW58918.2023.00043
Evaluating light probe estimation techniques for mobile augmented reality. - In: IEEE Xplore digital library, ISSN 2473-2001, (2023), S. 141-148
Realistic lighting approaches typically rely on physically-based rendering which in turn often makes use of image-based lighting. Enabling these techniques in augmented reality on mobile devices requires unique approaches to estimate light probes, given the limited camera and sensor data available. In this paper, we evaluate different time-dependent and time-independent techniques for light probe estimation in augmented reality applications that try to predict the environment lighting using single images or video streams in combination with inpainting techniques. We simulate real-world applications using an evaluation framework where a simulated mobile device captures camera streams from different scenarios following a pre-defined path. The resulting camera streams are fed to a total of six estimation techniques in order to create light probes which are then used to render virtual objects while applying various materials. By comparing the rendered images as well as the light probe estimations, we perform a quantitative evaluation. We show how approaches that are able to process continuous video streams can provide more plausible results in cases where sufficient camera movement is present. Additionally, we investigate the visual impression of different types of materials showing that rough surfaces with distinct colors are less likely to produce divergent estimation results.
https://doi.org/10.1109/CW58918.2023.00029