Modeling of an automatic vision mixer with human characteristics for multi-camera theater recordings. - In: IEEE access, ISSN 2169-3536, Bd. 11 (2023), S. 18714-18726
A production process using high-resolution cameras can be used for multi-camera recordings of theater performances or other stage performances. One approach to automate the generation of suitable image cuts could be to focus on speaker changes so that the person who is speaking is shown in the generated cut. However, these image cuts can appear static and robotic if they are set too precisely. Therefore, the characteristics and habits of professional vision mixers (persons who operate the vision mixing desk) during the editing process are investigated in more detail in order to incorporate them into an automation process. The characteristic features of five different vision mixers are examined, which were used under almost identical recording conditions for theatrical cuts in TV productions. The cuts are examined with regard to their temporal position in relation to pauses in speech, which take place during speaker changes on stage. It is shown that different professional vision mixers set the cuts individually differently before, in or after the pauses in speech. Measured are differences on average up to 0.3 seconds. From the analysis of the image cuts, an approach for a model is developed in which the individual characteristics of a vision mixer can be set. With the help of this novel model, a more human appearance can be given to otherwise exact and robotic cuts, when automating image cuts.
https://doi.org/10.1109/ACCESS.2023.3245804
Effects of binaural classroom noise scenarios on primary school children's speech perception and listening comprehension. - In: 51st International Congress and Exposition on Noise Control Engineering (INTER-NOISE 2022), (2023), S. 3214-3220
Assessment of visual quality and simulator sickness for omnidirectional videos. - Ilmenau, 2023. - viii, 186 Seiten
Technische Universität Ilmenau, Dissertation 2022
Ein Anwendungsfall für die aktuelle VR-Technologie mit Head Mounted Displays (HMDs) sind 360˚-Videos. Die valide Bewertung der Erlebnisqualität (Quality of Experience, QoE) für 360˚-Videos erfordert subjektive Tests. Solche Bewertungstests sind zeitaufwändig und erfordern ein gut durchdachtes Protokoll. Internationale Empfehlungen wie z. B. ITU-T Rec. P.910 und ITU-R Rec. BT.500-13 existieren, die Richtlinien für die Bewertung der Videoqualität von 2D-Videos auf 2D-Displays unter Einbeziehung von Testpersonen enthalten. Bis zu dieser Arbeit gab es jedoch keine solche Standardempfehlung für 360˚-Videos. Daher war es notwendig, eine Reihe von Richtlinien zu entwickeln, um die visuelle Qualität und die QoE-Bewertung für 360˚-Videos zu untersuchen. In dieser Arbeit werden umfangreiche Forschungsarbeiten zur Qualität und Lebensqualität von 360˚-Videos vorgestellt, die von Nutzern mit HMDs wahrgenommen werden, sowie eine Reihe von Testprotokollen für eine systematische Bewertung. Zunächst wurden konventionelle subjektive Testmethoden wie das Absolute Category Rating (ACR) und die Double Stimulus Impairment Scale (DSIS) zur Bewertung der Videoqualität eingesetzt, neben der im Rahmen dieser Arbeit neu vorgeschlagenen Modified ACR (M-ACR) Methode. Aufbauend auf der Zuverlässigkeit und allgemeinen Anwendbarkeit des Verfahrens über verschiedene Tests hinweg wird in dieser Arbeit ein methodischer Rahmen für die Bewertung der Qualität von 360˚-Videos vorgestellt. Zweitens bringt ein erhöhter Immersionsgrad bei 360˚-Videos das Problem der Simulatorkrankheit (Simulator Sickness) als weiteren QoE-Bestandteil mit sich. Daher wird in dieser Arbeit die Simulatorkrankheit analysiert, um die Auswirkungen verschiedener Einflussfaktoren zu untersuchen. Die gewonnenen Erkenntnisse zur Simulator Sickness im Zusammenhang mit 360˚-Videos tragen zu einem besseren Verständnis dieses speziellen Anwendungsfalls von VR bei. Darüber hinaus wird ein vereinfachter Fragebogen zur Simulatorkrankheit (SSQ) für die Selbsteinschätzung von Symptomen, die für 360˚-Videos relevant sind, vorgeschlagen, indem verschiedene Versionen von Fragebögen mit den State-of-the-Art-Varianten Cybersickness Questionnaire und Virtual Reality Symptom Questionnaire sowie den bestehenden SSQ-Skalen verglichen werden. Die Ergebnisse zeigen, dass die vereinfachte Version des SSQ sich auf die Symptome konzentriert, die für Studien mit 360˚-Videos relevant sind. Es wird gezeigt, dass er effektiv eingesetzt werden kann, wobei der reduzierte Satz von Skalen eine effizientere und damit umfangreichere Prüfung ermöglicht.
Towards the design and evaluation of more sustainable multimedia experiences: which role can QoE research play?. - In: ACM SIGMultimedia records, ISSN 1947-4598, Bd. 14 (2022), 3, 4, S. 1
In this column, we reflect on the environmental impact and broader sustainability implications of resource-demanding digital applications and services such as video streaming, VR/AR/XR and videoconferencing. We put emphasis not only on the experiences and use cases they enable but also on the "cost" of always striving for high Quality of Experience (QoE) and better user experiences. Starting by sketching the broader context, our aim is to raise awareness about the role that QoE research can play in the context of various of the United Nations' Sustainable Development Goals (SDGs), either directly (e.g., SDG 13 "climate action") or more indirectly (e.g., SDG 3 "good health and well-being" and SDG 12 "responsible consumption and production").
https://doi.org/10.1145/3630658.3630662
Investigating different cueing methods for auditory selective attention in virtual reality. - Berlin : Deutsche Gesellschaft für Akustik e.V.. - 1 Online-Ressource (4 Seiten)Online-Ausgabe: DAGA 2022 : 48. Jahrestagung für Akustik, 21.-24. März 2022, Stuttgart und Online, Seiten/Artikel-Nr: 1173-1176
An audio-only paradigm for investigating auditory selective attention (ASA) has previously been transferred into a classroom-type audio-visual virtual reality (VR) environment. Due to the paradigm structure, the participants were only focusing on a specific area of the VR environment during the entire experiment. In a more realistic scenario, participants are expected to interact with the scene. Therefore, this study investigates new cueing methods that may reduce the focus on one point in the virtual world and allow for further development of a close-to-real life scenario.
https://doi.org/10.18154/RWTH-2022-04388
Examining the auditory selective attention switch in a child-suited virtual reality classroom environment. - In: International journal of environmental research and public health, ISSN 1660-4601, Bd. 19 (2022), 24, 16569, S. 1-20
The ability to focus ones attention in different acoustical environments has been thoroughly investigated in the past. However, recent technological advancements have made it possible to perform laboratory experiments in a more realistic manner. In order to investigate close-to-real-life scenarios, a classroom was modeled in virtual reality (VR) and an established paradigm to investigate the auditory selective attention (ASA) switch was translated from an audio-only version into an audiovisual VR setting. The new paradigm was validated with adult participants in a listening experiment, and the results were compared to the previous version. Apart from expected effects such as switching costs and auditory congruency effects, which reflect the robustness of the overall paradigm, a difference in error rates between the audio-only and the VR group was found, suggesting enhanced attention in the new VR setting, which is consistent with recent studies. Overall, the results suggest that the presented VR paradigm can be used and further developed to investigate the voluntary auditory selective attention switch in a close-to-real-life classroom scenario.
https://doi.org/10.3390/ijerph192416569
Differential effects of task-irrelevant monaural and binaural classroom scenarios on children's and adults' speech perception, listening comprehension, and visual-verbal short-term memory. - In: International journal of environmental research and public health, ISSN 1660-4601, Bd. 19 (2022), 23, 15998, S. 1-17
Most studies investigating the effects of environmental noise on children’s cognitive performance examine the impact of monaural noise (i.e., same signal to both ears), oversimplifying multiple aspects of binaural hearing (i.e., adequately reproducing interaural differences and spatial information). In the current study, the effects of a realistic classroom-noise scenario presented either monaurally or binaurally on tasks requiring processing of auditory and visually presented information were analyzed in children and adults. In Experiment 1, across age groups, word identification was more impaired by monaural than by binaural classroom noise, whereas listening comprehension (acting out oral instructions) was equally impaired in both noise conditions. In both tasks, children were more affected than adults. Disturbance ratings were unrelated to the actual performance decrements. Experiment 2 revealed detrimental effects of classroom noise on short-term memory (serial recall of words presented pictorially), which did not differ with age or presentation mode (monaural vs. binaural). The present results add to the evidence for detrimental effects of noise on speech perception and cognitive performance, and their interactions with age, using a realistic classroom-noise scenario. Binaural simulations of real-world auditory environments can improve the external validity of studies on the impact of noise on children’s and adults’ learning.
https://doi.org/10.3390/ijerph192315998
Audiovisual database with 360˚ video and higher-order Ambisonics audio for perception, cognition, behavior, and QoE evaluation research. - In: 2022 14th International Conference on Quality of Multimedia Experience (QoMEX), (2022), insges. 6 S.
Research into multi-modal perception, human cognition, behavior, and attention can benefit from high-fidelity content that may recreate real-life-like scenes when rendered on head-mounted displays. Moreover, aspects of audiovisual perception, cognitive processes, and behavior may complement questionnaire-based Quality of Experience (QoE) evaluation of interactive virtual environments. Currently, there is a lack of high-quality open-source audiovisual databases that can be used to evaluate such aspects or systems capable of reproducing high-quality content. With this paper, we provide a publicly available audiovisual database consisting of twelve scenes capturing real-life nature and urban environments with a video resolution of 7680×3840 at 60 frames-per-second and with 4th-order Ambisonics audio. These 360˚ video sequences, with an average duration of 60 seconds, represent real-life settings for systematically evaluating various dimensions of uni-/multi-modal perception, cognition, behavior, and QoE. The paper provides details of the scene requirements, recording approach, and scene descriptions. The database provides high-quality reference material with a balanced focus on auditory and visual sensory information. The database will be continuously updated with additional scenes and further metadata such as human ratings and saliency information.
https://doi.org/10.1109/QoMEX55416.2022.9900893
Modeling of energy consumption and streaming video QoE using a crowdsourcing dataset. - In: 2022 14th International Conference on Quality of Multimedia Experience (QoMEX), (2022), insges. 6 S.
In the past decade, we have witnessed an enormous growth in the demand for online video services. Recent studies estimate that nowadays, more than 1% of the global greenhouse gas emissions can be attributed to the production and use of devices performing online video tasks. As such, research on the true power consumption of devices and their energy efficiency during video streaming is highly important for a sustainable use of this technology. At the same time, over-the-top providers strive to offer high-quality streaming experiences to satisfy user expectations. Here, energy consumption and QoE partly depend on the same system parameters. Hence, a joint view is needed for their evaluation. In this paper, we perform a first analysis of both end-user power efficiency and Quality of Experience of a video streaming service. We take a crowdsourced dataset comprising 447,000 streaming events from YouTube and estimate both the power consumption and perceived quality. The power consumption is modeled based on previous work which we extended towards predicting the power usage of different devices and codecs. The user-perceived QoE is estimated using a standardized model. Our results indicate that an intelligent choice of streaming parameters can optimize both the QoE and the power efficiency of the end user device. Further, the paper discusses limitations of the approach and identifies directions for future research.
https://doi.org/10.1109/QoMEX55416.2022.9900886
Can communication technologies reduce loneliness and social isolation in older people? : a scoping review of reviews. - In: International journal of environmental research and public health, ISSN 1660-4601, Bd. 19 (2022), 18, 11310, S. 1-20
Background: Loneliness and social isolation in older age are considered major public health concerns and research on technology-based solutions is growing rapidly. This scoping review of reviews aims to summarize the communication technologies (CTs) (review question RQ1), theoretical frameworks (RQ2), study designs (RQ3), and positive effects of technology use (RQ4) present in the research field. Methods: A comprehensive multi-disciplinary, multi-database literature search was conducted. Identified reviews were analyzed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework. A total of N = 28 research reviews that cover 248 primary studies spanning 50 years were included. Results: The majority of the included reviews addressed general internet and computer use (82% each) (RQ1). Of the 28 reviews, only one (4%) worked with a theoretical framework (RQ2) and 26 (93%) covered primary studies with quantitative-experimental designs (RQ3). The positive effects of technology use were shown in 55% of the outcome measures for loneliness and 44% of the outcome measures for social isolation (RQ4). Conclusion: While research reviews show that CTs can reduce loneliness and social isolation in older people, causal evidence is limited and insights on innovative technologies such as augmented reality systems are scarce.
https://doi.org/10.3390/ijerph191811310