Creaking doors, cracking switches and a ringing machine telegraph: Unusual sounds and noises were recently heard on the deep-sea salvage tug SEEFALKE, a museum ship of the German Maritime Museum DSM Bremerhaven. The reason for the special "concert" was a semester project by students of the Ilmenau Technical University. The goal of their mission: an audiovisual reconstruction of the ship's bridge and chart room in virtual reality (VR).
How can the historicity of sounds and noises be integrated into a museum narrative? What role do acoustics play in conveying knowledge in a museum in connection with the movement of visitors in the field of tension between the analog and the digital? These are the questions being addressed by the "Historical Sounds" project at the Electronic Media Group at TU Ilmenau. Four media technology students are also involved. At the end of February, they were on board the museum towboat SEEFALKE in Bremerhaven to reconstruct the spaces of the ship in an audiovisual VR environment and add historical sounds.
"Sensory experience plays a central role in museum education," explains department head Dr. Stephan Werner. As part of a semester project, he would like to work with the students to investigate the connection between room acoustics, spatial perception and knowledge transfer with the help of modern VR technology. For two days, the team of employees and students was active on board the SEEFALKE, making audiovisual recordings of the interiors of the ocean-going tug and later reproducing them positionally dynamically in a VR environment. The young scientists are working closely with cognitive psychologists from the Leibniz Institute for Knowledge Media and the German Maritime Museum.
Paul Popp, a master's student in media technology, was also on board: "The task of our media project is to scan the bridge of the museum ship in Bremerhaven with a 3D scanner and capture the acoustics of the rooms on the ship with a special microphone array. In the end, both parts should be joined together and the SEEFALKE can thus be experienced authentically in virtual reality."
Paul and his fellow students Kay Strama, Johannes Fried and Maurice Teuber were already able to work on several practical projects during his bachelor's degree: "For example, the Raspberry Pi project, in which we built an arcade game machine. What's particularly great about projects like this is the feeling of having a working result at the end, even if there were a few stumbling blocks along the way in between. With our current project, of course, it's super cool to travel to another city and work on the project there."
VR application with authentic historical soundscape
"First, we used our handheld 3D scanner to capture the bridge and chart room of the Seefalke," Paul explains. "In the process, we had to pay special attention to more complicated objects, such as the steering wheel." The students also placed speakers on board at 13 different locations to capture the room acoustics of the bridge and chart room with a special microphone at 15 listening positions. "We then used a normal microphone to record all kinds of sounds, such as the clacking of switches or slamming of doors."
In the next step, the students have to sift through the material and rework it: "For example, in the 3D scan, holes have to be filled by hand that the scanner didn't register correctly," says Paul. The young scientists then generate filters from the acoustic measurements using the so-called "Spatial Decomposition Method": "So that with headphones it sounds as if we were really there. We also have to simulate other filters so that the acoustics change when doors and windows are opened or closed."
At TU Ilmenau's new Ilmenau Interactive Immersive Technologies Center (I3TC) , which is dedicated to holistic research into technologies and applications of immersive media - virtual reality (VR) and augmented reality (AR), but also spatial audio and immersive video formats - as well as their use and evaluation, the students have excellent facilities for this. The goal of their media project is to build a VR application that allows users to move through the SEEFALKE with VR goggles, interact with objects, and additionally listen to the authentic historical soundscape of the deep-sea tug. Based on this, the Leibnitz Institute for Knowledge Media Tübingen will conduct an empirical study to investigate whether the transfer of information about the ship is more successful through authentic sounds and noises.
"It was a lot of fun to experience the SEEFALKE in real life after a lot of planning," says Paul: "On site, everything was a bit smaller than expected, and the drizzle challenged us a bit with all the technology. Overall, though, it was a really great experience to be able to do something like this as a student."
Contact
Dr. Stephan Werner
Acting Head of the Electronic Media Technology Group