his project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 618075.
Funding period: 01.12.2013 - 30.11.2016
Project description
Two!Ears replaces current thinking about auditory modeling by a systemic approach in which human listeners are regarded as multi-modal agents that develop their concept of the world by exploratory interaction. The goal of the project is to develop an intelligent, active computational model of auditory perception and experience in a multi-modal context. Our novel approach is based on a structural link from binaural perception to judgment and action, realized by interleaved signal-driven (bottom-up) and hypothesis-driven (top-down) processing within an innovative expert-system architecture. The system achieves object formation based on Gestalt principles, meaning assignment, knowledge acquisition and representation, learning, logic-based reasoning and reference-based judgment. More specifically, the system assigns meaning to acoustic events by combining signal- and symbol-based processing in a joint model structure, integrated with proprioceptive and visual percepts. It is therefore able to describe an acoustic scene in much the same way that a human listener and in terms of the sensations that sounds evoke (e.g. loudness, timbre, spatial extent) and their semantics (e.g., whether the sound is unexpected or a familiar voice). Our system will be implemented on a robotic platform, which will actively parse its physical environment, orientate itself and move its sensors in a humanoid manner. The system has an open architecture, so that it can easily be modified or extended. This is crucial, since the cognitive functions to be modeled are domain and application specific. Two!Ears will have significant impact on future development of ICT wherever knowledge and control of aural experience is relevant. It will also benefit research in related areas such as biology, medicine and sensory and cognitive psychology.
Website: http://www.twoears.eu/ (Note: Menu at the bottom of the window)
Project partners
- Neural Information Processing Group, TU Berlin
- Department of Electrical Engineering-Hearing Systems, Technical University of Denmark
- Institute of Communication Acoustics, Ruhr University Bochum
- The Institute for Intelligent Systems and Robotics, UPMC, France
- Robotics, Action and Perception Group, LAAS, France
- Institute of Communications Engineering, University Rostock
- Department of Computer Science, University of Sheffield, UK
- Human-Technology Interaction Group, Eindhoven University of Technology, Netherlands
- The Center for Cognition, Communication, and Culture, Rensselaer, USA
- Technische Universität Ilmenau, Institute for Media Technology, Audiovisual Technology Group
Project coordinator: Prof. Dr.-Ing. Alexander Raake
Project staff: Dr. Hagen Wierstorf
