The goal of this project is to build a smart sensing system for sound detection which is capable of information processing and (self-) adaption to the current environment. The underlying principles are inspired by the sound detection in the (human) inner ear, the cochlea. The sound sensors in the cochlea, the hair cells, are situated on the basilar membrane, which is deflected by the incoming sound. Inner hair cells transduce BM deflections into electrical signals, i.e. excite the auditory nerve in dependence on the incoming sound signal (Fig. 1a). Outer hair cells (OHC) can modify the input to the inner hair cells by active processes (motilities) and thus adapt the properties of sound detection and the detected signal itself (Fig. 1b). In this way, a nonlinear amplification of signals is realized, amplifying small signals and compressing large ones. These active processes enable the large dynamic range of 120dB sound pressure level, corresponding to pressures from 20*10-6Pa - 20Pa with a dynamic resolution of 1dB SPL and a frequency resolution of minimal 4Hz (for frequencies <2kHz). The dynamics of the outer hair cells are regulated by diverse feedbacks, in particular neural feedback, to amplify or suppress specific signals. This facilitates an adaption of sensing to the hearing environment and enables e.g. understanding of speech in hard to hear environments (‘cocktail party effect’).
Unfortunately, ageing, noise and toxics, the main causes for acquired hearing loss, result in hair cell damage and loss. OHC loss yields a compression of the detectable dynamic range, a limitation of the frequency range, an increase of hearing thresholds by up to 55dB SPL and problems in speech perception.
This motivated us to work on the development of artificial hair cells [1] with sensing properties similar to the cochlea’s. This means nonlinear and adjustable sensing, which can be used to dampen or amplify specific signals and vary the frequency resolution. These artificial hair cells could be used as:

1. replacement for damaged hair cells to compensate for hearing loss,

2. experimental system to study dynamics of nonlinear and coupled oscillators,

3. biomimetic microphone with integrated amplification, filtering and processing due to operation principle. Such a biomimetic microphone could possibly deliver more natural signals with less (artificial) signal distortion compared to conventional technologies, which are based on microphones with linear characteristics and subsequent extensive signal processing. This could be particularly interesting for hearing aid and cochlea implant technologies, since it could possibly reduce adaption of patients to the technologies and enable energy-efficient, fully implantable systems.

4. an example for a smart, self-adapting sensing system with integrated pre-processing. Such a system could decrease the need for data stream, data processing and storage, in particular in the context of the internet of things, since only relevant data is acquired and already pre-processed at the sensor level.

 

The operation principle of our artificial hair cells and the applied sensor are shown in Fig. 2. As sensors, we use so-called active cantilevers [2], which are silicon beams with integrated thermomechanical actuation and deflection sensing. Here deflection of the beam due to sound signals is transduced into an electrical signal by the piezo-resistive elements. A real-time feedback driving the actuator is applied to operate our sensor as a critical oscillator. Such oscillators automatically show a nonlinear amplification similar to one observed for sound detection in the cochlea. The feedback algorithms are derived from nonlinear dynamics theory and physiological models of hair cell dynamics. Artificial hair cells based on this set-up exhibit the desired nonlinear amplification and adjustability of detection properties [3].

As shown in Fig. 3, for the passive case (without feedback) the (normalized) sensor response is small and almost similar for different sound pressure levels, whereas for the active case (with feedback) the response increases strongly with decreasing sound pressure level. This corresponds to the amplification of low volume sounds (19 dB compared to passive case) and strongly increased signal-to-noise ratios, in particular for low volume sounds. Here the normalized response of the active sensor can be adjusted with the feedback parameters. The planned extension of the set-up to multi-sensor arrays with different resonance frequencies will enable the coverage of larger frequency ranges and the exploitation of nonlinear effects arising from the coupling of these nonlinear oscillators. The latter are thought to improve the amplification and frequency resolution even further. By combination of the adaptive sensors with neuromorphic, learning systems, which are capable of acoustic scene analysis and modification of the feedback, a self-standing sensor system can be realized, which adapts itself to the current hearing environment and sensing requirements.

[1] C. Lenk, S. Gutschmidt, I. W. Rangelow, “Vorrichtung und Verfahren zur Aufnahme von Schall in Gasen oder Flüssigkeiten”, Patent applied.
[2] I .W. Rangelow, T. Ivanov, A. Ahmad, M. Kästner, C. Lenk, I.S. Bozchalooi, Fangzhou Xia, K. Youcef-Toumi, M. Holz and A. Reum, J. Vac. Sci. & Tech. B 35, 06G101 (2017).
[3] C. Lenk, A. Ekinci, I. W. Rangelow, S. Gutschmidt, Conf Proc IEEE Eng Med Biol Soc. 2018, 4488-4491 (2018).