A breakthrough in auditory technology demonstrates a real-time brain-controlled system that enhances speech comprehension by identifying and amplifying a targeted speaker, paving the way for advanced hearing aids that respond to focus rather than merely sound.
Study: Brain-controlled selective hearing in real-time improves speech perception in multi-speaker environments. Image credit: OpenAI. Neural harmony in a bustling crowd. 2026. AI-generated illustration.
A recent study published in Nature Neuroscience has unveiled a cutting-edge brain-controlled auditory system designed to assist individuals in focusing on a single voice amidst loud environments. This innovative system recorded real-time brain activity from patients undergoing neurosurgery and demonstrated its ability to identify and amplify the voice of a designated speaker.
The research indicated that the system consistently enhanced speech understanding, even amid similar voices and background noise for participants with normal hearing. Notably, another group consisting of individuals with hearing loss reported preferring and better comprehending the amplified audio quality provided by the system.
The findings suggest potential advancements in developing brain-controlled hearing aids aimed at improving speech recognition and auditory clarity in bustling settings like restaurants and social gatherings.
Understanding the Brain-controlled Hearing Device
Individuals, especially those with hearing impairments, often face challenges in following conversations in crowded spaces, even when utilizing traditional hearing aids. This difficulty arises because conventional devices typically amplify all sounds, including distracting background noise.
To address this issue, researchers developed the Auditory Attention Decoding (AAD) technology, which detects the specific voice a person is listening to and amplifies it based on signals from brain activity. Prior studies had tested this technology under controlled laboratory conditions, but its real-time performance remained uncertain.
Study Design for Intracranial EEG Hearing
In the present study, the researchers evaluated the brain-controlled hearing system using four adults who were being monitored for epilepsy treatment. They recorded high-resolution brain activities through clinically implanted intracranial electroencephalography (iEEG) electrodes covering speech and sound processing areas during neurosurgery. The study participants, self-reported as having normal hearing, listened to conversations.
The team initially trained the system before testing it in real-time. During the offline training stage, participants listened to two simultaneous streams of conversation, simulating multiple speakers in a confined space discussing everyday topics. The complexity of the task was increased by utilizing similar voices, such as speakers of the same gender, and adding background sounds mimicking real-life scenarios.
Participants were instructed to focus on one conversation while ignoring the other. They confirmed this by pressing a button when they heard repeated words in their chosen dialogue. While listening, the team recorded the brain activity and then trained AAD models to identify patterns in brain signals and reconstruct the rhythm or „voice envelope“ of the primary speaker.
During the controlled online testing phase, utilizing realistic multi-talker and noise conditions, the technology continuously analyzed participants‘ brain signals to determine which speaker attracted their attention. Once identified, the system automatically amplified that voice while maintaining the overall sound level.
The researchers measured listening performance before and after the brain-controlled enhancements and assessed the system’s ability to adapt quickly when participants shifted their focus to another speaker. They also monitored natural changes in speaker identity and assessed mental effort through pupil dilation.
Results of Selective Hearing System Performance
The system demonstrated consistent effectiveness across various tests and auditory situations. Electrodes positioned above the superior temporal gyrus provided the most useful brain signals. During offline testing, the system accurately recognized the targeted voice in 72.0% to 90.3% of decoding windows for all participants. The technology reliably functioned even in challenging circumstances involving similar voices and ambient noise.
In real-time applications, the system automatically amplified the voice on which the listener concentrated. The researchers recorded a 12 dB improvement in the target-to-masking ratio, indicating a clearer distinction of the designated speaker’s voice from unmonitored sounds and noise.
Participants showed a marked preference for listening with the system activated, reporting enhanced speech comprehension. The researchers noted decreased pupil dilation when the system was in operation, suggesting lower cognitive load for participants as they engaged in conversations. Enhanced brain-based attention decoding correlated with participant preference for the auditory experience.
When directed to switch their focus to another speaker, the system adjusted with an average transition time of 5.1 seconds, indicating its capability to follow natural shifts in audience attention.
Participants with normal hearing reported improvements while using the Closed-Loop system, while individuals with hearing loss showcased enhanced speech understanding and preferred the improved audio quality.
Implications of Brain-controlled Hearing Technology
The findings highlight the potential of brain-controlled hearing systems to aid individuals in better comprehending speech in noisy environments by recognizing and amplifying the intended voice. The results suggest that such systems could be particularly relevant for everyday scenarios where listener attention is constantly shifting. However, the required invasive procedures for electrode implantation may limit routine use of this technology.
Despite this, scientists can leverage this system as a benchmark and proof of concept to develop more intelligent, personalized versions with less invasive brain-computer interface technologies in the future.
