Lip reading activates brain regions similar to real speech, researchers report.
Lip-read words can be decoded from the brain’s auditory regions similarly to heard speech, according to a new report that looked at how vision supports verbal perception.
The researchers used functional magnetic resonance imaging and electrodes implanted in the patients’ brains for the study.
David Brang, associate professor of psychology at the University of Michigan and the study’s senior author, says seeing a person’s facial movements often starts before sounds are produced. The auditory system uses these early visual cues to prime auditory neurons before the sounds are heard, he says.
The study indicates that integrating visual and auditory cues makes a person get more accurate and efficient speech information, which significantly enhances communication abilities.
Brang and colleagues sought to understand how the visual signals during lip reading are represented in the auditory system.
They used fMRI data from healthy adults and intracranial recordings from electrodes implanted in patients with epilepsy during auditory and visual speech perception tasks.
The findings revealed that lip-read words could be classified at earlier time points compared to heard words. This suggests that lip reading might involve a predictive mechanism that facilitates speech processing before auditory information becomes available, Brang says.
The results support a model in which the auditory system combines the neural distributions evoked by heard and lip-read words to generate a more precise estimate of what was said.
Brang says these findings suggest that the auditory system quickly integrates lip reading information to enhance hearing capabilities, especially in challenging auditory environments like noisy restaurants. Observing a speaker’s lips can influence our auditory perception even before any sounds are produced.
For people with hearing loss, this rapid use of lip reading information is likely even more pronounced, he adds.
“As hearing abilities decline, people increasingly rely on visual cues to aid their understanding,” Brang says. “The ability of visual speech to activate and encode information in the auditory cortex appears to be a crucial compensatory mechanism.”
This helps people maintain their hearing capacities as they age, underscoring the value of face-to-face communication in supporting auditory comprehension.
The study appears in Current Biology.
