For the first time scientists have used an implanted sensor to record brain signals associated with handwriting and used those signals to create text on a computer in real time.
As reported in the journal Nature, a clinical trial participant with cervical spinal cord injury used the system to “type” words on a computer at a rate of 90 characters per minute, more than double the previous record for typing with a brain-computer interface.
The participant had to merely think about the hand motions involved in creating written letters to complete the task.
The research team is hopeful that such a system could one day help to restore people’s ability to communicate following paralysis caused by injury or illness.
The new study is part of the BrainGate clinical trial, directed by Leigh Hochberg, a critical care neurologist and a professor at Brown University’s School of Engineering affiliated with the Carney Institute for Brain Science.
Frank Willett, a research scientist at Stanford University and the Howard Hughes Medical Institute (HHMI), led the study, which was supervised by Krishna Shenoy, a Stanford professor and HHMI investigator, and Jaimie Henderson, a professor of neurosurgery at Stanford.
“An important mission of our BrainGate consortium research is to restore rapid, intuitive communication for people with severe speech or motor impairments,” says Hochberg, who also directs the Center for Neurotechnology and Neurorecovery at Massachusetts General Hospital and the VA Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology at the Veterans Affairs Providence Healthcare System.
“Frank’s demonstration of fast, accurate neural decoding of handwriting marks an exciting new chapter in the development of clinically useful neurotechnologies.”
Faster communication
The BrainGate collaboration has worked for several years on systems that enable people to generate text through direct brain control.
Previous incarnations have involved trial participants thinking about the motions involved in pointing to and clicking letters on a virtual keyboard. That system enabled one participant to type 40 characters per minute, which was the previous record speed.
For the new study, the team wanted to find out if asking a participant to think about motions involved in writing letters and words by hand would be faster.
“We want to find new ways of letting people communicate faster,” Willett says. “This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text.”
The trial participant, a 65-year-old (at the time of the study) man, was paralyzed from the neck down by a spinal cord injury.
As part of the clinical trial, Henderson placed two tiny electrodes about the size of a baby aspirin in a part of his brain associated with the movement of his right arm and hand.
Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter.
With this system, the man could copy sentences and answer questions at a rate similar to that of someone the same age typing on a smartphone.
The system is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, Willett says.
Steady stream of bci refinements
The new research is the latest in a series of advances in brain-computer interfaces (BCIs) made by the BrainGate collaboration, which includes researchers from Brown University, Massachusetts General Hospital, Harvard Medical School, the Providence VA Medical Center, Stanford University, and Case Western Reserve University.
In 2012, the team published landmark research in which clinical trial participants were able, for the first time, to operate multidimensional robotic prosthetics using a BCI.
A steady stream of refinements to the system, as well as new clinical breakthroughs that have enabled people to directly control tablet apps and even move their own paralyzed limbs, have followed that work. Most recently, the team demonstrated the first human use of a wireless intracortical BCI that can transmit neural data at full bandwidth.
Hochberg says he’s grateful to clinical trial participants for making these breakthroughs and future ones possible.
“The people who enroll in the BrainGate trial are amazing,” Hochberg says. “It’s their pioneering spirit that not only allows us to gain new insights into human brain function, but that leads to the creation of systems that will help other people with paralysis.”
The National Institute of Neurological Disorders and Stroke and the NIH BRAIN Initiative, the National Institute on Deafness and Other Communication Disorders, Howard Hughes Medical Institute, the US Department of Veterans Affairs, L. and P. Garlick, S. and B. Reeves, the Wu Tsai Neurosciences Institute at Stanford, and the Simons Foundation Collaboration on the Global Brain funded the work.
Written by Kevin Stacey.
