Artificial neural networks decode human brain activity during performed and imagined movements

Filtering information for search engines, acting as an opponent during a board game or recognizing images: Artificial intelligence has far outpaced human intelligence in certain tasks. Researchers are showing how ideas from computer science could revolutionize brain research. They illustrate how a self-learning algorithm decodes human brain signals that were measured by an electroencephalogram (EEG).

Filtering information for search engines, acting as an opponent during a board game or recognizing images: Artificial intelligence has far outpaced human intelligence in certain tasks. Several groups from the Freiburg excellence cluster BrainLinks-BrainTools led by neuroscientist private lecturer Dr. Tonio Ball are showing how ideas from computer science could revolutionize brain research. In the scientific journal Human Brain Mapping they illustrate how a self-learning algorithm decodes human brain signals that were measured by an electroencephalogram (EEG).

It included performed movements, but also hand and foot movements that were merely thought of, or an imaginary rotation of objects. Even though the algorithm was not given any characteristics ahead of time, it works as quickly and precisely as traditional systems that have been created to solve certain tasks based on predetermined brain signal characteristics, which are therefore not appropriate for every situation.

The demand for such diverse intersections between human and machine is huge: At the University Hospital Freiburg, for instance, it could be used for early detection of epileptic seizures. It could also be used to improve communication possibilities for severely paralyzed patients or an automatic neurological diagnosis.

"Our software is based on brain-inspired models that have proven to be most helpful to decode various natural signals such as phonetic sounds," says computer scientist Robin Tibor Schirrmeister. The researcher is using it to rewrite methods that the team has used for decoding EEG data: So-called artificial neural networks are the heart of the current project at BrainLinks-BrainTools. "The great thing about the program is we needn't predetermine any characteristics. The information is processed layer for layer, that is in multiple steps with the help of a non-linear function. The system learns to recognize and differentiate between certain behavioral patterns from various movements as it goes along," explains Schirrmeister. The model is based on the connections between nerve cells in the human body in which electric signals from synapses are directed from cellular protuberances to the cell's core and back again. "Theories have been in circulation for decades, but it wasn't until the emergence of today's computer processing power that the model has become feasible," comments Schirrmeister.

Customarily, the model's precision improves with a large number of processing layers. Up to 31 were used during the study, otherwise known as "Deep Learning." Up until now, it had been problematic to interpret the network's circuitry after the learning process had been completed. All algorithmic processes take place in the background and are invisible. That is why the researchers developed the software to create cards from which they could understand the decoding decisions. The researchers can insert new datasets into the system at any time. "Unlike the old method, we are now able to go directly to the raw signals that the EEG records from the brain. Our system is as precise, if not better, than the old one," says head investigator Tonio Ball, summarizing the study's research contribution. The technology's potential has yet to be exhausted -- together with his team, the researcher would like to further pursue its development: "Our vision for the future includes self-learning algorithms that can reliably and quickly recognize the user's various intentions based on their brain signals. In addition, such algorithms could assist neurological diagnoses."