Neural Human-Machine Interfaces (HMI) are technologies that allow a human being to communicate his/her intention to external devices – such as computers, robots and prostheses – through the acquisition of signals from the central nervous system. To overcome the limitations of interfaces based on unimodal neural signals, the research community recently introduced the concept of hybrid neural interfaces based on the combination of brain and muscular signals. However, state-of-the-art approaches still relies on simple additive fusion techniques or heuristic decision rules, limiting the performance in complex decoding tasks. To go beyond the state-of-the-art, this thesis proposes to model the connectivity in the human neuromuscular system through a weighted undirected graph: the nodes of the graph represent the electroencephalography (EEG) and electromyography (EMG) channels, and the weight of the connections represents signals’ correlation. Then, a graph convolutional neural network (GCN) with learning structure was defined and implemented to let the model learn both the graph connections and the feature extraction automatically from the data. The GCN has been evaluated in different deep learning architectures combined with gated recurrent units (GRU), as well as with input signals in both the time and frequency domain, in a multi-class upper limb motion classification application. The obtained results show that the proposed cortico-muscular graph neural network (CMGNet) is capable of predicting reaching, grasping and wrist twisting with more than 99% accuracy on average. In addition, it shows promising performance in a challenging classification scenario with 11 different upper limb motions, outperforming state-of-the-art machine learning approaches. The encouraging results suggest that the development of advanced AI approaches which explicitly consider the inner function of the human neuromotor system may be the key to significantly improve the reliability of neuro-driven devices.
Neural Human-Machine Interfaces (HMI) are technologies that allow a human being to communicate his/her intention to external devices – such as computers, robots and prostheses – through the acquisition of signals from the central nervous system. To overcome the limitations of interfaces based on unimodal neural signals, the research community recently introduced the concept of hybrid neural interfaces based on the combination of brain and muscular signals. However, state-of-the-art approaches still relies on simple additive fusion techniques or heuristic decision rules, limiting the performance in complex decoding tasks. To go beyond the state-of-the-art, this thesis proposes to model the connectivity in the human neuromuscular system through a weighted undirected graph: the nodes of the graph represent the electroencephalography (EEG) and electromyography (EMG) channels, and the weight of the connections represents signals’ correlation. Then, a graph convolutional neural network (GCN) with learning structure was defined and implemented to let the model learn both the graph connections and the feature extraction automatically from the data. The GCN has been evaluated in different deep learning architectures combined with gated recurrent units (GRU), as well as with input signals in both the time and frequency domain, in a multi-class upper limb motion classification application. The obtained results show that the proposed cortico-muscular graph neural network (CMGNet) is capable of predicting reaching, grasping and wrist twisting with more than 99% accuracy on average. In addition, it shows promising performance in a challenging classification scenario with 11 different upper limb motions, outperforming state-of-the-art machine learning approaches. The encouraging results suggest that the development of advanced AI approaches which explicitly consider the inner function of the human neuromotor system may be the key to significantly improve the reliability of neuro-driven devices.
EEG and EMG graph CNN for upper limb movement classification
PALATELLA, ALESSIO
2021/2022
Abstract
Neural Human-Machine Interfaces (HMI) are technologies that allow a human being to communicate his/her intention to external devices – such as computers, robots and prostheses – through the acquisition of signals from the central nervous system. To overcome the limitations of interfaces based on unimodal neural signals, the research community recently introduced the concept of hybrid neural interfaces based on the combination of brain and muscular signals. However, state-of-the-art approaches still relies on simple additive fusion techniques or heuristic decision rules, limiting the performance in complex decoding tasks. To go beyond the state-of-the-art, this thesis proposes to model the connectivity in the human neuromuscular system through a weighted undirected graph: the nodes of the graph represent the electroencephalography (EEG) and electromyography (EMG) channels, and the weight of the connections represents signals’ correlation. Then, a graph convolutional neural network (GCN) with learning structure was defined and implemented to let the model learn both the graph connections and the feature extraction automatically from the data. The GCN has been evaluated in different deep learning architectures combined with gated recurrent units (GRU), as well as with input signals in both the time and frequency domain, in a multi-class upper limb motion classification application. The obtained results show that the proposed cortico-muscular graph neural network (CMGNet) is capable of predicting reaching, grasping and wrist twisting with more than 99% accuracy on average. In addition, it shows promising performance in a challenging classification scenario with 11 different upper limb motions, outperforming state-of-the-art machine learning approaches. The encouraging results suggest that the development of advanced AI approaches which explicitly consider the inner function of the human neuromotor system may be the key to significantly improve the reliability of neuro-driven devices.File | Dimensione | Formato | |
---|---|---|---|
Palatella_Alessio.pdf
accesso aperto
Dimensione
5.34 MB
Formato
Adobe PDF
|
5.34 MB | Adobe PDF | Visualizza/Apri |
The text of this website © Università degli studi di Padova. Full Text are published under a non-exclusive license. Metadata are under a CC0 License
https://hdl.handle.net/20.500.12608/39200