EEG and EMG graph CNN for upper limb movement classification

Neural Human-Machine Interfaces (HMI) are technologies that allow a human being to communicate his/her intention to external devices – such as computers, robots and prostheses – through the acquisition of signals from the central nervous system. To overcome the limitations of interfaces based on unimodal neural signals, the research community recently introduced the concept of hybrid neural interfaces based on the combination of brain and muscular signals. However, state-of-the-art approaches still relies on simple additive fusion techniques or heuristic decision rules, limiting the performance in complex decoding tasks. To go beyond the state-of-the-art, this thesis proposes to model the connectivity in the human neuromuscular system through a weighted undirected graph: the nodes of the graph represent the electroencephalography (EEG) and electromyography (EMG) channels, and the weight of the connections represents signals’ correlation. Then, a graph convolutional neural network (GCN) with learning structure was defined and implemented to let the model learn both the graph connections and the feature extraction automatically from the data. The GCN has been evaluated in different deep learning architectures combined with gated recurrent units (GRU), as well as with input signals in both the time and frequency domain, in a multi-class upper limb motion classification application. The obtained results show that the proposed cortico-muscular graph neural network (CMGNet) is capable of predicting reaching, grasping and wrist twisting with more than 99% accuracy on average. In addition, it shows promising performance in a challenging classification scenario with 11 different upper limb motions, outperforming state-of-the-art machine learning approaches. The encouraging results suggest that the development of advanced AI approaches which explicitly consider the inner function of the human neuromotor system may be the key to significantly improve the reliability of neuro-driven devices.