A Multimodal Deep Learning Framework for Improved Parkinson’s Disease Detection Using Biosignals: A BioVRSea Paradigm Study

Electroencephalography (EEG), electromyography (EMG), and center of pressure (CoP) signals capture complementary aspects of motor and postural control. These modalities have recently emerged as promising biomarkers for early detection of Parkinson’s disease. Despite encouraging progress with deep learning models applied to single modalities, existing approaches often suffer from high inter-subject variability and limited robustness. This limits their generalization to real-world clinical scenarios. This thesis aims to improve early-stage PD classification by developing a multimodal deep learning framework that effectively integrates EEG, EMG, and CoP signals while preserving their temporal structure. Literature-established architectures for EEG, EMG, and CoP were adapted into modality-specific encoders by modifying their structures to retain temporal dynamics within learned representations. These encoders were combined through fusion strategies, emphasizing sequence-level integration. Additionally, a novel Multiple Instance Learning (MIL) paradigm was introduced to assess its potential advantages over supervised learning. The proposed framework was evaluated on the BioVRSea dataset comprising 300 subjects (29 Parkinson’s disease patients and 271 healthy controls) using a stratified nested cross-validation scheme to ensure unbiased subject-level evaluation and prevent information leakage. In the supervised setting with data augmentation, multimodal fusion achieved a median balanced accuracy of 81.67% and an F1-score of 75.00%, with a [1st–99th] balanced accuracy percentile range of 23.54%. MIL-based experiments did not consistently outperform the supervised framework and showed strong dependence on architectural design, suggesting limited reliability under current dataset constraints. These results demonstrate that preserving temporal dynamics within modality-specific representations and integrating them through supervised multimodal fusion substantially improves classification performance. The findings highlight the potential of multimodal deep learning as a robust approach for early Parkinson’s disease detection from EEG, EMG, and CoP signals.