An Attention-Based Multi-Domain Bi-Hemisphere Discrepancy Feature Fusion Model for EEG Emotion Recognition

Electroencephalogram (EEG)-based emotion recognition has become a research hotspot in the field of brain-computer interface. Previous emotion recognition methods have overlooked the fusion of multi-domain emotion-specific information to improve performance, and faced the challenge of insufficient interpretability. In this paper, we proposed a novel EEG emotion recognition model that combined the asymmetry of the brain hemisphere, and the spatial, spectral, and temporal multi-domain properties of EEG signals, aiming to improve emotion recognition performance. Based on the 10-20 standard system, a global spatial projection matrix (GSPM) and a bi-hemisphere discrepancy projection matrix (BDPM) are constructed. A dual-stream spatial-spectral-temporal convolution neural network is designed to extract depth features from the two matrix paradigms. Finally, the transformer-based fusion module is used to learn the dependence of fused features, and to retain the discriminative information. We conducted extensive experiments on the SEED, SEED-IV, and DEAP public datasets, achieving excellent average results of 98.33/2.46<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula>, 92.15/5.13<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula>, 97.60/1.68<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula>(valence), and 97.48/1.42<inline-formula><tex-math notation="LaTeX">$\%$</tex-math></inline-formula>(arousal) respectively. Visualization analysis supports the interpretability of the model, and ablation experiments validate the effectiveness of multi-domain and bi-hemisphere discrepancy information fusion.