Fusing Acoustic and Electroencephalographic Modalities for User-Independent Emotion Prediction

Search and retrieval of multimedia content based on the evoked emotion comprises an interesting scientific field with numerous applications. This paper proposes a method that fuses two heterogeneous modalities, i.e. music and electroencephalographic signals, both for predicting emotional dimensions in the valence-arousal plane and for addressing four binary classification tasks, namely i.e. high/low arousal, positive/negative valence, high/low dominance, high/low liking. The proposed solution exploits Mel-scaled and EEG spectrograms feeding a k-medoids clustering scheme based on canonical correlation analysis. A thorough experimental campaign carried out on a publicly available dataset confirms the efficacy of such an approach. Despite its low computational cost, it was able to surpass state of the art results, and most importantly, in a user-independent manner.