A Systematic Evaluation of Adversarial Attacks against Speech Emotion Recognition Models

Speech emotion recognition (SER) has been constantly gaining attention in recent yearsdue to its potential applications in diverse fields and thanks to the possibilities offered bydeep learning technologies. However, recent studies have shown that deep learning modelscan be vulnerable to adversarial attacks. In this paper, we systematically assess this problemby examining the impact of various adversarial white-box and black-box attacks on differentlanguages and genders within the context of SER. We first propose a suitable methodologyfor audio data processing, feature extraction, and convolutional neural network long short-term memory (CNN-LSTM) architecture. The observed outcomes highlighted the considerablevulnerability of CNN-LSTM models to adversarial examples (AEs). In fact, all the consideredadversarial attacks are able to considerably reduce the performance of the constructed models.Furthermore, when assessing the efficacy of the attacks, minor differences were noted betweenthe languages analyzed as well as between male and female speech. In summary, this workcontributes to the understanding of the robustness of CNN-LSTM models, particularly in SERscenarios, and the impact of AEs. Interestingly, our findings serve as a baseline for a) developingmore robust algorithms for SER, b) designing more effective attacks, c) investigating possibledefenses, d) improved understanding of the vocal differences between different languages andgenders, and e) overall, enhancing our comprehension of the SER tas