We present an approach to the problem of real-time generation of music, driven by the affective state of the user, estimated from their electroencephalogram (EEG). This work is aimed at exploring strategies for real-time music generation applications using sensor data. Applications can range from responsive music for x-reality to art installations, and music generation as feedback in pedagogical contexts. We developed a Brain-Computer Interface in the open-source platform OpenViBE. It manages communication with the EEG device and computes the relevant features. A benchmark dataset is used to evaluate the performance of supervised learning methods on the binary classification task of valence and arousal. We also assessed the performance using a reduced number of electrodes and frequency-bands, in order to address the problems of lower budgets and noisy environments. Then, we address the requirements for a real-time music generation model and propose a modification to Magenta's MusicVAE, introducing a parameter for controlling inter-batch memory. In the end, we discuss possible strategies to map desired music features to a model's native input features. We present a Probabilistic Graphical Model to model the mapping from valence/arousal to MusicVAE's latent variables. We also address dataset dimensionality problems proposing three probabilistic solutions.
Listen to your Mind’s (He)Art: A System for Affective Music Generation via Brain-Computer Interface / M. Tiraboschi, F. Avanzini, G. Boccignone - In: Proceedings of the 18th Sound and Music Computing Conference / [a cura di] D.A. Mauro, S. Spagnol, A. Valle. - [s.l] : SMC, 2021. - ISBN 9788894541540. - pp. 146-153 (( Intervento presentato al 18. convegno Sound and Music Computing Conference tenutosi a Torino nel 2021.
Titolo: | Listen to your Mind’s (He)Art: A System for Affective Music Generation via Brain-Computer Interface | |
Autori: | AVANZINI, FEDERICO (Secondo) BOCCIGNONE, GIUSEPPE (Ultimo) | |
Parole Chiave: | bci; affective; music; bcmi; generation; ann; vae; magenta; model; smc; eeg; openViBE; pgm | |
Settore Scientifico Disciplinare: | Settore INF/01 - Informatica | |
Data di pubblicazione: | 2021 | |
URL: | https://zenodo.org/record/5054146/files/SMC2021.pdf | |
Tipologia: | Book Part (author) | |
Appare nelle tipologie: | 03 - Contributo in volume |
File in questo prodotto:
File | Descrizione | Tipologia | Licenza | |
---|---|---|---|---|
SMC_2021_paper_65.pdf | Articolo principale | Publisher's version/PDF | Open Access Visualizza/Apri |