Motivated by the state-of-art psychological research, we note that a piano performance transcribed with existing Automatic Music Transcription (AMT) methods cannot be successfully resynthesized without affecting the artistic content of the performance. This is due to 1) the different mappings between MIDI parameters used by different instruments, and 2) the fact that musicians adapt their way of playing to the surrounding acoustic environment. To face this issue, we propose a methodology to build acoustics-specific AMT systems that are able to model the adaptations that musicians apply to convey their interpretation. Specifically, we train models tailored for virtual instruments in a modular architecture that takes as input an audio recording and the relative aligned music score, and outputs the acoustics-specific velocities of each note. We test different model shapes and show that the proposed methodology generally outperforms the usual AMT pipeline which does not consider specificities of the instrument and of the acoustic environment. Interestingly, such a methodology is extensible in a straightforward way since only slight efforts are required to train models for the inference of other piano parameters, such as pedaling.

Acoustics-specific Piano Velocity Estimation / F. Simonetta, S. Ntalampiras, F. Avanzini - In: 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)[s.l] : IEEE, 2022 Sep. - ISBN 978-1-6654-7189-3. (( Intervento presentato al 24. convegno MMSP nel 2022 [10.1109/MMSP55362.2022.9948719].

Acoustics-specific Piano Velocity Estimation

F. Simonetta
Primo
;
S. Ntalampiras
Secondo
;
F. Avanzini
Ultimo
2022

Abstract

Motivated by the state-of-art psychological research, we note that a piano performance transcribed with existing Automatic Music Transcription (AMT) methods cannot be successfully resynthesized without affecting the artistic content of the performance. This is due to 1) the different mappings between MIDI parameters used by different instruments, and 2) the fact that musicians adapt their way of playing to the surrounding acoustic environment. To face this issue, we propose a methodology to build acoustics-specific AMT systems that are able to model the adaptations that musicians apply to convey their interpretation. Specifically, we train models tailored for virtual instruments in a modular architecture that takes as input an audio recording and the relative aligned music score, and outputs the acoustics-specific velocities of each note. We test different model shapes and show that the proposed methodology generally outperforms the usual AMT pipeline which does not consider specificities of the instrument and of the acoustic environment. Interestingly, such a methodology is extensible in a straightforward way since only slight efforts are required to train models for the inference of other piano parameters, such as pedaling.
Music; Transcription; Music Information Processing; Neural Networks; Deep Learning; NMF
Settore INF/01 - Informatica
Settore ING-INF/06 - Bioingegneria Elettronica e Informatica
Settore L-ART/07 - Musicologia e Storia della Musica
Book Part (author)
File in questo prodotto:
File Dimensione Formato  
2203.16294.pdf

accesso aperto

Descrizione: Accepted version
Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione 759.35 kB
Formato Adobe PDF
759.35 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/945979
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? 0
social impact