This paper employs the acoustic modality to address the human activity recognition (HAR) problem. The cornerstone of the proposed solution is the YAMNet deep neural network, the embeddings of which comprise the input to a fully-connected linear layer trained for HAR. Importantly, the dataset is publicly available and includes the following human activities: preparing coffee, frying egg, no activity, showering, using microwave, washing dishes, washing hands, and washing teeth. The specific set of activities is representative of a standard home environment facilitating a wide range of applications. The performance offered by the proposed transfer learning-based framework surpasses the state of the art, while being able to be executed on mobile devices, such as smartphones, tablets, etc. In fact, the obtained model has been exported and thoroughly tested for real-time HAR on a smartphone device with the input being the audio captured from its microphone.

Lightweight Audio-Based Human Activity Classification Using Transfer Learning / M. Nicolini, F. Simonetta, S. Ntalampiras - In: Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods / [a cura di] M. De Marsico, G. Sanniti di Baja, A. Fred. - [s.l] : ScitePress, 2023. - ISBN 978-989-758-626-2. - pp. 783-789 (( Intervento presentato al 12. convegno International Conference on Pattern Recognition Applications and Methods tenutosi a Lisbon nel 2013 [10.5220/0011647900003411].

Lightweight Audio-Based Human Activity Classification Using Transfer Learning

F. Simonetta
Secondo
;
S. Ntalampiras
Ultimo
2023

Abstract

This paper employs the acoustic modality to address the human activity recognition (HAR) problem. The cornerstone of the proposed solution is the YAMNet deep neural network, the embeddings of which comprise the input to a fully-connected linear layer trained for HAR. Importantly, the dataset is publicly available and includes the following human activities: preparing coffee, frying egg, no activity, showering, using microwave, washing dishes, washing hands, and washing teeth. The specific set of activities is representative of a standard home environment facilitating a wide range of applications. The performance offered by the proposed transfer learning-based framework surpasses the state of the art, while being able to be executed on mobile devices, such as smartphones, tablets, etc. In fact, the obtained model has been exported and thoroughly tested for real-time HAR on a smartphone device with the input being the audio captured from its microphone.
Audio Pattern Recognition; Machine Learning; Transfer Learning; Convolutional Neural Network; YAMNet; Human Activity Recognition
Settore INF/01 - Informatica
2023
https://www.scitepress.org/Papers/2023/116479/116479.pdf
Book Part (author)
File in questo prodotto:
File Dimensione Formato  
ICPRAM_2023_39_CR.pdf

accesso riservato

Tipologia: Post-print, accepted manuscript ecc. (versione accettata dall'editore)
Dimensione 2.68 MB
Formato Adobe PDF
2.68 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
116479.pdf

accesso aperto

Tipologia: Publisher's version/PDF
Dimensione 2.88 MB
Formato Adobe PDF
2.88 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/957085
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? ND
social impact