Improvements of robustness of speech recognition is one of the hottest topics in speech signal processing, particularly when applied within a noisy environment. Most of the research efforts focused in combining audio and visual data to implement an audio-visual speech recognition (AVSR) system. Bimodal approach demonstrated that a superior performance can be gained compared to the separate audio or visual approach. This paper proposes a fuzzy logic-based data fusion method that combines the recognition capabilities of two independent working systems namely the automatic speech recognition system (ASR) and the automatic visual recognition system (AVR). The main purpose is to boost the whole system’s performance keeping the ASR separate from the AVR. This approach provides a powerful method that enables simpler data fusion at decision level rather than the more complex at data and features level. Such complexity is also lowered due to the fuzzy logic-based implementation of the data fusion engine. Preliminary experimental results confirms the proposed approach.

Audio-visual fuzzy fusion for robust speech recognition / M. Malcangi, K. Ouazzane, P. Patel - In: The 2013 international joint conference on neural networks : august 4–9, 2013, Fairmont Hotel, Dallas, Texas, USA : final programPiscataway : IEEE, 2013. - ISBN 9781467361286. - pp. 582-589 (( convegno International Joint Conference on Neural Networks tenutosi a Dallas, Texas, USA nel 2013 [10.1109/IJCNN.2013.6706789].

Audio-visual fuzzy fusion for robust speech recognition

M. Malcangi;
2013

Abstract

Improvements of robustness of speech recognition is one of the hottest topics in speech signal processing, particularly when applied within a noisy environment. Most of the research efforts focused in combining audio and visual data to implement an audio-visual speech recognition (AVSR) system. Bimodal approach demonstrated that a superior performance can be gained compared to the separate audio or visual approach. This paper proposes a fuzzy logic-based data fusion method that combines the recognition capabilities of two independent working systems namely the automatic speech recognition system (ASR) and the automatic visual recognition system (AVR). The main purpose is to boost the whole system’s performance keeping the ASR separate from the AVR. This approach provides a powerful method that enables simpler data fusion at decision level rather than the more complex at data and features level. Such complexity is also lowered due to the fuzzy logic-based implementation of the data fusion engine. Preliminary experimental results confirms the proposed approach.
audio/visual automatic speech recognition ; fuzzy fusion ; speech-to-phoneme ; lips-to-viseme; speech-to-text
Settore INF/01 - Informatica
IEEE Computational Intelligence Society
International Neural Networks Society
Book Part (author)
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/225908
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 0
social impact