In recent years, facial expression recognition (FER) has becoming a growing topic in computer vision with promising applications on virtual reality and human–robot interaction. Due to the influence of illumination, individual differences, attitude variation, etc., facial expression recognition with robust accuracy in complex environment is still an unsolved problem. Meanwhile, with the wide use of social communication, massive data are uploaded to the Internet; the effective utilization of those data is still a challenge due to noisy label phenomenon in the study of FER. To resolve the above-mentioned problems, firstly, a double active layer-based CNN is established to recognize the facial expression with high accuracy by learning robust and discriminative features from the data, which could enhance the robustness of network. Secondly, an active incremental learning method was utilized to tackle the problem of using Internet data. During the training phase, a two-stage transfer learning method is explored to transfer the relative information from face recognition to FER task to alleviate the inadequate training data in deep convolution network. Besides, in order to make better use of facial expression data from Web site and further improve the FER accuracy, Unconstrained Facial Expression Database from Web site database is built in this paper. Extensive experiments performed on two public facial expression recognition databases FER 2013 and SFEW 2.0 have demonstrated that the proposed scheme outperforms the state-of-the-art methods, which could achieve 67.08% and 51.90%, respectively.

Weakly supervised facial expression recognition via transferred DAL-CNN and active incremental learning / Y. Xu, J. Liu, Y. Zhai, J. Gan, J. Zeng, H. Cao, F. Scotti, V. Piuri, R. Donida Labati. - In: SOFT COMPUTING. - ISSN 1432-7643. - 24:8(2020 Apr), pp. 5971-5985. [10.1007/s00500-019-04530-1]

Weakly supervised facial expression recognition via transferred DAL-CNN and active incremental learning

F. Scotti;V. Piuri
Penultimo
;
R. Donida Labati
Ultimo
2020

Abstract

In recent years, facial expression recognition (FER) has becoming a growing topic in computer vision with promising applications on virtual reality and human–robot interaction. Due to the influence of illumination, individual differences, attitude variation, etc., facial expression recognition with robust accuracy in complex environment is still an unsolved problem. Meanwhile, with the wide use of social communication, massive data are uploaded to the Internet; the effective utilization of those data is still a challenge due to noisy label phenomenon in the study of FER. To resolve the above-mentioned problems, firstly, a double active layer-based CNN is established to recognize the facial expression with high accuracy by learning robust and discriminative features from the data, which could enhance the robustness of network. Secondly, an active incremental learning method was utilized to tackle the problem of using Internet data. During the training phase, a two-stage transfer learning method is explored to transfer the relative information from face recognition to FER task to alleviate the inadequate training data in deep convolution network. Besides, in order to make better use of facial expression data from Web site and further improve the FER accuracy, Unconstrained Facial Expression Database from Web site database is built in this paper. Extensive experiments performed on two public facial expression recognition databases FER 2013 and SFEW 2.0 have demonstrated that the proposed scheme outperforms the state-of-the-art methods, which could achieve 67.08% and 51.90%, respectively.
Deep convolutional neural network; Facial expression recognition; Two-stage transfer learning; Weakly supervised active incremental learning
Settore INF/01 - Informatica
apr-2020
29-nov-2019
Article (author)
File in questo prodotto:
File Dimensione Formato  
soco2020.pdf

accesso aperto

Tipologia: Pre-print (manoscritto inviato all'editore)
Dimensione 746.68 kB
Formato Adobe PDF
746.68 kB Adobe PDF Visualizza/Apri
Xu2020_Article_WeaklySupervisedFacialExpressi.pdf

accesso riservato

Tipologia: Publisher's version/PDF
Dimensione 2.06 MB
Formato Adobe PDF
2.06 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2434/790379
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 11
  • ???jsp.display-item.citation.isi??? 9
social impact