FUNCTIONAL STATISTICAL LEARNING METHODS APPLIED TO HUMAN EMOTION RECOGNITION FROM FACIAL VIDEOS

The ever-growing fascination with automatically analyzing and understanding human behavior has inspired a profound focus on the evolution of facial expressions and the recognition of corresponding emotions. By harnessing functional statistical learning methods, we develop a comprehensive methodology that capitalizes on the dynamic properties of continuity and evolvability inherent in functional data extracted from facial videos, which possess distinct properties compared to the static facial images predominantly used in traditional research methods. Our approach employs multivariate function-on-scalar regression models and functional analysis of variance (FANOVA) to effectively separate shared information from group-specific influences and individual noise through paired group comparisons, even with limited sample sizes. The identified group patterns convey significant mean characteristics in grouped units and are further utilized as prior knowledge for multi-classification in a streamlined feature space, generating emotional agreement scores for incoming new samples. Both non-parametric and parametric multi-class classification methods are employed to assess the predictive capabilities of the multivariates. In summary, we seamlessly integrate the entire pipeline for various stages of training and testing processes within the domain of explainable automatic emotion recognition, unveiling compelling results and offering insightful interpretations that may shed new light on emotions and expressions.