Explaining Human Activities Instances Using Deep Learning Classifiers

The recognition of human activities in sensorized smart-home environments enables a wide variety of healthcare applications, including the detection of early symptoms of cognitive decline. The most effective Human Activity Recognition (HAR) methods are based on supervised Deep Learning classifiers. Those models are usually considered as black boxes, and the rationale behind their decisions is difficult to understand for human beings. The recent advances in eXplainable Artificial Intelligence (XAI) offer promising tools to make HAR models more transparent. The state-of-the-art explainable HAR methods provide explanations for the output of classifiers that periodically predict the performed activity on short time windows (usually in the range of 15-60 seconds). However, non-technical users may be more interested in investigating explanations associated with complete activity instances (e.g., an instance of the cooking activity may last 30 minutes). Unfortunately, temporally extending the time window harms the recognition rate of HAR classifiers. In this paper, we propose DeXAR++: a novel method that generates explanations for human activity instances based on deep learning classifiers. The sensor data time windows used for classification are encoded as images. DeXAR++ aggregates the explanations generated by a computer-vision XAI approach on each time window to obtain a single explanation for approximated activity instances. Moreover, DeXAR++ includes a novel visualization approach particularly suitable for non-expert users. We evaluate DeXAR++ with both automatic and user-based evaluation methodologies on a public dataset of activities performed in smart-home environments, showing that our results outperform the ones obtained by state-of-the-art methods.