Methods for Evaluating Conversational Agents’ Communicability, Acceptability and Accessibility Degree

Recently, there has been an increasing interest in language-based interactions with technology. Driven by the success of intelligent personal assistants, the number of conversation-based interactions is growing in several domains. Nevertheless, the literature highlights a lack of models specifically studied to analyse communication blocks and the level of accessibility and acceptability by the user that can characterise Human-Agent dialogue. This paper aims to learn how much an agent can be accessible, how much the communication is understandable, and if it brings to a successful conclusion by extending two known models, the UTAUT2 and CEM. For both models, we defined new indicators to analyse communicability, acceptability degrees and accessibility level using WCAG guidelines. We designed two conversational agents to test our models and conducted preliminary tests. A first agent is used to assist students in following remote digital courses, and a second to help older people in their daily activities and to monitor the indexes of active life defined to control the trend of older people’s physical-cognitive state.