Recognizing the Emotions Evoked by Artworks Through Visual Features and Knowledge Graph-Embeddings

Recognizing the emotion an image evokes in the observer has long attracted the interest of the community for its many potential applications. However, it is a challenging task mainly due to the inherent complexity and subjectivity of human feelings. Such a difficulty is exacerbated in the domain of visual arts, mainly because of their abstract nature. In this work, we propose a new version of the artistic knowledge graph we were working on, namely ArtGraph, obtained by integrating the emotion labels provided by the ArtEmis dataset. The proposed graph enables emotion-based information retrieval and knowledge discovery even without training a learning model. In addition, we propose an artwork emotion classification system that jointly exploits visual features and knowledge graph-embeddings. Experimental evaluation revealed that while improvements in emotion classification depend mainly on the use of visual features, the prediction of style, genre and emotion can benefit from the simultaneous exploitation of visual and contextual features and can assist each other in a synergistic way.