Novel Methods Using Human Emotion and Visual Features for Recommending Movies

Abstract

This master thesis investigates novel methods using human emotion as contextual information to estimate and elicit ratings when watching movie trailers. The aim is to acquire user preferences without the intrusive and time-consuming behavior of Explicit Feedback strategies, and generate quality recommendations. The proposed preference-elicitation technique is implemented as an Emotion-based Filtering technique (EF) to generate recommendations, and is evaluated against two other recommendation techniques. One Visual-based Filtering technique, using low-level visual features of movies, and one Collaborative Filtering (CF) using explicit ratings. In terms of \textit{Accuracy}, we found the Emotion-based Filtering technique (EF) to perform better than the two other filtering techniques. In terms of \textit{Diversity}, the Visual-based Filtering (VF) performed best. We further analyse the obtained data to see if movie genres tend to induce specific emotions, and the potential correlation between emotional responses of users and visual features of movie trailers. When investigating emotional responses, we found that \textit{joy} and \textit{disgust} tend to be more prominent in movie genres than other emotions. Our findings also suggest potential correlations on a per movie level. The proposed Visual-based Filtering technique can be adopted as an Implicit Feedback strategy to obtain user preferences. For future work, we will extend the experiment with more participants and build stronger affective profiles to be studied when recommending movies.