Nudging Towards Health in a Conversational Food Recommender System Using Multi-Modal Interactions and Nutrition Labels
Castiglia, Giovanni; el Majjodi, Ayoub; Starke, Alain Dominique; Narducci, Fedelucio; Deldjoo, Yashar; Calò, Federica
Chapter
Published version
![Thumbnail](/bora-xmlui/bitstream/handle/11250/3040178/KaRS_22.pdf.jpg?sequence=6&isAllowed=y)
Åpne
Permanent lenke
https://hdl.handle.net/11250/3040178Utgivelsesdato
2022Metadata
Vis full innførselSamlinger
Originalversjon
In: 4th Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys 2022Sammendrag
Humans engage with other humans and their surroundings through various modalities, most notably speech, sight, and touch. In a conversation, all these inputs provide an overview of how another person is feeling. When translating these modalities to a digital context, most of them are unfortunately lost. The majority of existing conversational recommender systems (CRSs) rely solely on natural language or basic click-based interactions. This work is one of the first studies to examine the influence of multi-modal interactions in a conversational food recommender system. In particular, we examined the effect of three distinct interaction modalities: pure textual, multi-modal (text plus visuals), and multi-modal supplemented with nutritional labeling. We conducted a user study (𝑁=195) to evaluate the three interaction modalities in terms of how effectively they supported users in selecting healthier foods. Structural equation modelling revealed that users engaged more extensively with the multi-modal system that was annotated with labels, compared to the system with a single modality, and in turn evaluated it as more effective.