Emotion-Aware In-Car Feedback: A Comparative Study

Kevin Mwaita, Rahul Bhaumik, Aftab Ahmed, Adwait Sharma, Antonella De Angeli, Michael Haller

Research output: Contribution to journalArticlepeer-review


We investigate personalised feedback mechanisms to help drivers regulate their emotions, aiming to improve road safety. We systematically evaluate driver-preferred feedback modalities and their impact on emotional states. Using unobtrusive vision-based emotion detection and self-labeling, we captured the emotional states and feedback preferences of 21 participants in a simulated driving environment. Results show that in-car feedback systems effectively influence drivers’ emotional states, with participants reporting positive experiences and varying preferences based on their emotions. We also developed a machine learning classification system using facial marker data to demonstrate the feasibility of our approach for classifying emotional states. Our contributions include design guidelines for tailored feedback systems, a systematic analysis of user reactions across three feedback channels with variations, an emotion classification system, and a dataset with labeled face landmark annotations for future research.
Original languageEnglish
JournalMultimodal Technologies and Interaction
Early online date25 Jun 2024
Publication statusPublished - 25 Jun 2024

Data Availability Statement

The data presented in this study are available on request from the
corresponding author

Cite this