FACIAL EXPRESSION RECOGNITION AS A MEASURE OF USER-DESIGNER EMPATHY

In human-centered product design and development, understanding the users is essential. Empathizing with the user can help designers to gain deeper insights into the user experience and their needs. However, capturing real time empathy during user interaction and the degree to which empathy enhances user understanding remains unclear. To narrow this gap, this study aims to explore the use of facial expression recognition during a videotaped user interview as means to capture empathy. Mimicry and synchrony have been shown to be predictors of empathy in cognitive psychology. In this study, we adapt this method to 46 user-designer interviews. The results show that the user and designer show mimicry in their facial expressions, which indicates that affective empathy can be captured via facial recognition. However, we find that the user's facial expressions might not represent their actual emotional tone, which can mislead the designer. Further, we do not find a link between the observed mimicry of facial expressions and the understanding of mental contents, which hints that the affective and some cognitive parts of user empathy may not be directly connected. Further studies are needed to understand how facial expression analysis can further be used to study and advance empathic design.