With technological developments, individuals are increasingly able to delegate tasks to autonomous agents that act on their behalf. This may cause individuals to behave more fairly, as involving an agent representative encourages individuals to strategise ahead and therefore adhere to social norms of fairness. Research suggests that an audio smiling agent may further promote fairness as it provides a signal of honesty and trust. What is still unclear is whether presenting a multimodal smiling agent (by using visual and auditory cues) rather than a unimodal smiling agent as normally available commercially (using only an auditory cue e.g., Siri) could amplify the impact of smiles. In the present study, participants (N = 86) played an ultimatum game either directly with another player (control), through a smiling multimodal and unimodal agent or through a neutral multimodal and unimodal agent. Participants’ task was to offer a number of tickets to the other player from a fixed amount. Results showed that when playing the ultimatum game through a smiling multimodal agent, participants offered more tickets to the other player compared to the control condition and the other agent conditions. Hence, exploiting multisensory perception to enhance an agent’s expression may be key for increasing individuals' pro-social behaviour when interacting through such an agent.
Original languageEnglish
Pages (from-to)65-77
Number of pages13
JournalJournal on Multimodal User Interfaces
Issue number2
Early online date23 May 2023
Publication statusPublished - 30 Jun 2023


  • Agent-mediated decision making
  • Autonomous agents
  • Fairness
  • Multisensory expression
  • Smiling
  • Ultimatum game

ASJC Scopus subject areas

  • Signal Processing
  • Human-Computer Interaction


Dive into the research topics of 'Investigating the influence of agent modality and expression on agent-mediated fairness behaviours'. Together they form a unique fingerprint.

Cite this