Explainable Agents for Less Bias in Human-Agent Decision Making

Avleen Malhi*, Samanta Knapic, Kary Främling

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

8 Citations (Scopus)
32 Downloads (Pure)


As autonomous agents become more self-governing, ubiquitous and sophisticated, it is vital that humans should have effective interactions with them. Agents often use Machine Learning (ML) for acquiring expertise, but traditional ML methods produce opaque results which are difficult to interpret. Hence, these autonomous agents should be able to explain their behaviour and decisions before they can be trusted by humans. This paper focuses on analyzing the human understanding of the explainable agents behaviour. It conducts a preliminary human-agent interaction study to investigate the effect of explanations on the introduced bias in human-agent decision making for the human participants. We test the hypothesis where different explanation types are used to detect the bias introduced in the autonomous agents decisions. We present three user groups: Agents without explanation, and explainable agents using two different algorithms which automatically generate different explanations for agent actions. Quantitative analysis of three user groups (n = 20, 25, 20) in which users detect the bias in agents’ decisions for each explanation type for 15 test data cases is conducted for three different explanations types. Although the interaction study does not give significant findings, but it shows the notable differences between the explanation based recommendations and non-XAI recommendations in human-agent decision making.

Original languageEnglish
Title of host publicationExplainable, Transparent Autonomous Agents and Multi-Agent Systems - 2nd International Workshop, EXTRAAMAS 2020, Revised Selected Papers
EditorsDavide Calvaresi, Amro Najjar, Michael Winikoff, Kary Främling
Number of pages18
Publication statusPublished - 2020
MoE publication typeA4 Article in a conference publication
EventInternational Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems - Auckland, New Zealand
Duration: 9 May 202013 May 2020
Conference number: 2

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12175 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


WorkshopInternational Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems
Abbreviated titleEXTRAAMAS
Country/TerritoryNew Zealand


  • Explainable agents
  • Explanation type
  • Human-agent decision making
  • Human-agent interaction


Dive into the research topics of 'Explainable Agents for Less Bias in Human-Agent Decision Making'. Together they form a unique fingerprint.

Cite this