Human-in-the-loop machine learning is widely used in artificial intelligence (AI) to elicit labels for data points from experts or to provide feedback on how close the predicted results are to the target. This simplifies away all the details of the decision-making process of the expert. In this work, we allow the experts to additionally produce decision rules describing their decision-making; the rules are expected to be imperfect but to give additional information. In particular, the rules can extend to new distributions, and hence enable significantly improving performance for cases where the training and testing distributions differ, such as in domain adaptation. We apply the proposed method to lifelong learning and domain adaptation problems and discuss applications in other branches of AI, such as knowledge acquisition problems in expert systems. In simulated and real-user studies, we show that decision rule elicitation improves domain adaptation of the algorithm and helps to propagate expert's knowledge to the AI model.

Original languageEnglish
Title of host publication26th International Conference on Intelligent User Interfaces, IUI 2021
Number of pages5
ISBN (Electronic)9781450380171
Publication statusPublished - 14 Apr 2021
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Intelligent User Interfaces - Virtual, Online, College Station, United States
Duration: 13 Apr 202117 Apr 2021
Conference number: 26


ConferenceInternational Conference on Intelligent User Interfaces
Abbreviated titleIUI
Country/TerritoryUnited States
CityCollege Station


  • domain adaptation
  • human computer interaction
  • human-in-the-loop
  • knowledge elicitation
  • user studies


Dive into the research topics of 'Decision Rule Elicitation for Domain Adaptation'. Together they form a unique fingerprint.

Cite this