[Context and Motivation] Many recent studies highlight explainability as an important requirement that supports in building transparent, trustworthy, and responsible AI systems. As a result, there is an increasing number of solutions that researchers have developed to assist in the definition of explainability requirements. [Question] We conducted a literature study to analyze what kind of candidate solutions are proposed for defining the explainability requirements of AI systems. The focus of this literature review is especially on the field of requirements engineering (RE). [Results] The proposed solutions for defining explainability requirements such as approaches, frameworks, and models are comprehensive. They can be used not only for RE activities but also for testing and evaluating the explainability of AI systems. In addition to the comprehensive solutions, we identified 30 practices that support the development of explainable AI systems. The literature study also revealed that most of the proposed solutions have not been evaluated in real projects, and there is a need for empirical studies. [Contribution] For researchers, the study provides an overview of the candidate solutions and describes research gaps. For practitioners, the paper summarizes potential practices that can help them define and evaluate the explainability requirements of AI systems.
Original languageEnglish
Title of host publicationRequirements Engineering: Foundation for Software Quality
Subtitle of host publication30th International Working Conference, REFSQ 2024, Winterthur, Switzerland, April 8–11, 2024, Proceedings
EditorsDaniel Mendez, Daniel Mendez, Ana Moreira, Ana Moreira
Number of pages18
ISBN (Electronic)978-3-031-57327-9
ISBN (Print)978-3-031-57326-2
Publication statusPublished - 8 Apr 2024
MoE publication typeA4 Conference publication
EventInternational Working Conference on Requirements Engineering: Foundation for Software Quality - Winterthur, Switzerland
Duration: 8 Apr 202411 Apr 2024
Conference number: 30

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Working Conference on Requirements Engineering
Abbreviated titleREFSQ


  • Explainability Requirements
  • Explainable AI
  • AI Systems
  • Explainability Practices


Dive into the research topics of 'Candidate Solutions for Defining Explainability Requirements of AI Systems'. Together they form a unique fingerprint.

Cite this