Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI | Barredo Arrieta et al., Information Fusion 58 (2020), 82-115
Great survey of current XAI techniques. Includes two comprehensible taxonomies of XAI methods:
- Taxonomy of recent contributions related to XAI and various ML methods
- Separate taxonomy specifically for deep learning
Goal:
Our ultimate goal is to provide newcomers to the field of XAI with a thorough taxonomy that can serve as reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability. (82)
1. Introduction
So the focus of the taxonomy is on XAI methods and specific models. Is it correct that the main audience then is XAI researchers?
Grounded on a first elaboration of concepts and terms used in XAI-related research, we propose a novel definition of explainability that places audience ( Fig. 2 ) as a key aspect to be considered when explaining a ML model. We also elaborate on the diverse purposes sought when using XAI techniques, from trustworthiness to privacy awareness, which round up the claimed importance of purpose and targeted audience in model explainability (83)
So their definition of explainability (see section 2.1, 2.2) revolves around the target audience ánd also factors in the purpose of explanation. The question for me at this point is: does the user also appear in the taxonomy itself?
Linking target audience and goals
Figure 2 is a diagram relating target audiences to explanation goals. It is inspired by an IBM publication: F. Rossi, AI Ethics for Enterprise AI, 2019.
- Domain experts/expert users (e.g. medical doctor)
- Goal: Trust the model, gaining scientific knowledge
- Regulatory entities/agencies
- Goal: Certify model compliance with legislation, auditing ...
- Managers and executive board members
- Assess regulatory compliance, understanding corporate applications of AI
- Data scientists, developers, product owners
- Ensure/improve product efficiency, research, explore new functionalities
- (End) users affected by the model decisions
- Verify fairness of decisions, understand decisions ...
2. Explainability: What, why, how
Definition of XAI proposed:
Given an audience, an explainable Artificial Intelligence is one that produces details or reasons to make its functioning clear or easy to understand. (85)
This definition ... "implicitly assumes that the ease of understanding and clarity targeted by XAI techniques for the model at hand reverts on different application purposes, such as a better trustworthiness of the model’s output by the audience." (85)
But again, the question is: does the taxonomy also help mapping XAI techniques onto particular user requirements?
Table 1 does contain a list of XAI Goals and their main target audience, with a set of references.
Goals:
Unfortunately, scarce contributions have attempted to define such goals from a conceptual perspective [5,13,24,30]. (85)
Main distinction: transparent vs. post-hoc
Further distinguishes within transparent:
- Algorithmic Transparency
- Decomposability
- Simulatability
Post-hoc:
- Text explanations
- Visualizations
- Local explanations
- Explanations by example
- Explanations by simplification
- Feature relevance
3. Transparent machine learning models
Similar to the ones pointed out by Molnar.
4. Post-hoc explainability techniques for machine learning
Overview of specific techniques. See paper. Two taxonomies:
Taxonomy 1
Taxonomy 1: "contributions dealing with post-hoc explainability of shallow ML models, which collectively refers to all ML models that do not hinge on layered structures of neural processing units ( Section 4.2 );" (92)
- Data type: Blue (image), green (text), red (tabular)
- Structure:
- (Post-hoc / Transparent)
- For post-hoc -> (Model-Agnostic / Model-specific)
- For model-specific -> ( Model type -> Explanation category -> XAI method + refs )
- For model-agnostic -> ( explanation category -> XAI method + refs )
Taxonomy 2
Taxonomy 2: "techniques devised for deep learning models, which correspondingly denote the family of neural networks and related variants, such as convolutional neural networks, recurrent neural networks ( Section 4.3 ) and hybrid schemes encompassing deep neural networks and transparent models." (92)
5. XAI: opportunities, challenges and future research
- See trade-off explainability and accuracy in AI
-
Summarizes existing metrics and measures, but emphasizes the need for more unified metrics of explainability
-
TODO Challenges for DL
- TODO connection of XAI and adversial machine learning!
- TODO XAI and uncertainty in the output
Guidelines for ensuring interpretable AI models
See D. Leslie, Understanding artificial intelligence ethics and safety, 2019, doi: 10.5281/zenodo.3240529
- Consider three factors: Contextual factors, potential impacts, domain-specific needs
- Understanding of the purpose of the AI
- Which level of complexity of explanation is required by the target audience
- Understand performance and interpretability of existing technology, models and methods (serves as a reference)
- Based on previous points and understanding of the problem at hand, only choose models with less interpretability when this is required by the computational complexity of the task at hand; i.e. prefer interpretable methods where possible.
- In the case of black box models, weigh ethics-, fairness-, and safety-related concerns
- "detailed articulation, examination and evaluation of applicable explanatory strategies" (102)
- "analysis of whether the coverage and scope of the available explanatory approaches match the requirements of the domain and application context where the model is to be deployed" (103)
- "formulation of an interpretability action plan that sets forth the explanation delivery strategy, including a detailed time frame for the execution of the plan, and a clearance of the roles and responsibilities of the team involved in the workflow" (103)
- "Rethink interpretability in terms of the cognitive skills, capacities and limitations of the individual human" (103)
- To a large part this mean you need to study human mental models to get a firmer grasp on how to deliver explanations and make them as accessible as possible
- I'd still this is still very much lacking.