Principles and Practice of Explainable Machine Learning | Belle, Papantonis
Specific focus on machine learning.
Here, we have undertaken a survey to help industry practitioners (but also data scientists more broadly) understand the field of explainable machine learning better and apply the right tools. Our latter sections build a narrative around a putative data scientist, and discuss how she might go about explaining her models by asking the right questions. (1)
including the principles that allow us to study transparent models vs opaque models, as well as model-specific or model-agnostic post-hoc explainability approaches. We also briefly reflect on deep learning models, and conclude with a discussion about future research directions. (1)
Introduction
Identifies target audiences and their concerns / explanation goals:
- Data scientists: debugging, improving performance
- Business owners: fit of model with business strategy and purpose
- Model Risk analysts: check robustness, approve for deployment
- Regulators: inspect reliability, assess impact on users
- Consumers: transparency of decisions, can they impact them?
General concerns:
- Correctness
- Robustness
- Bias
- Improvement
- Transferability
- Human comprehensibility
Scope
Data-driven methods, particularly machine learning and pattern recognition models, with primary goal of classification or prediction on statistical association.
Explainability
All aspects here are very similar to the ones outlined by Barredo Arrieta (2020) et al. . This paper cites Barredo Arrieta so I think the former is the more original one.
Transparency
- Simulatability
- Decomposability
- Algorithmic Transparency
Evaluation
- Comprehensibility
- Fidelity
- Accuracy
- Scalability
- Generality
Types of explanation
- Text explanation
- Visual explanation
- Local explanations
- Explanations by example
- Explanations by simplification
- Feature relevance explanations
Exploring XAI: 5 "Frameworks"
The authors summarize the XAI literature through five "frameworks" which "can be thought of as a comparative exposition and/or visualization of sorts" (5).
Goal is to understand:
- limitations of models deployed in industry
- available approaches for such models
- alternative models with better explainability
Taxonomy framework

The provided taxonomy serves as a "master" framework for the other frameworks,which elaborate on specific aspects:
we arrange models in terms of the kinds of explainability that are enabled, to be seen as a taxonomy. The subsequent frameworks will be based on this taxonomy, and can be seen as elaborations on the distinction between transparent and opaque ML models (Transparency framework), followed by a description of the capabilities of explainability approaches (XAI Capability framework). (6)
Transparency framework
Comparison of ML models by
- Simulatability
- Decomposability
- Algorithmic Transparency
- Possibly: required post-hoc methods
XAI capability framework
Comparison of XAI methods by:
- Swapping
- Explanation type
- Model agnostic yes/no
- Data: categorical/continuous
- Intermediate transformation yes/no
- Independent features yes/no
- Shapley values yes/not necessarily/no
- Examples yes/no
Explanation type framework
"contrast the types of post-hoc explanations at a conceptual level: for example, what might local explanations offer in contrast to model simplification strategies" (7)
Data Scientist Strategy Framework
"we motivate a narrative for a putative data scientist, Jane, and discuss how she might go about explaining her models by asking the right questions. We recommend a simple strategy and outline sample questions that motivate certain types of explanations." (8)
Skipped
Specific explanations of all framework components skipped.
Future directions
As we mentioned before, recent research has showcased how a number of popular, widely used, XAI approaches are vulnerable to adversarial attacks [74]. Information like that raises questions about whether the outcome of a XAI technique should be trusted or it has been manipulated. In addition, other related issues about the fitness of some of the proposed techniques to address general explainability can be found in the literature [50]. (21)
Another line of research that has recently gained traction is about designing hybrid models, combining the expressiveness of opaque models with the clear semantics of transparent models, as in [62], where linear regression is combined with neural networks, for example. This direction could not only help bridge the gap between opaque and transparent models, but could also aid the development of state-of-the-art performing explainable models. (21)
Finally, as XAI matures, notions of causal analysis should be incorporated to new approaches [59, 65]. This is already a major driver in fundamental problems in other areas, such as fairness and bias in machine learning [27, 51], so we should expect it to play an integral part in the future of the XAI literature. (21)