Bridging AI Developers and End Users: an End-User-Centred Explainable AI Taxonomy and Visual Vocabularies

See Jin 2021 for a worked out prototyping framework that uses this taxonomy. This visualized framework is from Jin 2021:

 Jin2021-User-centered XAI framework.PNG

Premise: not enough attention paid to the "end users of AI, who may be lay persons or domain experts such as doctors, drivers, and judges." (1)

Literature survey that places "existing explanatory forms" into the "end-user-friendly XAI taxonomy" (1).

"It provides a practical toolkit for AI developers to define the explanation problem from a user-centred perspective, and expand the visualization space of explanations to develop more end-user-friendly XAI systems" (1)

Introduction

Majority of the XAI work is focused on AI experts for inspecting, debugging, visualizing AI models. Other end users are ignored. These other end users include:

  • lay persons
  • domain experts (doctors, bankers, judges, drivers)

Goal is to "bridge the gap between the XAI techniques and end uses" (1)

"End uses" is a bit ambiguous: more tailored towards end users or end use cases?

So on the side of XAI techniques, the paper's taxonomy only recognizes three forms of explanations based on:

  1. feature attribute
  2. instance
  3. decision rules/trees

which correspond "to the granularity of the learned representations of ML models at the feature level, instance level, and decision boundary level." (1)

On the side of "end uses" the taxonomy recognizes three levels:

  1. Counterfactual / contrastive
  2. Contextual
  3. Causal

Which are selected from properties of human(-friendly) explanations.

  • selected
  • causal
  • contrastive
  • counterfactual (as a preferred form of causal reasoning)
  • contextual, a social process (there are one and the same or two?); typically conversational

It is weird that the authors put forward counterfactual reasoning as the typical way of causal reasoning (which does make sense), but then strongly distinguish these two in the taxonomy.

Also, if doesn't becomes clear in the text itself how the "contextual" property is used.

The authors explicitly choose for "visualization as the primary explanation format in XAI" (1) which may actually be the constraining factor in what makes it into their taxonomy.

End-user-centred XAI taxonomy

Feature attribution

Authors indicate a first mapping. Explanations based on features relate to

  • causal explanation: humans typically interpret prominent features as having a causal effect on the prediction
  • contrastive explanation: see how the important features differ between instance and reference instance
  • counterfactual: using feature-sampling or perturbation

Two types of explanations distinguished:

  1. Signal (requires access to model internals)
  2. Model-agnostic

Visual vocabulary:

Image data:

  • Salience map
  • Masks
  • Segmentation maps
  • Bounding boxes

Tabular / text:

  • Plots (scatter, bar, box, waterfall, treemap, wrapped bars, packed bars, piled bars, Zvinca plots, tornado plot, violin plot, beeswarm plot)
  • 2D or 3D heatmaps for feature interactions
  • Feature-feature interactions: matrix heatmap, node-link network, contingency wheel.

Instance

  • Intuitive for humans if instances are themselves structural and interpretable (e.g. image or text)
  • Contextual: "carrier more contextual information where the typical or untypical/counterfactual features reside" (2)

    • Bit vague description to me!
  • Similar instances ( kNN, case-based reasoning )

  • Prototypes (k-mediods, MMD-critic )

No mention of visual vocabulary here, but I think this is because the assumption is that showing the data instances themselves is already intuitive, i.e. no "extra" visualization is needed other than displaying the data instance.

Decision-trees and rules

  • Causal: supports a causal chain of reasoning (but may be of a surrogate model so not model the actual causal chain)
  • Contrastive and Counterfactual: since the decision boundary is explicit, easy to generate contrastive or counterfactual instances

XAI methods:

  • Learning decision trees from black box models ( model distillation, disentangle model's representation )
  • Rule-learning (Bayesian Rule Lists, LORE, Anchors)

Visual vocabulary:

  • Decision tree ( node-link tree, treemap, cladogram, hyperbolic tree, dendrogram, flow chart )
  • Rule lists (IF-THEN text, matrix, table)

Discussion and conclusion

Why is this taxonomy user-centred?

"The essence of using them is to combine the proper vocabularies to generate a comprehensive explanation for various audiences and usage scenarios" (2)

The scope of the taxonomy is limited because it restricts itself to XAI forms deemed user-centered, i.e. methods that relate to desirable properties of human explanation and are associated with visualization techniques. This does not cover the whole field of XAI.

Work not evaluated with end-users.