Transparency in Artificial Intelligence | Stefan Larsson and Fredrik Heintz, Internet Policy Review, Vol. 9 Issue 2.
- conceptual distinction transparency in AI and algorithmic transparency
- transparency is a multifaceted concept
- transparency "expresses a conceptual metaphor of more general significance"
- proposal of possible categorisation
There's a bit of ambiguity in terms like "algoritmic transparency", because they often also include broader concerns than the algorithmic technological aspect alone.
Most problems will also not be visible from the perspective of the code alone:
The challenge is one of relations between data and algorithms, emergent properties of the machine learning process, very likely to be unidentifiable from a review of the code. This also means that it is important to consider the context of the combination of machine learning algorithms, the underlying training data and the decisions they inform (Kemper & Kolkman, 2019) (p.3)
The authors thus argue that a more systemic approach to AI transparency makes more sense.
Historical and Conceptual Development of Transparency
Steady increase in the appearance of the concept in papers from 1990 on, but interestingly enough, not really used before that! Probably in response to economic-political scandals that caused a plea for more governance. Also related to information assymetries.
Note that in some disciplines "transparency" actually refers to the physical property.
The Metaphorical Framing of Transparency
the benign conception of transparency relates to a deeper cognitive frame linked to knowing and understanding. (p.5)
Understanding is often framed in terms of "light, brightness, clarity - and transparency" (p.5)
The opposite also holds: the obscureness of the "black box" terminology and "black box society" (Pasquale).
Transparency is effectively used as a normative ideal that can steer behavior.
Because transparency has such a positive metaphorical framing, Koivisto (2016) argues that its negative connotations are not researched enough.
Neighbouring Concepts - Openness and Explainability
'Openness' is also framed with positive values. Think "open data", "open source", "open access".
In some cases, "transparency" runs up against (structural) limits:
transparency as a policy approach is “subsumed by a discourse of consumer empowerment that has been rendered meaningless in the contemporary environment of pervasive commercial surveillance” (Crain, 2018, p. 89). (p.7)
The "explainability" notion is more narrow than transparency:
This xAI-notion of transparency is narrower and more algorithmic model-oriented than for example the necessary transparency (and “explicability”) expressed by AI HLEG (2019) to achieve an ethically sound and trustworthy AI.
Multidisciplinary
Interesting remark w.r.t. the assumptions behind explainability and the relation to transparency and trust.
Why is explainable AI developed? Literature mentions:
- Interpretability
-
Trust
-
Assumption: w.r.t. transparency one has to consider how people understand explanations.
- So arguably AI systems are not used in practice partly because people do not trust them enough
- XAI could then help people understand systems, so they can in turn trust them
Interesting: XAI literature usually "does not cite or build on explanatory frameworks based in social science" (p. 7)
As the opening of the ‘black box’ may bring a number of challenges of a legal, technological and conceptual nature, suggested by Wachter, Mittelstadt and Russell (2017), the notion of transparency in AI – as applied on markets and interacting with humans and institutions – could benefit from a wider definition than the more narrowly defined xAI (see Mittelstadt, Russell, & Wachter, 2019).
XAI could be extended with:
- legal aspects of proprietorship
- avoid abuse
- too much openness can be an incentive to game the system; "perversive effects of disclosure" (p. 8; see Laat (2018))
- data and algorithm user literacy
- roles of symbols and metaphors for communication / explanation
As concluded by Mittelstadt, Russell and Wachter (2019), there is a fundamental distinction between explainability models and explanations in philosophy and sociology, that is, everyday explanations for humans are contrastive, selective, and social (Miller, 2017), which is not the same as the “interpretability” and explainability found in the xAI domain. (p.8)
- complexity of data ecosystems and the trading of consumer-data
- Does the GDPR "right to an explanation" lead to a "transparency fallacy"? (Edwards & Veale, 2017)
- the challenges of distributed personalized outcomes of AI systems, which obscures their effects for oversight
Ethical and legal relevance of transparency in AI
Trend: emergence of "AI ethics guidelines" with a strong emphasis on transparency:
Jobin et al. (2019) mapped and analysed the current corpus of principles and guidelines on ethical AI. They conclude that of the 84 analysed guidelines, 88% have been published after 2016 and that the most common concept argued for is ‘transparency’. (p.9)
Criticism on GDPR's "right to explanation", where there is unclarity about when it can be triggered, as well as the following more fundamental point about what constitutes an "explanation" in this sense:
Edwards and Veale (2017) argue that even if it was a clear right warranted by the GDPR, the legal conception of explanations as “meaningful information about the logic of processing” may not be provided by the kind of machine learning explanations computer scientists have developed in response (compare to point 4 above).
Conclusion
Transparency in AI, consequently, can best be seen as a balancing of interests and a governance challenge demanding multidisciplinary development to be adequately addressed. (p. 10)
For further reading
- Seven checkpoints for transparency: Koene et al., 2019
Koene, A., Clifton, C., Hatada, Y., Webb, H., & Richardson, R. (2019). A governance framework for algorithmic accountability and transparency (Study No. PE 624.262) Panel for the Future of Science and Technology, Scientific Foresight Unit (STOA), European Parliamentary Research Service.
- One the usefulness of explanations of machine learning models and predictions: Mittelstadt, Russell and Wachter (2019)
Mittelstadt, B., Russell, C., & Wachter, S. (2019). Explaining explanations in AI. Proceedings of the conference on fairness, accountability, and transparency - FAT* ‘19, 279–288. https://doi.org/10.1145/3287560.3287574
- Challenges caused by opening the black box: Wachter, Mittelstadt and Russell (2017)
Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2). https://jolt.law.harvard.edu/assets/articlePDFs/v31/Counterfactual-Explanations- without-Opening-the-Black-Box-Sandra-Wachter-et-al.pdf
- GDPR (already read)
Wachter, S., Mittelstadt, B., & Floridi, L. (2017). Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation. International Data Privacy Law, 7(2), 76–99. https://doi.org/10.1093/idpl/ipx005
- de Laat 2018: perverse effects of disclosure.
de Laat, P. B. (2018). Algorithmic decision-making based on machine learning from Big Data: Can transparency restore accountability? Philosophy & technology, 31(4), 525–541. https://doi.org/10.1007/s13347-017-0293-z
- Does the GDPR "right to an explanation" lead to a "transparency fallacy"? (Edwards & Veale, 2017)
Edwards, L., & Veale, M. (2017). Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for. Duke Law & Technology Review, 16(1), 18–84. https://scholarship.law.duke.edu/dltr/vol16/iss1/2