From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practice | Morley, Floridi, Kinsey, Elhalal
Introduction
Because ML and deep learning are now so impactful, the AI ethics debate has also gone mainstream. But so far, it has focused on establishing principles ("what") rather than practices ("how").
Aim:
Our intention in presenting this research is to contribute to closing the gap between principles and practices by constructing a typology that may help practically-minded developers apply ethics at each stage of the Machine Learning development pipeline, and to signal to researchers where further work is needed. (2141)
Focused on ML, hopefully applicable to AI more broadly as well.
Dual advantage of AI: 1) capitalise on opportunities while 2) foreseeing and preventing or minimizing harm.
Map created in Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 205395171667967. https ://doi.org/10.1177/20539 51716 67967 9. is reference for this paper's typology
Content:
1) Explanation why there's been to much focus on principles rather than practices 2) Explanation methodology of planned research into creating "applied ethical AI typology" 3) Theoretical framework for interpreting results 4) Discussion focuses on "uncertain utility of the tools and methods identified as well as their uneven distribution" 5) A way forward: multi-disciplinary approach to create and evaluate tools and methods; ensure there's a "how" for every "what".
Moving from Principles to Practices
There's growing consensus in many documents on AI principles. A few principles are shared in more than half of the many documents.
Example: OECD principles
This paper summarizes them as follows:
1) Beneficence: "beneficial to, and respectful of, people and the environment" (2145) 2) Non-maleficence: "robust and secure" (2145) 3) Autonomy: "respectful of human values" (2145) 4) Justice: "fair" (2145) 5) Explicability: "explainable, accountable and understandable" (2145)
This consensus allows subsequent work on the "how".
This whole section is a justification of this paper but also of my own work! Making AI principles concrete also requires a technical effort (even though this is just part of the story):
The availability of these ‘agreed’ principles supports but does not yet bring about actual change in the design of algorithmic systems (Floridi 2019a). As (Hagendorff 2019) notes, almost all of the guidelines that have been produced to date suggest that technical solutions exist, but very few provide technical explanations. As a result, developers are becoming frustrated by how little help is offered by highly abstract principles when it comes to the ‘day job’ (Peters and Calvo 2019). This is reflected in the fact that 79% of tech workers report that they would like practical resources to help them with ethical considerations (Miller and Coldicott 2019). Without this more practical guidance, other risks such as ‘ethics bluewashing’14 and ‘ethics shirk ing’15 remain (Floridi 2019c). (1247)
Work to do!
This is likely to be hard work. The gap between principles and practice is large, and widened by complexity, variability, subjectivity, and lack of standardisation, including variable interpretation of the ‘components’ of each of the ethical principles (Alshammari and Simpson 2017). Yet, it is not impossible if the right questions are asked (Green 2018; Wachter et al. 2017) and closer attention is payed to how the design process can influence (Kroll 2018) whether an algorithm is more or less ‘ethically-aligned.’ (1247)
Interesting: this is an argument why it's also cost-beneficial to start closing the gap between principles and practices now already. TODO make a note about this. How to convince AI tech people ethics is worth it.
the lack of guidance may (a) result in the costs of ethical mistakes outweighing the benefits of ethical success (even a single critical ‘AI’ scandal could stifle innovation): (b) undermine public acceptance of algorithmic systems; (c) reduce adoption of algorithmic systems; and (d) ultimately create a scenario in which society incurs significant opportunity costs (Cookson 2018). Thus, the aim of this research project is to identify the methods and tools already available to help developers, engineers, and designers of ML reflect on and apply ‘ethics’ (Adamson et al. 2019) so that they may know not only what to do or not to do, but also how to do it, or avoid doing it (Alshammari and Simpson 2017). (1247)
Methodology
Goal: "design a typology, for the very practically minded ML community (Holzinger 2018), that would ‘match’ the tools and methods identified to the ethical principles (...)" (2147)
- Considering ethical principles at the different stages of algorithmic development
- Based in ICO auditing framework for AI
- Inspired by Saltz & Dewar framework. TODO read!
To create this typology, and inspired by Saltz and Dewar (2019) who produced a framework that is meant to help data scientists consider ethical issues at each stage of a project, the ethical principles were combined with the stages of algorithmic development outlined in the overview of the Information Commissioner’s Office (ICO) auditing framework for Artificial Intelligence and its core components,16 as shown in Table 3. The intention is that this encourages ML developers to go between decision and ethical principles regularly. (2148)
TODO: Ref Saltz & Dewar: Saltz, J. S., & Dewar, N. (2019). Data science ethical considerations: A systematic literature review and proposed project framework. Ethics and Information Technology. https ://doi.org/10.1007/s10676-019-09502-5.
- Identify tools and methods, and their makers, i.e. fill in the typology
- Initially interviews were done, but these mainly showed that no tools were really used. This however was not really evidence of a huge lack of tools, but more about the failure to apply them in real-use cases
- So in the end, a standard literature review was done.
- Review recommendations, theories, methodologies, tools, see where they fit in the typology
- High-level principles were translated in requirements. Inspired by EU_ethics_guidelines_thrustworthy_ai.md W.r.t. point 3, some references and concepts:
This approach is also used in the disciplinary ethical guidance produce for inter- net-mediated researchers by the Belmont Report (Anabo et al. 2019), and by La Fors et al. (2019) who sought to integrate existing design-based ethical approaches for new technologies by matching lists of values the practical abstraction from mid-level ethics (principles) to what (Hagendorff 2019) calls ‘microethics.’ This translation is a process that gradually reduces the indeterminacy of abstract norms to produce desiderata for a ‘minimum-viable-ethical-(ML)product’ (MVEP) that can be used by people who have various disciplinary backgrounds, interests and priorities (Jacobs and Huldtgren 2018). The outcome of this translation process is shown in Table 5.
- "match lists of values [from?- ejw] the practical abstraction" from mid-level ethics (principles) to what Hagendorff calls microethics. See Hagendorff-Ethics_of_AI_Ethics_Evaluation_of_Guidelines (zin loopt niet?!)
- MVEP: Minimum Viable Ethical Product
- Reference is to Jacobs and Huldtgren 2018, but I haven't found this term there.
Applied Ethics Typology
Review table 3. (2149)
On x-axis we have principles "Beneficence, Non-Maleficence, Autonomy, Justice, Explicability"
On "y-axis" of the table, we have the following columns, from left to right:
1) Business and use-case development * Problem/improvements are defined and use of AI is proposed 2) Design phase * The business case is turned into design requirements for engineers 3) Training and test data procurement * Initial data sets are obtained to train and test the model 4) Building * AI application is built 5) Testing * The system is tested 6) Deployment * When the AI system goes live 7) Monitoring * Performance of the system is assessed
Connecting high-level ethical principles to tangible system requirements
See table 5. Akin to EC's requirements for trustworthy AI.
Framing the Results
- Goal is not "tick-box" ethics, cf. Hagendorff.
- Interestingly, also a shift from prescriptive "ethics by design" to dialogic pro-ethical development.
- Find the right method and tool for the given context
- Nice idea to make a searchable database out of this.
Additionally, the purpose of presenting the typology is not to give the impression that the tools act as means of translating the principles into definitive ‘rules’ that technology developers should adhere to, or that developers must always complete one ‘task’ from each of the boxed. This only promotes ethics by ‘tick-box’ (Hagendorff 2019). Instead, the typology is intended to eventually be an online searchable database so that developers can look for the appropriate tools and methodologies for their given context, and use them to enable a shift from a prescriptive ‘ethics-by-design’ approach to a dialogic, pro-ethical design approach (Anabo et al. 2019; Floridi 2019b).
Discussion of Initial Results
Three highlights:
1) There is an overreliance on "explicability" 2) There is a focus on protecting the "individual", rather than protecting collectives 3) There is a lack of usability of the tools
Overreliance on explicability
Interesting point:
1) skewedness for tools w.r.t. principles 2) but also uneven distribution in terms of the phase of the design process * e.g. the tools for beneficence are almost all for in the planning stage * whereas "non-maleficent" is well covered in each phase
But specifically, the most skewed is explicability. But why is that actually?
1) "The first and simpler is that the ‘problem’ of ‘interpreting’ an algorithmic decision seems tractable from a mathematical standpoint, so the principle of explicability has come to be seen as the most suitable for a technical fix (Hagendorff 2019)." (2153) 2) "The second is that ‘explicability’ is not, from a moral philosophy perspective, a moral principle like the other four principles. Instead, it can be seen as a second order principle, that has come to be of vital importance in the ethical-ML community because, to a certain extent, it is linked with all the other four principles " (2154-2155)
So explicability is an instrumental value, which is also a main point of Canca-Operationalizing AI Ethics Principles.
But... is that a bad thing? Well... (and I fully agreed with this!):
However, as the majority of tools and methods that sit in the concentration at the intersection of explicability and testing are primarily statistical in nature, this would be a very mechanistic view because such ‘solutions’—e.g. LIME (Ribeiro et al. 2016), SHAP (Lundberg and Lee 2017), Sensitivity Analysis (Oxborough et al. 2018)—do not really succeed in helping developers provide meaningful explanations (Edwards and Veale 2018) that give individuals greater control over what is being inferred about them from their data. As such, the existence of these tools is at most necessary but not sufficient. (2155)
So in other words, when the other four ethical values are, in terms of tooling, so dominantly approached from the perspective of explainability; and when these "explanations" remain so technical and statistical, you obfusciate what is really needed from ethical AI, namely a socio-technical transparent system that is aligned with human values.
For this, we need a richer tooling landscape with dedicated tools for other ethical values in various design phases:
From a more humanistic, and realistic perspective, in order to satisfy all the five principles a system needs to be designed from the very beginning to be a transparent sociotechnical system (Ananny and Crawford 2018). To achieve this level of transparency, accountability or explicability, it is essential that those analysing a system are able to “understand what it was designed to do, how it was designed to do that, and why it was designed in that particular way instead of some other way” (Kroll 2018). This kind of scrutiny will only be possible through a combination of tools or processes that facilitate auditing, transparent development, education of the public, and social awareness of developers (Burrell 2016). As such, there should ideally be tools and methods available for each of the boxes in the typology, accepting that there may be areas of the typology which are more significant for ML practitioners than others. (2155)
Ultimately, it's about giving individuals more control. (In this regard I found the Twitter announcement on algorithmic choice interesting. What's the point of making the effect of an algorithm on what an end-user sees more transparent? Well, that they can then meaningfully adjust how the algorithm affects them! It's about agency and respecting human autonomy, that the algorithm follows the preferences and the goals of a human (rather than the human being force-fed whatever the algorithm feeds him or her).
This also taps into the question what an explanation for humans is. The field of XAI is not very well aligned with existing research into what constitutes a meaningful explanation for humans. See Mittelstadt, Russell, and Wachter - Explaining Explanations in AI .
For example, is giving a counter-factual explanation really enough for individuals trying to understand an AI's decision? Counter-factual explanations are one example of an "example-based" explanation in XAI. If you apply for a loan and you don't get it, is it sufficient to get back a hypothetical data point (which may be 100-dimensional) which would have been accepted? Or would it be sufficient to receive a number that expresses a degree of influence on the final outcome for a particular feature (given the single data point that is you, as an individual)? (See discussion of this example under "An individual focus" i.e. next section)
Another benefit of a more equally distributed tooling landscape, is that then you have the flexibility to prioritize certain types of values in certain contexts; instead of basically being limited to prioritizing explicability since that's where most of the tooling is available.
An individual focus
I find that the title of this section is not descriptive of the core of the matter. Lack of insight into and tooling for assessing/controlling the impact on users.
Main point:
The next observation of note is that few of the available tools surveyed provide meaningful ways to assess, and respond to, the impact that the data-processing involved in their ML algorithm has on an individual, and even less on the impact on society as a whole (Poursabzi-Sangdeh et al. 2018) (2156)
(See the relatively empty "deployment" column in the typology). Kind of makes sense though, because how would you capture human-machine interaction with tools? This is too bad though, because this is where a lot of the impact of AI in society happens.
Example of counter-factual explanations:
If such methods become the de facto means of providing explanations, the extent to which the ‘algorithmic society’ is interpretable to the general public will be very limited. And counterfactual explanations could easily be embraced by actors uninterested in providing factual explanations, because the counterfactual ones provide a vast menu of options, which may easily decrease the level of responsibility of the actor choosing it. For example, if a mortgage provider does not offer a mortgage, the factual reasons may be a bias, for example the gender of the applicant, but the provider could choose from a vast menu of innocuous, counterfactual explanations—if some variable x had been different the mortgage might have been provided—e.g., a much higher income, more collaterals, lower amount, and so forth, without ever mentioning the factual cause, i.e. the gender of the applicant. (2156)
Interesting remark: assumption that prediction=decision is used in practice. Very obscure!
For example, currently there is an assumption that prediction = decision, and little research has been done (in the context of ML) on how people translate predictions into actionable decisions. As such, tools that, for example, help developers pro-ethically design solutions that do not overly restrict the user’s options in acting on this prediction (i.e. tools that promote the user’s autonomy) are in short supply (Klein- berg et al. 2017). F users feel as though their decisions are being curtailed and con- trolled by systems that they do not understand, it is very unlikely that these systems will meet the condition of social acceptability, never mind the condition of social preferability which should be the aim for truly ethically designed ML (Floridi and Taddeo 2016).
Lack of usability
Most methods (e.g. several frameworks for developing ethical AI) are not actionable. In more concrete cases, there's code available, but realistically speaking these methods are only available to technical developers i.e. the skill barrier is very high (when the goal is understanding AI):
However, the vast majority of these tools and methods are not actionable as they offer little help on how to use them in practice (Vakkuri et al. 2019). Even when there are open-source code libraries available, documentation is often limited, and the skill-level required for use is high. (2157)
For the end-users of AI, this doesn't really do much at all! And for developers, "explaining" AI still takes a lot of effort, which kind of counteracts the whole point of "tooling" which is to make life easy. (Of course it does make the life of the developer significantly easier, but is it enough in terms of applied ethics?)
For example, if a client comes towards you with a complaint about a decision, would you really do a whole data analysis to "explain" the AI's prediction (and how the decision followed from that)?
This is a very problematic but to the point criticism... there is a severe lack of how to evaluate explainability and a positive societal impact, which really makes it harder to make people invest into tooling:
For example Binns (2018a) asks how an accountable system actually will be held accountable for an ‘unfair’ decision in a way that is acceptable to all. This makes it almost impossible to measure the impact, ‘define success’, and document the performance (Mitchell et al. 2019) of a new design methodology or tool. As a result, there is no clear problem statement (and therefore no clear business case) that the ML community can use to justify time and financial investment in developing much-needed tools and techniques that truly enable pro-ethical design. Consequently, there is no guaranatee that the so-called discursive devices do anything other than help the groups in society who already have the loudest voices embed and protect their values in design tools, and then into the resultant ML systems. (2157)
A way forward
Very interesting point.
Legal scholars or scholars in humanities or ethics or philosophy are used to working with concepts for which there is a myriad of definitions and contextual factors to take into account.
The ML developer community, in contrast, may be less used to dealing with this kind of difficulty, and more used to scenarios where there is at least a seem- ingly quantifiable relationship between input and output. As a result, the existing approaches to designing and programming ethical ML fail to resolve what (Arvan 2018) terms the moral-semantic trilemma, as almost all tools and methods highlighted in the typology are either too semantically strict, too semantically flexible, or overly unpredictable (Arvan 2018). (2158)
This section is a plea for a multi-disciplinary approach. Embrace uncertainty. Accept that AI is built on assumptions and that human behavior is incredibly complex. Realise that trade-offs are most likely not avoidable.
Goals:
- Tools and methods should not only reflect the plurality of values of developers, but of society.
- Tools should be probing and open to input and facilitate discussion
- Not think and then code, but think and code.
- In other words, it will accelerate the move away from the ‘move fast and break things’ approach towards an approach of ‘make haste slowly’ (festina lente) (Floridi 2019a).
- Cf. I think it was Raji et al. who said: "Hey this accountability framework is laborous and will slow development down, but that's actually a good thing"
- Avoid that ethical principles are "written into the business and use-case, but coded out by the time a system gets to deployment" (2158)