The Ethics of AI Ethics: An Evaluation of Guidelines | Thilo Hagendorff, Minds and Machines (2020) 30:99-120

  • Overview AI Ethics by evaluation of 22 AI guidelines: overlaps and omissions
  • How are "ethical principles and values implemented in the practice of research, development and application of AI systems" (99)
  • How can the "effectiveness in the demands of AI ethics" (99) be improved?

Introduction

This paper departs from the realization that in AI ethics a large amount of ethical guidelines is being created, but that they do not have a large impact in human decision-making in AI and machine learning.

Advice is given to "overcome the relative ineffectiveness of these guidelines" (99).

Main issue: ethics in general (so also AI ethics) "lacks mechanisms to reinforce its own normative claims" (99) outside of reputational damage or repercussions by professional bodies.

Hagendorff argues that this relative weakness is actually attractive to companies, because it allows them to suggest that self-regulation is sufficient (i.e. no truly binding legislations are needed).

When companies or research institutes formulate their own ethical guidelines, regularly incorporate ethical considerations into their public relations work, or adopt ethically motivated “self-commitments”, efforts to create a truly binding legal framework are continuously discouraged. (100)

  • "may serve to pretend that accountability can be devolved from state authorities and democratic institutions upon the respective sectors of science or industry." (100)
  • "ethics can also simply serve the purpose of calming critical voices from the public, while simultaneously the criticized practices are maintained within the organization." (100)

In this cynical light, ethics guidelines are then a nice way to calm the nerves of criticasters without actually changing much. How do we instead make sure ethics guidelines are effective in practice?

Guidelines in AI Ethics

This paper mainly deals with comprehensive AI guidelines, rather than guidelines for specific applications.

Overlaps / Multiple entries

  • In 80% of the guidelines accountability, privacy and fairness are mentioned
  • Interestingly, these are the ones where the availability of technical solutions is relatively high

  • Accountability and explainable AI

    • Mittelstadt, B., Russell, C., Wachter, S. (2019). Explaining explanations in AI. In Proceedings of the conference on fairness, accountability, and transparency—FAT* ‘19 (pp. 1–10)
  • Fairness and discrimination aware data mining
    • Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumeé, III, H., Crawford, K. (2018). Datasheets for datasets. arXiv (pp. 1–17).
    • Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society, 4(2), 1–17.
    • Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., Vertesi, J. (2018). Fairness and abstrac- tion in Sociotechnical Systems. In ACT conference on fairness, accountability, and transparency (FAT) (vol. 1, No. 1, pp. 1–17).
  • Privacy
    • Baron, B., Musolesi, M. (2017). Interpretable machine learning for privacy-preserving pervasive systems. arXiv (pp. 1–10).

The FAT community apparently has a strong focus on technical solutions.

Overview available tools should be somewhere in:

  • Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., West, S. M., Richardson, R., Schultz, J., Schwartz, O. (2018). AI now report 2018 (pp. 1–62).

In contrast, almost no guideline talks about AI in contexts of care, nurture, help, welfare, social responsibility or ecological networks (103)

The EU_ethics_guidelines_thrustworthy_ai.md do to some extent, however.

Hagendorff argues that many AI guidelines take a "male" perspective because they are focused on technical solutions and follow an "ethics of justice", versus a female perspective that talks about contexts of application (see quote above) and the networks in which AI is embedded, following an "ethics of care" (103).

See the reports of AI Now (references at bottom of p. 103) as an example of the "female" approach.

(What's a bit weird about the "male vs. female" argument here, is that first Hagendorff computes that actually the proportion of women in these committees is practically balanced with 41.7%, but then Hagendorff seems to actively look for problems by excluding the committes of AI Now, as if they don't count! Then the balance is 31.3% women, apparently).

The following point is more interesting:

Finally, yet importantly, it is noteworthy that almost all guidelines sug- gest that technical solutions exist for many of the problems described. Nevertheless, there are only two guidelines which contain genuinely technical explanations at all—albeit only very sparsely. The authors of the guideline on the “Malicious Use of AI” provide the most extensive commentary here (Brundage et al. 2018).

Omissions

Some omissions are obviously due to the selection of guidelines, which focuses on generic guidelines and for example not on specific guidelines for robotics.

Some omissions are easily justified while others seem to be overall excluded without good justification.

Unjustified exclusion:

  • Political abuse
    • "in the context of automated propaganda, bots, fake news, deepfakes, micro targeting, election fraud, and the like." (105)
  • "hidden" social and ecological costs of AI systems
    • why hidden in quotation marks?
    • Importance of AI for a sustainable society
    • But usage context of AI may contradict this: lithium mining, e-waste, one-way use of minerals, energy consumption, low-wage clickworkers labeling datasets, content moderation (see p. 105 for references)
  • lack of diversity within the AI community
  • robot ethics
  • how algorithmic decision relates to human decision-making (e.g. superior?)
  • "Finally, yet importantly, not a single guideline raises the issue of public–private partnerships and industry-funded research in the field of AI" (106)
    • AI research, also at universities, is heavily funded by corporate partners and this may threaten independent research.

Justifiable exclusion:

  • Deliberations on AGI (speculative)
  • Deliberations on machine consciousness (speculative)

Hagendorff argues these more speculative things could be included in guidelines in terms of preventing certain research directions. He also remarks that the people caring about the above speculative topics are generally from a background where they do not engage hands-on with AI technology.

AI in Practice

Business Versus Ethics

A critical look at this global AI market and the use of AI systems in the economy and other social systems sheds light primarily on unwanted side effects of the use of AI, as well as on directly malevolent contexts of use. (106)

Many examples where AI in practice can go wrong:

  • US military invests 2 billion dollars in cyber warfare and weaponized AVs
  • Governments
    • automated propaganda and disinformation campaigns
    • social control
    • surveillance
    • facial recognition
    • sentiment analysis
    • social sorting
    • improved interrogation techniques
  • Companies:
    • Job loss by companies using AI
    • unmonitored AI experiments on society without consent
    • impact of data breaches
    • unfair, biased algorithms
    • unsafe AI products
    • "use trade secrets to disguise harmful or flawed AI functionalities" (106; refers to Whittaker 2018; no idea what this would mean!)
    • AI rush to put immature products on market
  • Criminals
    • AI for cyberattacks
    • stealing information
    • attacking IT infrastructure
    • rig elections
    • fraud using deepfakes or voice synthesis
    • use ML to find personal traits

AI Race

"AI race" between China, USA and Europe.

The danger is that in this rhetoric of "us vs. them" ethical considerations will be seen as impediments (that are to be removed).

L.O.L. this is the first time I've seen a Derrida reference in an AI related article. He cites the whole of Of Grammatology though... How am I supposed to find the relevant passage?

Ethics, on the other hand, in its considerations and theories always stresses the danger of an artificial differentiation between in- and outgroups (Derrida 1997).

But Hagendorffs point is valid: thinking in terms of an arms race is detrimental for the responsible development of beneficial and safe AI.

Ethics in practice

The following study did empirical research as to whether reading ethical guidelines made any difference on the conduct of professionals in scenarios requiring ethically-relevant decision-making:

The results are disillusioning: “No statistically significant difference in the responses for any vignette were found across individuals who did and did not see the code of ethics, either for students or for professionals.” (McNamara et al. 2018, 4). (p. 108)

On a more general level, ethics is often pitted against economical concerns in practice:

Countless companies are eager to monetize AI in a huge variety of applications. This strive for a profitable use of machine learning systems is not primarily framed by value- or principle-based ethics, but obviously by an economic logic. Engineers and developers are neither systematically educated about ethical issues, nor are they empowered, for example by organizational structures, to raise ethical concerns. (108)

Note that Verbeeks begeleidingsethiek , as well as principles of Value-sensitive design and responsible innovation are methodologies for an ethics that is not opposed to technology, but inherently part of the design process.

From a business perspective, ethics is more about public relation management. In this sense, it may be relevant to critically assess to what extent AI ethics is business ethics .

This is especially problematic because "AI systems are used in areas of high societal significance such as health, police, mobility or education." (109)

Loyalty to guidelines

Actual compliance to ethical principles is poor according to Hagendorff.

He mentions progress in privacy, fairness, explainability.

Privacy:

  • Progress in privacy-friendly learning algorithms using cryptography and differential or stochastic privacy
  • But even though that's good, AI is only making progress in the first place to a large extent due to the insane amounts of available (personal) data
    • social media
    • smartphone apps
    • IoT

In the end, I would argue that the current AI boom coincides with the emergence of a post-privacy society. In many respects, however, this post-privacy society is also a black box society (Pasquale 2015), in which, despite technical and organizational efforts to improve explainability, transparency and accountability, massive zones of non-transparency remain, caused both by the sheer complexity of technological systems and by strategic organizational decisions. (110)

Page 110 provides a whole list of cases where ethical compliance is severely underachieved.

Advances in AI Ethics

Technical instructions

An obvious idea for making ethical guidelines more concrete and effective is to couple them to concrete technical implementations. But...

Ultimately, it is a major problem to deduce concrete technological implementations from the very abstract ethical values and principles. What does it mean to implement justice or transparency in AI-systems? What does a “human-centered” AI look like? (111)

Very interesting and potentially where my role as a philosopher/developer lies: AI-ethics needs to become micro-ethics:

Ethicists must partly be capable of grasping technical details with their intellectual framework. That means reflecting on the ways data are generated, recorded, curated, processed, disseminated, shared, and used (Bruin and Floridi 2017), on the ways of designing algorithms and code, respectively (Kitchin 2017; Kitchin and Dodge 2011), or on the ways training data sets are selected (Gebru et al. 2018). In order to analyze all this in sufficient depth, ethics has to partially transform to "microethics". This means that at certain points, a substantial change in the level of abstraction has to happen insofar as ethics aims to have a certain impact and influence in the technical disciplines and the practice of research and development of artificial intelligence (Morley et al. 2019). On the way from ethics to "microethics", a transformation from ethics to technology ethics, to machine ethics, to computer ethics, to information ethics, to data ethics has to take place. As long as ethicists refrain from doing so, they will remain visible in a general public, but not in professional communities. (111)

See ethics_of_technology_needs_to_become_microethics.md .

Example of initiatives:

  • Gebru et al. 2018: Standardized data sheets indicating the properties of training data sets, to check if they are suitable, with info on
    • intention of collection
    • data type
    • data collection procedure
    • pre-processing steps
    • In any case, this would increase transparency (not sure if this in itself also increases fairness, like is suggested)

Virtue ethics

A great plea to complement the deontological character of ethics guidelines or principles with a virtue ethical approach.

I argue that the prevalent approach of deontological AI ethics should be augmented with an approach oriented towards virtue ethics aiming at values and character dispositions. Ethics is then no longer understood as a deontologically inspired tick-box exercise, but as a project of advancing personalities, changing attitudes, strengthen responsibilities and gaining courage to refrain from certain actions, which are deemed unethical.

An obvious consequence is that "ethics" in this sense becomes harder to measure. This must really bum out managers with a "business" mindset :-)! I just hope they won't start doing personality assessments with questionnaires.

In a virtue ethical approach, ethics is not just that one guy standing by the sidelines pointing his fingers and shouting "Bad! Bad!". Ethics should not stifle activity but accompany it.

When following the path of virtue ethics, ethics as a scientific discipline must refrain from wanting to limit, control, or steer (Luke 1995). Very often, ethics or ethical guidelines are perceived as something whose purpose is to stop or prohibit activity, to hamper valuable research and economic endeavors (Boddington 2017, 8). I want to resign this negative notion of ethics. It should not be the objective of ethics to stifle activity, but to do the exact opposite, i.e. broadening the scope of action, uncovering blind spots, promoting autonomy and freedom, and fostering self-responsibility. (113)

This all are considerations very similar to Verbeek's guidance ethics .

Hagendorff mentions moral backpropagation here.

Institutional recommendations

  • "adoption of legal framework conditions" (113)
  • "establishment of mechanisms for an independent auditing of technologies" (113)
  • "establishment of institutions for complaints, which also compensate for harms caused by AI systems" (113)
  • "expansion of university curricula in particular through content from ethics of technology, media, and information" (113)