Ethics Guidelines for Thrustworthy AI | by the Independent High-Level Expert Group on Artificial Intelligence set up by the European Commission

Source: https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai

Summary

Thrustworthy AI has three components:

1) lawful 2) ethical 3) robust (both from technical and social perspective)

The guidelines are about point 2 and 3.

The guidelines "seek to go beyond a list of ethical principles, by providing guidance on how such principles can be operationalised in socio- technical systems." (p.2)

On three layers of abstraction:

1) Underlying ethical principles and correlated values * Respect for human autonomy * Prevention of harm * Fairness * Explicability * Acknowledging tensions between these principles * Special attention to vulnerable groups of people and power asymmetries * Anticipate and mitigate potential risks of AI 2) Realization (botch technical and non-technical) of thrustworthy AI in terms of seven requirements * Human agency and oversight * Techincal robustness and safety * Privacy and data governance * Transparency * Diversity, non-discrimination and fairness * Environmental and societal well-being * Accountability 3) Assessing trustworthy AI by operationalising the key requirements of point 2 * Have an assessment list and adapt it to the use case * Stay open to identifying and implementing (new) requirements and evaluation of existing solutions, involving stakeholders etc.

Introduction

  • AI systems need to be human-centric
  • "ensure that we can trust the socio- technical environments in which they are embedded"
  • make thrustworthy AI beneficial from a competitive viewpoint
  • i.e. maximize benefits of AI systems while minimizing their risks

We therefore identify Trustworthy AI as our foundational ambition, since human beings and communities will only be able to have confidence in the technology’s development and its applications when a clear and comprehensive framework for achieving its trustworthiness is in place. (3)

  • "it is not simply components of the AI system but the system in its overall context that may or may not engender trust."
  • so it "requires a holistic and systemic approach, encompassing the trustworthiness of all actors and processes that are part of the system’s socio-technical context throughout its entire life cycle" (5)

Stakeholders:

  • "addressed to all AI stakeholders designing, developing, deploying, implementing, using or being affected by AI, including but not limited to companies, organisations, researchers, public services, government agencies, institutions, civil society organisations, individuals, workers and consumers."

Different situations, different challenges

  • The guidelines offer a horizontal foundation for trustworthy AI.
  • Different types of AI systems raise different types of issues, so adaption to particular use-cases and additional sectorial approaches should be explored

Framework

As said before, lawful AI is not the focus.

Ethical AI does not coincide with lawful AI

  • Laws often lag behind technological developments
  • May not suitably address current issues

So ensure trustworthy AI, AI also needs to be aligned with ethical norms.

Even if an AI is designed in alignment with ethical principles, it may still cause unintentional harm. There should be safeguards to prevent this and to ensure AI systems are "safe, secure and reliable".

Robustness is both technical and non-technical (i.e. it considers the context and environment of operation).

Chapter 1: Foundations of trustworthy AI

def AI ethics:

AI ethics is a sub-field of applied ethics, focusing on the ethical issues raised by the development, deployment and use of AI. Its central concern is to identify how AI can advance or raise concerns to the good life of individuals, whether in terms of quality of life, or human autonomy and freedom necessary for a democratic society (8)

Why ethical reflection on AI?

  • "it can stimulate reflection on the need to protect individuals and groups at the most basic level." (8)
    • Main focus of the guidelines
  • " it can stimulate new kinds of innovations that seek to foster ethical values" (8)
    • How exactly?
    • "Trustworthy AI can improve individual flourishing and collective wellbeing by generating prosperity, value creation and wealth maximization. It can contribute to achieving a fair society, by helping to increase citizens’ health and well-being in ways that foster equality in the distribution of economic, social and political opportunity." (8)

This effort also requires building and maintaining an ethical culture around AI because ethical codes cannot replace ethical reasoning:

A domain-specific ethics code – however consistent, developed and fine-grained future versions of it may be – can never function as a substitute for ethical reasoning itself, which must always remain sensitive to contextual details that cannot be captured in general Guidelines. Beyond developing a set of rules, ensuring Trustworthy AI requires us to build and maintain an ethical culture and mind-set through public debate, education and practical learning.

Specific meaning of fundamental rights in these guidelines

Look to fundamental rights in EU Treaties, EU Charter, international human rights law as "the most promising foundations for identifying abstract ethical principles and values, which can be operationalised in the context of AI" (9)

EU Charter: dignity, freedoms, equality, solidarity, citizen's rights, justice. Their common root is respect for human dignity. This in turn should be the basis of a "human-centric" approach to AI where "the human being enjoys a unique and inalienable moral status of primacy in the civil, political, economic and social fields." (10)

  • Lawful AI: Fundamental rights can be seen as legally enforceable rights insofar as they are legally binding in certain (EU) contexts
  • Ethical AI: insofar as fundamental rights are bestowed by virtue of being a human they underpin ethical AI, even when ethical norms are not necessarily binding

"Fundamental rights" in these guidelines are understood in the second sense.

Interesting footnote about an "unduly individualistic account of the human" (10)

It should be noted that a commitment to human-centric AI and its anchoring in fundamental rights requires collective societal and constitutional foundations in which individual freedom and respect for human dignity is both practically possible and meaningful, rather than implying an unduly individualistic account of the human. (footnote 17, p.10)

Basis for trustworthy AI

TODO: What's the difference with the NS policy, which only has 5 values?

Human dignity:

In this context, respect for human dignity entails that all people are treated with respect due to them as moral subjects, rather than merely as objects to be sifted, sorted, scored, herded, conditioned or manipulated. AI systems should hence be developed in a manner that respects, serves and protects humans’ physical and mental integrity, personal and cultural sense of identity, and satisfaction of their essential needs.

Interesting detail here is that for example the Google_AI_principles.md only mention material harm, which presumably excludes the commitment to mental integrity that this report underlines.

Freedom of the individual:

In an AI context, freedom of the individual for instance requires mitigation of (in)direct illegitimate coercion, threats to mental autonomy and mental health, unjustified surveillance, deception and unfair manipulation. In fact, freedom of the individual means a commitment to enabling individuals to wield even higher control over their lives, including (among other rights) protection of the freedom to conduct a business, the freedom of the arts and science, freedom of expression, the right to private life and privacy, and freedom of assembly and association.

I'm very concerned that AI, right now, does not score well in this respect!

Respect for democracy, justice and the rule of law:

  • "maintain and foster democratic processes"
  • "respect the plurality of values and life choices of individuals"
  • no not disrupt or "undermine democratic processes, human deliberation or democratic voting systems" (all on p. 11)
  • etc.

Some initial thoughts: algorithmic mediation of what information people are shown only threatens this perspective and can also disrupt processes of human deliberation that underlie democratic processes.

Equality, non-discrimination and solidarity:

"This goes beyond non-discrimination, which tolerates the drawing of distinctions between dissimilar situations based on objective justifications." (11)

Q: I don't fully understand why the above sketched situation is undesirable.

In an AI context, equality entails that the system’s operations cannot generate unfairly biased outputs (e.g. the data used to train AI systems should be as inclusive as possible, representing different population groups). This also requires adequate respect for potentially vulnerable persons and groups,21 such as workers, women, persons with disabilities, ethnic minorities, children, consumers or others at risk of exclusion. (11)

Interesting is also that equality is here only defined in terms of fair data. But couldn't inequality also appear in other parts of an AI pipeline?

Citizen's rights:

AI systems offer substantial potential to improve the scale and efficiency of government in the provision of public goods and services to society. At the same time, citizens’ rights could also be negatively impacted by AI systems and should be safeguarded. When the term “citizens’ rights” is used here, this is not to deny or neglect the rights of third-country nationals and irregular (or illegal) persons in the EU who also have rights under international law, and – therefore – in the area of AI systems.

Ethical principles for AI systems

These are "rooted in fundamental rights" (11) from the EU Charter and are specified as "ethical imperatives, such that AI practitioners should always strive to adhere to them." (11).

Respect for human autonomy

  • Humans "must be able to keep full and effective self-determination over themselves" (12; my emphasis)
  • "be able to partake in the democratic process" (12)
  • "AI systems should not unjustifiably subordinate, coerce, deceive, manipulate, condition or herd humans." (12)
    • Which begs the questions: which conditions are identified in which manipulation is justified? National security?
  • Human-centric design decisions that leave "meaningful opportunity" (12; my emphasis)
  • Human oversight ( see Chapter II; I'm curious how they flesh out this concept here)
  • Contribute to meaningful work and working environment

The focus on "meaningful" in these points is philosophically interesting.

Prevention of harm

  • "the protection of human dignity as well as mental and physical integrity" (12)
  • "technically robust and it should be ensured that they are not open to malicious use" (12)
  • Attention to vulnerable persons (include them in the design process) and pay attention to not causing or exacerbating power asymmetries

Fairness

There are many interpretations of fairness. But these guidelines distinguish "both a substantive and a procedural dimension" (12).

Substantive:

  • "ensuring equal and just distribution of both benefits and costs" (12)
  • "ensuring that individuals and groups are free from unfair bias, discrimination and stigmatisation" (12)
  • Do not contribute to deceiving people or them being "unjustifiably impaired in their freedom of choice." (12)
  • "AI practitioners should respect the principle of proportionality between means and ends" (12)

Interesting dynamic: "If unfair biases can be avoided, AI systems could even increase societal fairness" (12). The reasoning is that if there are tools to rid data of bias, the employment of unbiased AI applications can actually lead to fairer decisions than is the "baseline" is society right now, since the biased data was biased precisely because it was a reflection of unfair society.

Does removing unfair bias imply promoting fairness?

Procedural:

  • "ability to contest and seek effective redress against decisions made by AI systems and by the humans operating them." (13)
  • this implies that some entity can be identified for be held accountable for decision-making
    • and this entails that the "decision-making processes shouldl be explicable"

Explicability

  • "processes need to be transparent, the capabilities and purpose of AI systems openly communicated, and decisions – to the extent possible – explainable to those directly and indirectly affected. Without such information, a decision cannot be duly contested." (13)

What is an explanation in this context?

Distinction between explicable and black-box models:

An explanation as to why a model has generated a particular output or decision (and what combination of input factors contributed to that) is not always possible. These cases are referred to as ‘black box’ algorithms and require special attention. In those circumstances, other explicability measures (e.g. traceability, auditability and transparent communication on system capabilities) may be required, provided that the system as a whole respects fundamental rights. The degree to which explicability is needed is highly dependent on the context and the severity of the consequences if that output is erroneous or otherwise inaccurate. (13)

Tensions between principles

Predictive policing applications are a good example where prevention of harm and principle of human autonomy are conflicting.

There is no general solution for this so the guidelines here refer to the relevant parties to "approach ethical dilemmas and trade-offs via reasoned, evidence-based reflection rather than intuition or random discretion" (13).

In some cases there is however no trade-off to be made. For example, human dignity must be upheld in each and every case.

Chapter 2: Realising Trustworthy AI

Guidance on implementation and realisation of trustworthy AI using seven requirements that build forth on the ethical principles of chapter 1. This chapter also includes available technical and non-technical methods for that purpose.

2.1. Requirements for trustworthy AI

Stakeholders: developers, deployers, end-users, broader society.

Requirements (non-exhaustive!)

  1. Human agency and oversight
  2. Technical robustness and safety
  3. Privacy and data governance
  4. Transparency
  5. Diversity, non-discrimination and fairness
  6. Societal and environmental wellbeing
  7. Accountability

These should be continuously evaluated and considered during the life cycle of an AI system, but their implementation may be different in various contexts and depending on the concrete application.

1. Human agency and oversight

Goes back to the principle of respect of human autonomy.

Fundamental rights
  • Where risk of infringement of fundamental rights exists, do an impact assessment before deployment
  • Assessment should "include an evaluation of whether those risks can be reduced or justified as necessary in a democratic society in order to respect the rights and freedoms of others" (15))
  • Add mechanisms for receiving external feedback
Human agency
  • "Users should be able to make informed autonomous decisions regarding AI systems." (my emphasis, p. 16)
  • "given the knowledge and tools to comprehend and interact with AI systems to a satisfactory degree and, where possible, be enabled to reasonably self-assess or challenge the system" (16)
  • Goal-alignment: "AI systems should support individuals in making better, more informed choices in accordance with their goals." (16)
    • "overall principle of user autonomy must be central to the system’s functionality" (16)
    • Does that imply autonomy should be embedded into the design of the system?
  • Protection against automatic-decision making when "legal effects are produced", see Art. 22 of GDPR
Human oversight and governance mechanisms
  • Human-in-the-loop (HITL)
    • "capability of human intervention in every decision cycle of the system, which is many cases is neither possible nor desirable." (16; my emphasis)
  • Human-on-the-loop (HOTL)
    • Monitoring of operation and possibility of intervention during design cycle
  • Human-in-command (HIC)
    • "HIC refers to the capability to oversee the overall activity of the AI system (including its broader economic, societal, legal and ethical impact) and the ability to decide when and how to use the system in any particular situation." (16)
    • "This can include the decision not to use an AI system in a particular situation, to establish levels of human discretion during the use of the system, or to ensure the ability to override a decision made by a system" (16)

There is a trade-off between the possibility of exercising oversight and testing and governance: "All other things being equal, the less oversight a human can exercise over an AI system, the more extensive testing and stricter governance is required." (16)

Notice the terminology of and difference between: decision cycle, design cycle. I'm not so familiar with HIC.

2. Technical robustness and safety

Closely related to principle of prevention of harm.

Resilience to attack and security

Preventative approach that anticipates possible unintended behaviors. Protect against adversial agents and changes in operating environment.

Types of attack:

  • Data poisoning
  • Model leakage
  • Targeting infrastructure (both software and hardware)
Fallback plan and general safety

Fallback may be:

  • "switch from a statistical to rule-based procedure" (17)
  • "ask for a human operator before continuing their action" (17)

General safety: same as above; risk assessment and proactive development of safety measures, also against unintended consequences.

Accuracy
  • "explicit and well-formed development and evaluation process can support, mitigate and correct unintended risks from inaccurate predictions" (17)
  • quantification of risk! "it is important that the system can indicate how likely these errors are." (17)
  • the more risk associated with the system (e.g. when directly affecting human lives), the more important accuracy becomes

Q: explore methods where ML systems indicate the likelihood of errors; the converse formulation of this problem is to indicate e.g. for classification a degree of uncertainty.

Reliability and reproducibility
  • Reliable: work for a range of inputs and range of situations
  • Replication: replicability of AI experiments is important for accurate description and risk assessment of AI systems
    • Includes data collection, development process etc.

3. Privacy and data governance

Again relates to principle of prevention of harm.

Note: in these guidelines privacy is approaches as a fundamental right. But for example Volkman argues that privacy is in fact not a fundamental concept/right.

"Digital records of human behaviour may allow AI systems to infer not only individuals’ preferences, but also their sexual orientation, age, gender, religious or political views. " (17)

"To allow individuals to trust the data gathering process, it must be ensured that data collected about them will not be used to unlawfully or unfairly discriminate against them." (17)

Quality and integrity of data

Bias:

When data is gathered, it may contain socially constructed biases, inaccuracies, errors and mistakes. This needs to be addressed prior to training with any given data set. (17; my emphasis)

Q: This point mentions that these issues must be addressed prior to training. That seems to exclude potential ML fairness methods that address bias in the data during training. Isn't the EU policy too restrictive here?

  • Usage of data and data set must be tested and documented at each step
  • This also applies to AI-systems deployed elsewhere!
    • Isn't this tricky and perhaps restrictive? See Q below.
  • Resilience against malicious data insertions (an adversial attack)

Q: In practical terms, how feasible is it to guarantee data integrity of the whole data collection and ML training process for applications developed out-house?

Access to data

Proper protocols; only qualified and competent personnel with a justifiable need should have access.

4. Transparency

Linked to the principle of explicability. Transparency of:

  • Data
  • AI System
  • Business models
    • Interesting addition, didn't expect that!
Traceability

Traceability facilitates auditability and explainability.

Document all data and processes leading up to an AI system's decision:

  • Data gathering
  • Data labeling
  • Algorithms used
  • Decisions made (by AI system)

The goal is to allow us to trace back the decision-taking process in case AI-decisions are erroneous. This in turn helps fixing potential issues and avoiding future mistakes.

Explainability

Explainability pertains both to:

  • Technical processes leading to an AI-system's decision
  • Human decisions (e.g. decision to apply the AI system in a particular application area)

The ethical guidelines identify a trade-off between accuracy and explainability:

Moreover, trade-offs might have to be made between enhancing a system's explainability (which may reduce its accuracy) or increasing its accuracy (at the cost of explainability). Whenever an AI system has a significant impact on people’s lives, it should be possible to demand a suitable explanation of the AI system’s decision-making process. Such explanation should be timely and adapted to the expertise of the stakeholder concerned (e.g. layperson, regulator or researcher). (18)

See trade-off_explainability_and_accuracy_in_AI.md

Business model transparency about the role of AI:

In addition, explanations of the degree to which an AI system influences and shapes the organisational decision-making process, design choices of the system, and the rationale for deploying it, should be available (hence ensuring business model transparency). (18)

Communication
  • Always make sure that people know they are dealing with AI
  • Provide opt-out of this interaction to ensure compliance with fundamental rights (e.g. as identified in chapter 1)
  • Communicate the limitations of AI systems to practitioners and end-users.
    • E.g. when is the model not accurate.

5. Diversity, non-discrimination, fairness

Linked to principle of fairness.

  • "enable inclusion and diversity throughout the entire AI system's life cycle" (18)
  • "involvement of affected stakeholders" (18)
  • "equal access through inclusive design processes as well as equal treatment" (18)

Q: Equal treatment regardless? Will this be further specified? Aristotle: "equals should be treated equally and unequals unequally."

Avoidance of unfair bias

Again, notice the emphasis on unfair bias. Sources of unfair bias:

  • "inadvertent historic bias"
  • "incompleteness"
  • "bad governance models" (18)
    • What does this mean?

This can lead to:

  • "unintended (in)direct prejudice and discrimination against certain groups or people, potentially exacerbating prejudice and marginalisation" (18)

"the intentional exploitation of (consumer) biases or by engaging in unfair competition, such as the homogenisation of prices by means of collusion or a non-transparent market."

Ways of avoiding:

  • "Identifiable and discriminatory bias should be removed in the collection phase where possible." (18)
    • Note that this is more nuanced than before in the section on "quality and integrity of data", where it was said that bias etc. needs to be countered before training.
  • "putting in place oversight processes to analyse and address the system’s purpose, constraints, requirements and decisions in a clear and transparent manner." (18)
    • "The way in which AI systems are developed (e.g. algorithms’ programming) may also suffer from unfair bias." (18)
    • Indeed, so this means that some forms of bias need to be countered after the data collection phase!
    • Curious about examples

Q: Are there inconsistencies in these guidelines' position on where to tackle bias? Some section suggest it needs to be tackled before training; others also mention countermeasures in later stages of machine learning pipelines.

Accessibility and universal design
  • People should be able to use AI systems "regardless of their age, gender, abilities or characteristics" (18)
    • with "abilities" it seems to be meant that people with disabilities should have equal access
    • I'd add another interpretation, namely that often some "digital literacy" is required here as an ability, especially to preserve human agency.
  • Apply "Universal Design" principles with accessbility standards, and avoid a one-size-fits=all approach (19).
    • cf. Article 42 of the Public Procurement Directive
Stakeholder participation

Advisable to promote stakeholder participation and long-term feedback mechanisms to ensure trust.

6. Societal and environmental well-being

In line with principle of fairness and prevention of harm.

  • Other sentient beings and the environment should be considered as stakeholders in broader society.
    • Very interesting! Curious what this concretely looks like, e.g. what's the current relation between AI and animal welfare?
    • On the other hand, I'm curious to what extent the label "stakeholder" implies agency; and how meaningful it is to qualify animals as stakeholders.
  • Foster research into AI solutions for problems of global concern
  • "Ideally, AI systems should be used to benefit all human beings, including future generations." (19)
Sustainable environmentally friendly AI
  • Asses system development, deployment, use process, supply chain in terms of sustainability (e.g. resource usage and energy consumption)
  • Encourage sustainable AI
Social impact

"Ubiquitous exposure to social AI systems47 in all areas of our lives (be it in education, work, care or entertainment) may alter our conception of social agency, or impact our social relationships and attachment. While AI systems can be used to enhance social skills,48 they can equally contribute to their deterioration. This could also affect people’s physical and mental wellbeing. The effects of these systems must therefore be carefully monitored and considered." (19)

Society and Democracy
  • Consider AI's "effect on institutions, democracy and society at large. The use of AI systems should be given careful consideration particularly in situations relating to the democratic process, including not only political decision-making but also electoral contexts." (19)

7. Accountability

Linked to principle of fairness

"ensure responsibility and accountability for AI systems and their outcomes, both before and after their development, deployment and use" (19)

Auditibility
  • Auditibility: enable "assessment of algorithms, data and design processes"
    • Does not imply giving up IP or business models
    • Cf. NS policy: does not necessarily imply open source code
    • Clearly enables trust
Minimisation and reporting negative impacts
  • Protection for whistle-blowers, NGOs etc. for reporting legitimate concerns
  • Impact assessments proportional to potential risks
Deliberation about trade-offs

"This entails that relevant interests and values implicated by the AI system should be identified and that, if conflict arises, trade-offs should be explicitly acknowledged and evaluated in terms of their risk to ethical principles, including fundamental rights. In situations in which no ethically acceptable trade-offs can be identified, the development, deployment and use of the AI system should not proceed in that form" (20)

Redres

Ensure accessible mechanisms for redressing negative impacts

("Redress" -> Herstellen )

2.2.1 Technical methods for realizing Trustworthy AI

Architectures

Anchor the ethical requirements into the system's architecture:

  • white list: behavior that the system should always follow
  • black list: restrictions the system should never transgress
  • complex provable guarantees on system behavior

In dynamical systems this should be integrated into the generic "sense-plan-act" cycle.

X-by-design

  • values-by-design
    • Explicitly link the abstract ethical principles to concrete implementation decisions
  • privacy-by-design
  • security-by-design
  • fail-safe shutdown
  • resume normal operations after forced shutdown

Explanation methods

Field of XAI is dedicated to this.

Issues with neural nets, in particular:

  • "Training processes with neural nets can result in network parameters set to numerical values that are difficult to correlate with results." (21)
  • "Moreover, sometimes small changes in data values might result in dramatic changes in interpretation, leading the system to e.g. confuse a school bus with an ostrich." (21)
  • Previous point can be the target of adversial attacks

So XAI is also important for resilience and reliability!

Testing and validating

Traditional testing is not sufficient because AI behavior is non-deterministic and context-dependent.

This means that even though some implemented "concept" may be tested extensively, failure of a system may only show up when deployed in a realistic scenario.

Goal: "the underlying model must be carefully monitored during both training and deployment for its stability, robustness and operation within well- understood and predictable bounds." (24)

Q: That sounds very hard! How can we validate the underlying process that leads to decisions in neural networks? This is very much a state-of-the-art research problem I think.

Testing:

  • Include all components of the AI system
  • Ensure a diversity of perspectives are taken into account
  • Ensure different perspectives by using multiple diverse metrics
  • Adversial testing
  • Bug bounties for responsible reporting of vulnerabilities

Quality of Service indicators

Have (fine-grained) indicators that an AI system is tested in place, to promote trust and be transparent.

2.2.2. Non-technical methods for realizing Trustworthy AI

The guidelines mention with emphasis that all these should be evaluated on an ongoing basis.

Regulation

E.g.

  • Product safety legislation
  • Liability frameworks

Codes of conduct

These guidelines can be adapted by companies and stakeholders themselves to take their corporate responsibility.

  • Key Performance Indicators
  • Corporate codes of conduct
  • Internal policy documents

Standardisation

"Beyond conventional standards, co- regulatory approaches exist: accreditation systems, professional codes of ethics or standards for fundamental rights compliant design. Current examples are e.g. ISO Standards or the IEEE P7000 standards series, but in the future a possible ‘Trustworthy AI' label might be suitable, confirming by reference to specific technical standards that the system, for instance, adheres to safety, technical robustness and transparency." (22)

Certification

Certification cannot replace responsibility. But the availability of certificates is useful for indicating the trustworthiness of AI, because it is not reasonable to expect that just anyone can fully understand the workings and implications of an AI system.

Accountability via governance frameworks

E.g.

  • Appoint a person in charge of ethical issues concerning AI systems
  • Ethics panel or board (internal or external) to give oversight and advice
  • Not a replacement of legal oversight

Education to foster an ethical mind-set

  • ensure widespread knowledge of the impact of AI systems
  • stimulate participation in societal development
  • "basic AI literacy"
  • this education process requires ethicists with "proper skills and training" w.r.t. AI

Stakeholder participation and dialogue

  • Ensure benefits of AI are available to all
  • Involve all stakeholders to get widespread diverse perspectives

Diversity and inclusive design teams

"It is critical that, as AI systems perform more tasks on their own, the teams that design, develop, test and maintain, deploy and procure these systems reflect the diversity of users and of society in general" (23)

Chapter 3: Assessing Trustworthy AI

Extensive (concept) assessment list is provided. It's most effective to look at the original document here. I'll only make record some separate sentences here that caught my attention.

"It is strongly recommended to log all results both in technical terms and in management terms, ensuring that the problem solving can be understood at all levels in the governance structure" (p. 25)

On diversity, non-discrimination and fairness there are some interesting points (again review the original, these are just incomplete highlights).

  • "Did you assess whether there is any possible decision variability that can occur under the same conditions?" (30)
    • Are different decisions made under the same conditions, and does this have impact on fairness, fundamental rights etc.?
    • I didn't notice this term before, but I like it
  • "Did you ensure an adequate working definition of “fairness” that you apply in designing AI systems?" (30)
    • Did you consider other definitions?
    • "Did you ensure a quantitative analysis or metrics to measure and test the applied definition of fairness" (30)

Accountability:

"Did you establish a mechanism to identify relevant interests and values implicated by the AI system and potential trade-offs between them?" (30)

Appendix: Examples of opportunities and critical concerns

See the guidelines for examples following in these categories.

Opportunities:

  • Climate action, sustainable infrastructure
  • Health and well-being
  • Quality education, digital transformation

Critical concerns:

  • Identifying and tracking individuals with AI
    • In particular automatic identification
    • May have "unexpected impact on many psychological and sociocultural levels" (33)
    • Differentiate identification, tracing and tracking, targeted surveillance, mass surveillance
  • Covert AI systems: people should always be aware they are dealing with AI
    • Border cases, such as "AI-filtered voice spoken by a human" (34)
    • "It should be borne in mind that the confusion between humans and machines could have multiple consequences such as attachment, influence, or reduction of the value of being human." (34)
    • "The development of human-like robots74 should therefore undergo careful ethical assessment" (34)
  • AI-enabled citizen scoring
    • Protect freedom and autonomy, threat to autonomy and non-discrimination
    • "Scoring should only be used if there is a clear justification, and where measures are proportionate and fair. Normative citizen scoring (general assessment of “moral personality” or “ethical integrity”) in all aspects and on a large scale by public authorities or private actors endangers these values, especially when used not in accordance with fundamental rights, and when used disproportionately and without a delineated and communicated legitimate purpose." (34)
    • Transparency is not the solution! "Note that transparency cannot prevent non-discrimination or ensure fairness, and is not the panacea against the problem of scoring."
    • "Ideally the possibility of opting out of the scoring mechanism when possible without detriment should be provided – otherwise mechanisms for challenging and rectifying the scores must be given." (36)
  • Lethal autonomous weapon systems (pretty obvious)
  • Potential longer-term concerns

Related notes

See index_ethical_AI .

The guidelines' definition of bias .

Sources that may be interesting to read

  • L. Floridi, Soft Ethics and the Governance of the Digital, Philosophy & Technology, March 2018, Volume 31, Issue 1, pp 1–8.

  • For a definition of direct and indirect discrimination, see for instance Article 2 of Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation. See also Article 21 of the Charter of Fundamental Rights of the EU.

  • See the EU Agency for Fundamental Rights’ paper: “BigData: Discrimination in data-supported decision making”, 2018, http://fra.europa.eu/en/publication/2018/big-data-discrimination.

Other remarks, thoughts, reflections

  • These guidelines consistently remark that they are not a replacement for legal requirements. Indeed, ethics != legal. But you could argue that AI ethics is being used in a regulatory manner. If you are really really cynical, you could argue that the application of ethics in this capacity preemptively aims to reduce the need for hard regulation, while ethics itself at the end of the day does not have concrete means for compliance (because that's law!).

  • Ethical principles, norms, guidelines etc. may follow from ethical reflection, but are not ethical reflection in itself. I quite like that these guidelines sometimes remind the reader that just following the guidelines "blindly" is not a substitute for ethical reflection, which remains necessary in the face the constant stream of new developments in AI and their application in new contexts.