Malle, B. F. (2016). Integrating robot ethics and machine morality: the study and design of moral competence in robots. Ethics and Information Technology, 18(4), 243–256. http://doi.org/10.1007/s10676-015-9367-8

@robot @ethics @machine @morality @Malle

Premise:

  1. "Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots;"
  2. "machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented."
  3. These studies are rarely explicitly connected to each other

Approach:

Instead of searching for a fixed set of criteria of a robot’s moral competence I identify the multiple elements that make up human moral competence and probe the possibility of designing robots that have one or more of these human elements, which include: moral vocabulary; a system of norms; moral cognition and affect; moral decision making and action; moral communication. (243)

Takes an optimistic view:

if robotic design truly commits to building morally competent robots, then those robots could be trustworthy and productive partners, caretakers, educators, and members of the human community.

Two questions of robot ethics

  1. Robot ethics
    • E.g. ethical design (Wynsberge 2013)
    • values of implementation (Hofmann 2013)
    • Robot Rights (Petersen 2007)
  2. Machine morality
    • Moral agency (Floridi and Sander, 2004)
    • Justification for lethal military robots (Arkin 2009)
    • Mathematical proofs for moral reasoning (Bringsjord et al. 2006)

But these types of questions are interrelated:

Attempts to develop morality in machines raise many ethical questions (Wallach and Allen 2008), and ethical concerns about robot design (Scheutz and Crowell 2007) may be addressed by developing adequate moral capacities in robots. (244)

The main benefit of such an integration:

how a robot’s moral competence could help resolve some of the ethical concerns about robots in society and perhaps even guide us to new opportunities of how robots could make valuable contributions to society. (244)

From moral agency to moral competence

Normally the question of machine morality is framed as such: can robots be "moral agents"?

There is a great difficulty in reaching a definition, both on the word "moral" or the word "agent" and especially their combination.

Malle's alternative approach: define the key components of being morally capable in the case of humans. This would offer a general framework for moral competence in robots, rather than a strict definition.

Main points here (both on 245):

  1. The amount of competence we expect is dynamic and context-dependent. E.g. we do not expect full moral competence from children and this is not problematic in itself.

On this approach, research and design into moral robots is a dynamic and adjustable process (Powers 2011), guided by scientific research that has no pressures to sell products but is subject to societal ethical deliberations about design, distribution, and beneficial deployments of robots.

  1. Differing and dynamic moral competences should inform questions of robot ethics, e.g. whether it would make sense to award robots rights

Elements of moral competence

def of competence: "A competence is an aptitude, a qualification, a dispositional capacity to deal adequately with certain tasks."

Relevant aspects (245):

  1. Moral decision making and action. "Doing the right thing"
  2. Moral cognition: permissibility, wrongness, blame, the role of reasoning and emotion
  3. Moral communication: negotiating blame, justification, apology, forgiveness.

All these three points presuppose some norm system and a vocabulary to represent that system (cognition) and use it and communicate it. Adding up, we have 5 competences:

  1. A moral vocabulary
  2. A system of norms

Moral vocabulary

Some subdivisions (246). We need a vocabulary of

  1. norms and their properties
  2. norm violations
  3. response to violations

See paper for details.

Moral norms

Acquisition

Children seem to learn morality by example.

Preprogramming all relevant moral norms is a pointless task.

More promising is a mix of unsupervised and supervised learning.

Such a robot would be raised rather than simply programmed (248)

Norm representation

Are norms similar to goals? Would make implementation in robots very clear. But, human goals can transgress norms and thus:

A simple goal-based action control system would not turn humans into norm-following creatures. But could a goal-based system be sufficient for moral robots? (247)

Or are norms "embedded in the semantic-linguistic networks people build through normal language acquisition"?

Norms would be remembered and activated alongside representations of objects, actions, states.

Moral cognition and affect

Moral reasoning:

A unique feature of human blame judgments is that intentional and unintentional violations trigger distinct subsequent processing steps (Malle et al. 2014). When perceiving a violation as an intentional action, people search for the agent’s reasons for performing the action (‘‘motive’’ in legal context); when perceiving a violation as an unintentional event, people search for preventability information—whether the agent should have and could have prevented this event. All together, it is quite clear that human moral cognition is not a specialized ‘‘module’’ but builds on ordinary cognition of social events embedded in a norm system (Cushman and Young 2011; Guglielmo et al. 2009). (248)

Do robots need affect in order to be moral?

Clearly, affective phenomena can influence moral judgments, often accompany moral judgments, and probably facilitate learning and enforcing moral norms. But there is little evidence for the claim that affective phenomena are necessary or constitutive of those judgments (Avramova and Inbar 2013; Huebner et al. 2009). If affect is not necessary or constitutive of moral judgments, then even an affectless robot could be competent to make moral judgments. (248)

Moral decision making and action

  • Determinants of action
  • Free will
  • Selfishness and empathy:

Empathy, however, has at least one function in human life that cannot be so easily replaced by an unselfish goal system: to create trust in others. A person who responds empathically to the plight of others (especially the ones he may have victimized) is trusted far more than someone who coldly assesses a moral situation (even if the assessment is technically correct). A robot may be able to build such trust if it can perceive the plight and pain of others and express its perception in a sympathetic language (even if it doesn’t genuinely ‘‘feel’’ the plight and pain). These communications, however, should not be mere verbal scripts or deceptive attempts to coax the human’s trust. For advanced human–machine interactions to succeed the robot must somehow be able to demonstrate that it values things (Littman 2001; Scheutz 2012), that it cares about certain outcomes (Wynsberghe 2013). (249)

This is a great challenge.

  • Skepticism about choice capacity
    • Can robots make anything but trivial decisions?
    • Frame problem (Ford and Hayes 1991; Pylyshyn 1987)
    • Can we

Interesting question: when you program a robot to be moral (from a particular perspective and set of norms) is that robot then really moral, because the robot does not seem to have a choice?

One might object that a robot that cannot help but make moral decisions (because it is programmed to be unselfishly moral) does not have a choice and therefore is not a morally competent actor. This objection can be met in at least two ways. First, the kind of robot envisioned here could not possibly be programmed to act morally for all possible futures. It would have to be equipped with a number of guiding norms to start with but also learn many new norms, and it could therefore fail to act morally out of ignorance and, with feedback, do better next time. Second, some situations pose a decision problem in which not all relevant norms can be satisfied simultaneously. Such moral dilemmas require a genuine choice between imperfect options, and very often each of the options can be morally justified by reference to some accepted norms. So from one perspective the robot would be making the wrong moral decision, from another perspective it would be making the right moral decision. Either way, it would make a decision (Scheutz et al. 2015).

  • Decision dilemmas

Moral communication

Skipped.

Who is responsible?

Double edged blade of autonomy.

On the one hand, autonomy is desirable because it is unlikely robots can adequately act in complex social situations.

One argument for autonomy is this: Because only robots with autonomy (and its constituent abilities of choice, reasoning, learning, etc.) could possibly master the demanding task of interacting with humans, collaborating with them, taking care of them, or teaching them. We wouldn’t and shouldn’t feel comfortable deploying preprogrammed robots in any of the important domains of societal need for which we lack financial and person resources—such as education, safety, medicine, elderly and disabled care—and for which we therefore need the help of technology. Successful performance in such tasks cannot simply rely on prior programs, because human behavior is too complex and variable.

On the other hand, autonomy can introduce a responsibility gap.

Conclusion

From the perspective of integrating machine morality and robot ethics, here are a few answers to questions:

‘‘Machine morality’’ and ‘‘robot ethics’’ must be closely connected in the design and deployment of social robots. Rather than discuss in general ethical terms whether robots should become an integrated part of human society, we need to pose ethical questions relative to a set of possible moral competences a robot could have. Should robots have autonomy? Yes, if we want robots to be genuine interaction partners, but then they also must have moral competences such as an ability to follow complex norm systems and responsiveness to moral criticism. Should robots always obey human commands? Perhaps yes if their moral reasoning capacities are very limited, but no if they have the capacity to recognize human error (e.g., an immoral command) and to resolve norm conflicts. Should we allow robots to kill humans? We may have a strict value that machines must not kill. Alternatively, if part of the justification for such a strict stance is robots’ currently limited moral competence, then the situation may change when, or if, morally competent robots demonstrably uphold laws as reliably as humans—or perhaps even more reliably because they would never commit emotionally motivated atrocities. Should robots have rights and protection? As robots acquire increasing moral competence, especially moral judgment and decision making, it may well be unethical to deny them all moral standing. Their rights may be limited, however, and vary as a function of their value and specific role in society (e.g., caretaker, teacher, repair assistant).