Why Ethics Matters for Autonomous Cars

Terminology:

These refer primarily to future vehicles that may have the ability to operate without human intervention for extended periods of time and to perform a broad range of actions. I will also use “cars” to refer loosely to all motor vehicles, from a motorcycle to a freight truck;

This paper highlights the dangers of morality by math

Why ethics matters

Discrimination issue with utilitarian thinking:

For AVs people often sketch ethical dilemmas in which 1) a crash is unavoidable 2) not choosing results in a catastrophe in any case, and 3) one must choose the lesser of two evils.

Imagine choosing to hit a young girl and an old grandma. Even though people generally might choose the little girl, due to various reasons, choosing the grandma implies making a discrimination on age.

However, the ethical code of the industry

"commits itself and its 430,000+ members “to treat fairly all persons and to not engage in acts of discrimination based on race, religion, gender, disability, age, national origin, sexual orientation, gender identity, or gender expression” [23]. Therefore, to treat individuals differently on the basis of their age, when age is not a relevant factor, seems to be exactly the kind of discrimination the IEEE prohibits [18, 33]. Age does not appear to be a relevant factor in our scenario as it might be in, say,

Such a discrimination threatens the right to life and human dignity.

Simply deciding this dilemma by mere chance is also not alright, because there might be good reasons to prefer one option over the other; and such an indifference also does not seem very respectful towards one's right to life.

Beyond crash avoidance

Neither is braking a straightforward option. Not only is it often not physically possible, it is also not always the right response in case of danger to break hard, e.g. in slippery conditions.

Giving back control to the user is too slow, as users are estimated to need up to 40s to regain full awareness. (Cf. 1-2 sec reaction time needed).

In other words, the problem moves from crash avoidance to crash-optimization.

Crash optimization Means Targeting

When we start doing "morality by math" we have to make particular preferences explicit.

In the young girl vs grandma dilemma, one could choose the young girl based on age, but another reason could be the preference for reducing damage to the car itself and its passengers. If that is a priority, it makes sense to hit the girl instead since she is smaller. Similarly, hitting a smaller car would then be preferable to hitting a big SUV.

If instead the priority is to protect other drivers and pedestrians, we should instead hit the heaviest object we can find, e.g. a truck. This will likely kill us, but not the driver of the truck.

Crash optimization resembles (military) targeting algorithms and may involve deliberate and systematic discrimination.

Beyond harm (Issues with crash optimization)

Imagine having to decide between hitting a motorcyclist with and one without helmet.

Hitting the one with the helmet might be preferred as that one has higher chances of surviving. However, this implies that wearing a helmet, which is a responsible and desirable choice, is penalized because you are more likely to be targeted by an autonomous vehicle. Similarly, a safer car such as Volvo, would then be an indesirable choice. Paradoxically, it is more safe w.r.t. AVs to not drive safely yourself.

Conversely, one could reverse the reasoning and argue that one should hit the motorcyclist without helmet, because one should were a helmet; despite more likely killing the target in that scenario.

Scenarios that implicate ethics

Deer on the road

Many factors to take into account:

  • What type of animal
    • How heavy?
    • Is it a pet?
  • Left or right driving country?
  • Traffic incoming in the other direction?

Other factors relevant to the decision-points above include: the road-shoulder type (paved, gravel, none, etc.), the condition of the car’s tires and brakes, whether the car’s occupants are seat-belted, whether the car is transporting dangerous cargo that could spill or explode, proximity to hospital or emergency rescue, damage to property such as houses and buildings, and more. These variables influence the probability of an accident as well as expected harm, both of which are needed in selecting the best course of action.

Whether or not sensing technologies will improve enough to deliver answers to our questions above, a programmer or OEM would still need to assign costs or weights to various actions and objects as best as they can. Yet these values are not intrinsic to or discoverable by science or engineering. Values are something that we humans must stipulate and ideally agree upon. In constructing algorithms to control an autonomous car, ethics is already implied in the design process.

Major difference between people and AVs making such decisions: whereas humans might make a very bad decision instinctual under heavy time pressure and emotional distress, this is not likely to lead to a criminal persecution due to the absence of bad intent and these circumstances. In the case of AVs though, such ethical decisions have to be premeditated and this changes a lot.

Self-sacrifice

The point of thought-experiments is to draw out ethical intuitions and to test principles. E.g. show that reasonable principles can lead to controversial results.

E.g. consequentialism: minimize harm and maximize whatever it is that matters, e.g. the number of happy lives.

In that case, self-sacrifice by your AV is very reasonable if that avoids hitting a bus full of school kids.

A new point here is that this self-sacrifice is perhaps noble if voluntarily chosen, but not so much when your AV decides to sacrifice you.

Transparency and consent are clearly needed here.

Ducking Harm

Is avoiding a crash always the best option? Intuitively yes, but even then there are scenarios where this is controversial.

Imagine a truck behind you not breaking on time. The AV could dash out of the way, but this would clear a path towards a school class of children passing a crossing. Is the programmer then responsible for these deaths? Either negligence or premeditated homicide?

Trolley problems

Consequentialist: switch track to kill less people.

Non-consequentialist: switching constitutes a decision to kill one person vs. simply letting five people die. The assumption being that the latter option implies less responsibility for the deaths.

This distinction matters for liability reasons as well, from the perspective of the manufacturer of the car. I.e. taking over control by the AV makes the producer of the AV responsible, whereas not taking over control from the human simple lets the human kill five people, avoiding responsibility.

Other issues

  • Does ownership of the AV matter? (E.g. a public bus?)
  • Are AVs a threat to the insurance industry?
  • Are AVs susceptible to hacking?
  • Can we safeguard privacy in AVs?
  • Will new forms of abuse be introduced? (E.g. "gaming" AVs to trick them into particular behavior)
  • How do AVs communicate with other AVs and human drivers?

Conclusions

Peter Singer: "Yes, Moore's Law is operative, but so is Murphy's Law"

Source: Singer, P.W.: The robotic revolution. Brookings Institution. http://www.brookings.edu/research/ opinions/2012/12/11-robotics-military-singer (2012). Accessed 8 July 2014 44.

"Without looking at ethics, we are driving with one eye closed." (82)