Psychological roadblocks to the adoption of self-driving vehicles Self-driving | Azim Shariff, Jean-François Bonnefon and Iyad Rahwan.
This article reads as a guide into tricking people to trust automation.
Ethical dilemmas
They discuss ethical dilemmas, but do not offer a philosophical account. They instead adapt a market perspective: companies that state they design cars to always hurt others rather than the passenger can count on public outrage, whereas the ethically more sensible utilitarian approach is likely to scare away consumers. Basically: who buys a car when you have the fear that it might decide to harm you at one point?
To resolve those issues from a market perspective, they somewhat cynically propose to leverage the "altruistic" appeal of ethical cars.
Allowing the altruistic benefits of autonomous vehicles to reflect on the consumer can change the conversation about autonomous vehicle ethics and prove itself to be a marketing asset. (page 2)
N.B. the article/comment has no page numbers, so I guess then also do not use them in referencing.
Algorithm aversion
Because ADS accidents will receive extreme media coverage, the risks and dangers of ADS might be over-exaggerated. This is detrimental to adoption of the technology. On top of that, there likely is the compound effect algorithm aversion:
Worse, for autonomous vehicles, these reactions may be compounded by algorithm aversion, the tendency for people to more rapidly lose faith in an erring decision-making algorithm than in humans making comparable errors. (3)
Though human themselves, and ultimately answerable to the public, legislators should resist capitulating to the public’s fears of low-probability risks. They should instead educate the public about the actual risks and, if moved to act, do so in a calculated way, perhaps by offering the public ‘fear placebos' — high-visibility, low-cost gestures that do the most to assuage the public’s fears without undermining the real benefits that autonomous vehicles might bring.
Asymmetric info and theory of the machine mind
Need to make sure not only failures but also successes are covered in the media:
e. Passengers will be acutely aware of the cars’ rare failures — leading to the issues described above — but may be blissfully unaware of all the small successes and optimizations.
This is the more interesting point, from a philosophical perspective:
This asymmetry of information is part of a larger psychological barrier to the trust in autonomous vehicles: the opacity to the decision-making occurring under the hood. If trust is characterized by the willingness to yield vulnerability to another entity, it is critical that people can comfortably predict and understand the behaviour of the other entity.
There is thus a push for transparency in the decision making process. There is the fundamental issue with the explainability of machine learning parameters, which they point out. But the point that is most interesting to me, is that the authors emphasize that more transparency is not always better from the point of view of the end user.
This leaves underlying decision-making processes opaque even to the programmer (let alone the passenger). But even if a detailed account of the computer’s decisions were available, it would only offer the end-user an incomprehensible deluge of information.
In fact, usually design tries to hide as much complexity as possible in favor of a better user experience:
The trend in many ‘lower stakes’ computer interfaces (for example, web browsers) has thus been in the opposite direction — hiding the complex decision-making of the machine to present a simple, minimalistic user experience. For autonomous vehicles, although some transparency can improve trust, too much transparency into the explanations for the car’s actions can overwhelm the passenger, thereby increasing anxiety.
Given that an understanding in the sense of having a complete informational/technical picture of the decision process is infeasible for modern learning automata, we should take another approach:
Thus, what is most important for generating trust and comfort is not full transparency but communication of the correct amount and type of information to allow people to develop mental models (an abstract representation of the entity’s perceptions and decision rules) of the cars — a sort of theory of the machine mind.
See: Gromet, D. M., Kunreuther, H. & Larrick, R. P. Proc. Natl Acad. Sci. USA 110, 9314–9319 (2013).
One question is thus how autonomous vehicles should communicate with various traffic participants:
Moreover, autonomous vehicles will need to communicate not just with their passengers, but with pedestrians, fellow drivers and the other stakeholders on the road.
Humans use various signals, indicators, horns, gestures etc. to communicate intentions; most importantly "through assumptions based on the mental models formed of drivers". In other words, there's a lot of folk psychology going on here, folks.
Q: Is that a question of the tracking or tracing requirement of MHC? Or a different issue altogether?
The authors suggest:
Everyone on the road will need to adjust their human models to those of autonomous vehicles, and the more research delineating what information people find crucial and comforting, the more seamless and less panicky this transition will be.
There's seems to be a research question lurking beneath that remark: Should we adapt our mental models to autonomous vehicles, or should we design our autonomous vehicles such that the do not mismatch human mental models?
I agree with the conclusion that there is a moral imperative to think about the societal impact of autonomous vehicles and the corresponding change in our social contract. But I do not agree that much with the reason of the authors. We are not responsible for victims caused by human drivers by not doing everything to speed up and smoothen the transition to autonomous vehicles. Instead we are responsible for making sure this new technology does not fundamentally disrupt that social contract in a way that is harmful for society and introduces new, more fundamental ethical issues. We have a responsibility to at least, given current epistemic uncertainty, foresee these ethical issues so that we may deal with them before it is too late. (Cf. responsible innovation).
We believe it is morally imperative for behavioural scientists of all disciplines to weigh in on this contract. Every day the adoption of autonomous cars is delayed is another day that people will continue to lose their lives to the non-autonomous human drivers of yesterday.
Some thoughts after reading
Idea for research: the importance of folk psychology for the tracing condition?
In order to recognize a machine's decision as one's own, it a requirement is that someone fundamentally appreciates the capabilities of the system etc. Machine learning: limited explainability and transparency. "Black box". Another way of saying "knowing the capabilities, etc" is: forming a correct or appropriate mental model. 1) Especially when we consider that humans tend to explain machines using human mental models anyways, it's maybe best to use this given to our benefit. Research into this. 2) Traceability requires a moral appreciation and valuation of the system. There are links between folk psychology and our moral understanding of behavior.