In reply to →
“In college I took an Engineering Ethics course. It focused heavily on mistakes.”
“I wish this course had covered the idea that people might be harmed not because of an implementation mistake, but because an engineer did a good job building the wrong thing.”
I think this nice zine displays awareness of what David Koepsell dubbed the “eventual” fallacy of technological risk, which is something along the lines of: “someone will develop this anyways, so we may as well do it (us being the good guys)". From Koepsell 2010 - On Genies and Bottles: Scientists’ Moral Responsibility and Dangerous Technology R&D:
The ‘‘eventual’’ fallacy is committed by simple recourse to the truism that over an infinite time-span, all natural truths will be discovered, and all potential uses and misuses of technology will be developed, so present research on any science or application of technology is morally justified (131)
With this type of reasoning any research can be justified. The response to the “eventual” fallacy is contained in the following lines from the zine:
“But I chose to make this tool instead of a different tool. I choose to work on it now instead of doing other things.”
That is, you should ask yourself: why should we research this or produce this tool now? Are the risks of developing this tool or researching this thing outweighed by current more pressing concerns and risks?