(Adapted with permission from An Introduction to Software Engineering Ethics by Shannon Vallor and Arvind Narayanan.)
Before starting this experience, please read What do we mean when we talk about ethics?.
We noted above that failures of critical software systems can result in catastrophic loss of life or injury to the public. If such failures result, directly or indirectly, from software engineers’ choices to ignore their professional obligations, then these harms are clearly the consequences of unethical professional behavior. Those responsible each bear the moral weight of this avoidable human suffering, whether or not this also results in legal, criminal or professional punishment.
But what other kinds of harms do software engineers have an ethical duty to consider, and to try to prevent? Consider the following scenario:
Mike is a father of 3, and in order to save for their college educations, he has been working two jobs since his kids were born. His daughter Sarah has worked as hard as she can in high school to get high grades and SAT scores; as a result of her hard work she has been accepted to a prestigious IvyLeague college, and the deposit for her first year is due today. If the deposit goes unpaid, Sarah loses her spot in the freshman class. Mike paid the bill last week, but today he gets an email from the college admissions office saying that his payment was rejected for insufficient funds by his bank, and if he does not make the payment by the end of the day, Sarah will lose her place and be unable to attend in the Fall.
Panicked, Mike calls the bank – he had more than enough money in his savings to cover the bill, so he cannot understand what has happened. The bank confirms that his account had plenty of funds the day before, but cannot tell him why the funds are gone now or why the payment was rejected. They tell him there must be some ‘software glitch’ involved and that they will open an investigation, but that it will take weeks to resolve. They will only restore the funds in his account once the investigation is completed and the cause found. Mike has no other way to get the money for the deposit on such short notice, and has to tell Sarah that he couldn’t cover the bill despite his earlier promise, and that she won’t be attending college in the Fall.
Karen is a young lawyer at a prestigious firm with an incredibly hectic and stressful schedule, who needs to organize what little free time she has more efficiently. She has just downloaded a new app called Errand Whiz onto her iPhone; this app merges information from Karen’s to-do list, information on her purchasing habits from retail stores she shops at, and GPS software to produce the most efficient map and directions for running errands on her days off. Based on what it knows about what she needs to purchase and her general shopping habits, it tells Karen what locations of her favorite stores to visit on a given day, in what order and by what routes – this way she can get her errands done in the least amount of time, traveling the least number of miles.
To accomplish this, the app aggregates information not only about where she lives and shops, but also tracks what she typically buys in each store, how much she buys, what she typically pays for each item. This collected data is not stored on Karen’s phone, but on a separate server that the app links to when it needs to create a shopping map. The app encourages users to log in via Facebook, as the developers have made a deal with Facebook to sell this data to third-party advertisers, for the purpose of targeting Facebook ads to Karen and her friends.
In what ways could Karen potentially be harmed by this app, depending on how it is designed and how her shopping data is handled and used? Identify a few harmful scenarios you can think of, and the types of harm she could suffer in each.
Which if any of these harms could result from ethical failings on the part of the people who developed Errand Whiz? How, specifically?
What actions could the people behind Errand Whiz take to prevent these harms? Are they ethically obligated to prevent them? Why or why not? Justify by reference to either the ACM Code of Ethics or the Software Engineering Code of Ethics and Professional Practice.
Ideally, these scenarios have helped to broaden your understanding of the ethical scope of software engineering. In considering and protecting the ‘health, safety and welfare’ of the public, we must not limit our thinking to those contexts in which our design choices or coding practices have the potential to cause someone’s death, or cause them direct physical injury. The harms that people can suffer as a result of failures by software engineers to consider their ethical obligation to the public are far more numerous and more complex than we might think.