Facebook sparked outrage this summer when it published results of a study conducted on unwitting users. The study looked at whether people who were shown more positive or negative words in friends’ posts would write more positive or negative words themselves, apparently without considering the ethics of manipulating users’ emotions. (Indeed, users shown more negative words were more negative in their posts and vice versa.) “Was this designed to create maximum benefits for the end user? I can’t really see that anywhere in that research,” says Marc Smith, a sociologist who spent 10 years as a researcher at Microsoft. “Where is it saying, ‘This is how we will now deal with people who are borderline depressed, we’re going to start steering them toward happier stuff?'”
The study highlighted one of the tech industry’s uglier secrets: Companies like Netflix, Facebook, and Google have tremendous power to influence users’ behavior, but little incentive to wield that power ethically. These companies can encourage seemingly positive behavior, like voting, getting to know prospective partners, and reporting bullies. But what happens when users’ welfare conflicts with that of a company? There is no system in place to protect the user. A handful of classes at universities are attempting to prepare tomorrow’s designers to make decisions that consider user well-being first.
MIT and RISD: Science Fiction To Science Fabrication
Sophia Brueckner, a research assistant at MIT Media Lab, developed a new type of ethics class for technology inventors after working at Google, where even the smallest change to a product could impact tens of millions of people instantly. “I realized I could make people really happy or sad or, if you think about neuroplasticity, it’s actually changing the way the neurons in our brains work,” she says. “It made me feel very humble.”
The class, which she taught at the Rhode Island School of Design and the MIT Media Lab, attempted to teach a sense of responsibility to technology inventors through science fiction, a genre in which writers have been thinking deeply about the impact of today’s technologies for decades. “It encourages people to have that long-term version that I think is missing in the world of innovation right now,” she says, “What happens when your idea scales to millions of people? What happens when people are using your product hundreds of times a day? I think the people who are developing new technologies need to be thinking about that.”
Students in Brueckner’s class built functional prototypes of technologies depicted by science fiction texts. One group created a “sensory fiction” book and wearable gadget that, in addition to adding lights and sounds to a story, constricts the body through air pressure bags, changing temperature and vibrating “to influence the heart” depending on how the narrative’s protagonist feels. Another group was inspired by a dating technology in Dave Eggers’s The Circle that uses information scraped from the Internet about a date to give suggestions about how to impress him or her. They created an interactive website about a friend using his public information to see how he would react to the idea. A third group imagined how a material that could transition from liquid to solid on command like the killing material “ice-nine” from Kurt Vonnegut’s Cat’s Cradle could be used as a prototyping tool.
It’s impossible to predict exactly what impact a technology will have on individuals or on society while you’re designing it. The invention of the car resulted in an increase in teenage pregnancy. “I’m sure Mr. Ford did not think, ‘I know, this is a technology for making more Americans,'” Smith, the sociologist, says. Still, Brueckner argues, studying science fiction can help technology creators understand that their decisions will have long-term impacts, even if they might not be able to predict exactly what they are. “You realize that small decisions can have a vast impact on a lot of people in the future, and that it’s important to be thoughtful about what you’re creating.”
Stanford’s Calming Technology Lab
Neema Moraveji, the founding director of Stanford’s Calming Technology Lab and a cofounder of breath-tracking company Spire, has a different approach for teaching students to consider the human impact of what they are designing. His classes teach students to create technology that actively promotes a calm or focused state of mind, and he co-authored a paper that laid out several suggestions for technology designers, including:
- Letting users control or temporarily disable interruptions, the way that TweetDeck allows users to control from whom to receive notifications on Twitter.
- Avoiding overload through the number of features available and the way information is presented. For instance, a Twitter app that opens to the least-recent tweet, “gives users the sense that they must read through all the tweets before they are done.”
- Using a human tone or humor
- Providing positive feedback such as “Thanks for filling out the form” and “You successfully updated the application” in addition to error alerts
- Including easy ways to interact socially, such as Likes and Retweets, which allow people to interact without worrying about how they appear to others.
- Avoiding time pressure when not necessary.
- Incorporating natural elements like “soothing error tones, naturalistic animations, and desktop wallpapers taken from the natural world.”
The lab has worked on projects like a “calming email client” that rewards you for good behavior, a “personal peace firewall,” and a text message-based system for encouraging people to “smile at each other through their cellphones.” Students have proposed apps for tracking sleep, teaching asthematics breathing techniques dealing with an attack, eating mindfully, and keeping in touch with loved ones.
Classes Further Afield
Though Stanford has the only calming technology lab I could find, many other schools have found ways to step outside traditional ethics classes for design and engineering students. The University of California, Berkeley’s chemical engineering department also offers a science fiction course. MIT teaches a memoir and technology course. Both of those schools, as well as Harvard University, Cornell University, the University of Michigan, and the University of Wisconsin-Madison have programs in the quickly growing field of Science and Technology Studies (STS), which looks at science and technology in social, political, and cultural contexts and often include courses focused on ethical design.
What About Today’s Tech Inventors?
It’s great, you might say, that university students are considering the long-term impact of new technology through lenses like science fiction, but it still leaves the problem of convincing technology companies that make the products we use to move user well-being up the priority list.
Throughout history, the invention of dangerous technologies has been followed by mandated mitigating technologies that make them less dangerous. The death machine that is the automobile, for instance, eventually got seatbelts and airbags, and office buildings that are difficult to evacuate during a fire got sprinkler systems. First insurance companies required them, then the law did. Smith imagines today’s technology following a similar trajectory. But he doesn’t imagine this happening any time soon. “It took decades to go from automobile to seatbelt,” he says. “It took 150 years to go from steam engine to ‘not death machine.’ Technologies emerge, and over time, they are domesticated.”
Moraveji argues that it’s actually in companies’ best interests to consider the well-being of users first. “Attention is a fragile domain and people can get very annoyed very quickly unless the value generated by the product is high enough to withstand the annoyance factor,” he explained in a Fast Company Live Chat. “From a user’s perspective, in a world where everybody is trying to get our attention, there is an opportunity for a new positive association created with a product that not only creates value but cares about my state of mind and makes me feel taken care of (instead of only making me feel worried, anxious, overwhelmed, and generally frazzled).”
There are plenty of examples where company interests and user interests align. Another Facebook study, the New York Times reported earlier this month, aims to make its users nicer by, for instance, tweaking tools that allow a teenager who wants a post removed to communicate his or her feelings to its author. Creating a place where teenagers feel more empathetic is better for both Facebook, which wants them to stick around, and for teenagers, who don’t want to be bullied.
But there are also plenty of situations in which what is good for a company can be destructive for a user. Staying on Facebook for a longer time is good for Facebook, but is it good for users? Infinite scroll keeps you on a page longer, but it can also feel overwhelming. Facebook can increase voter turnout with a few tweaks to its algorithm, but is it pushing users toward a certain political party in the process? If it does, is that okay? These are the sorts of dilemmas that some universities are hoping to prepare their students to address.
For now, there are products like Spire, for which Moraveji recently took a leave of absence from Stanford to co-create. The gadget, a breath monitor that clips to a pants waistband or bra, alerts uses when they are getting tense and suggests quick activities like breath exercises they can use to regain focus or calm. It’s like a seatbelt for the whole Internet–no participation from companies like Facebook or Google required.