Fast company logo
|
advertisement

CO.DESIGN

The hardest job in Silicon Valley is a living nightmare

In the documentary ‘The Cleaners,’ we meet the people who do the exhausting, ambiguous, and traumatizing work of content moderation.

The hardest job in Silicon Valley is a living nightmare

A content moderator in Manila, one of the many hired by Silicon Valley to monitor online content. [Image: Gebrueder Beetz Filmproduktion]

BY Mark Wilson6 minute read

“My mom always told me, if I don’t study well, I’ll end up as a scavenger. All they do is pick up garbage. They rely on garbage. It’s the only livelihood they know.”

It’s an insight that a young Filipino woman makes when realizing that her own job in technology is very much the same thing. Her position: Content moderator, one of perhaps thousands of human beings who scan, assess, and delete media of everything from beheadings to child pornography uploaded to services like Facebook, YouTube, and Twitter each day. Her quota? 25,000 photos a day. Her rules? Dictated by corporations–but often vague at best.

This scene comes from The Cleaners, a documentary by German filmmakers Moritz Riesewieck and Hans Block, who spent the last several years gaining access to Manila’s clandestine world of content moderation. The city has become Silicon Valley’s cleanup capital. As the Facebooks and Googles of the world outsource much of their content moderation to third-party companies, those companies hire these contractors abroad to actually do the work.

“It’s a really secretive industry,” says Block. “When someone is working for Facebook, they have to say they work for ‘The Honey Badger Project.’ They use codewords so no one knows they work for the clients.”

When the filmmakers began shooting the documentary in 2015, they spent the first eight months traveling back and forth between Berlin and Manila, just trying to track down the companies performing this labor, and then forge relationships with their employees to go on camera and talk about the work.

“They have private security firms to make sure no one talks to journalists. They took photos of our team with a warning and said, ‘If you talk to those guys you will be immediately fired,” Block recounts.

“We had to decide [how to film.] They have airport scanners in their offices, so it wasn’t at all possible to bring a hidden camera in. Even if we could, we wouldn’t have liked to do that,” says Riesewieck. “It wasn’t worth harming the workers, and it doesn’t make sense visually.” At best, The Cleaners would have been a GoPro documentary filmed in a backpack.

[Image: Gebrueder Beetz Filmproduktion]

Instead, The Cleaners takes some creative liberties to tell the story, which turns Manila into an important character in the process. I’d estimate 90% of the film and its interviews are shot at night, with the shadowy city as its backdrop. This gives Manila a dystopian Gotham or Blade Runner look, the directors tell me, but it also highlights an important point about the job: When the sun is up in the U.S., that’s when Americans are posting the most on social media. So as a result, content moderators work the night shift.

The moderators themselves–all of whom are unnamed, some of whom are shot to carefully to mask their identity–do appear to be working in the film, inside a dark, empty office building. The film never reveals that this is not their real place of employment, but it is a suitable set piece all the same.

“We coincidently found a hidden, abandoned office where this work had been done, with all the workstations still set up. It was just left behind because the company changed the site,” says Riesewieck. “We rented it, and we could invite employees into this office without risking too much harm.”

Here, we see the workers make quick decisions over heinous images, filtering their own rationale through regulations that have been handed down to them. Hitting a green or red button determines whether an image sticks around or is deleted. Even this content moderator interface–while technically a re-creation for the film–is faithful to its source material.

“We precisely rebuilt the original moderation software according to data we got from several different insiders,” the directors explain. “So the content moderators in our film actually moderate in exactly the same way they do it during their shifts.”

advertisement
Illma Gore, a Los Angeles-based artist whose work was banned from Facebook in 2016 after she posted a painting depicting a nude Donald Trump [Image: Gebrueder Beetz Filmproduktion]

The only difference is that the moderators speak aloud for our benefit, giving us a stream of consciousness that reveals the decision-making process. “This is Donald Trump in nude,” one worker narrates, while accessing the now famous painting by Illma Gore imagining his micropenis. “He’s not a strong enough leader to handle [sic]. That’s why his penis is somehow small. He’s not that manly enough to handle the huge task as a President of America. It’s delete. Why? It degrades Donald Trump’s personality so it must be deleted.”

In the view of this moderator, and perhaps the platform’s own protocols (neither the specific company nor its rules are revealed), Trump was being cyberbullied.

Through the course of the film, Riesewieck and Block don’t try to answer any big questions or offer a satisfying conclusion to their viewers, but they do very convincingly demonstrate how deep the problems of content moderation go. There is the immense personal toll and psychological trauma these workers face–each person has a story to tell about that one image they just can’t scrub from their mind.

“It seemed like it was a joke. We didn’t know if it was real. He was attempting to kill himself using a rope. It was live, not uploaded,” one moderator recounts. “Maybe thousands of viewers. There were comments like: ‘Don’t do it, don’t commit suicide!’ Others were joking: ‘Do it! Do it!’ As long as he hasn’t actually committed suicide, we’re not allowed to stop his stream. Because if we stop it, we’re the ones who get in trouble….When he started to get up on the chair, and he put the rope around his neck, that’s when we got nervous that he was going to go ahead with it. Then he killed himself.”

Even though the live stream is never shown, the film forces you to relive these memories right alongside the moderators in what turns out to be an exhausting hour and a half. The Cleaners also highlights the greater societal toll. Silicon Vally has handed powerful tools of censorship to a relative few people. Their guidelines are provided by profit-driven social media companies. And ultimately, no single person can ever have the immense, globe-spanning social and political context necessary to determine, say, whether that image is war journalism or ISIS propaganda. Is that video too terrible to see? Or should we all have to see some of this bad stuff, lest we allow Silicon Valley’s veneer of optimism to taint our terrible reality?

[Image: Gebrueder Beetz Filmproduktion]

Riesewieck and Block assure me that they didn’t want this film to be one-sided–and indeed, some of the best perspective of the film is offered in an interview with Nicole Wong, a former policy maker Google and Twitter, who walks through complicated decisions like leaving the hanging of Saddam Hussein on YouTube.

But Facebook, Google, and Twitter are ultimately protected from face-to-face scrutiny throughout the film, because they all refused to take part in it. The closest we ever get to knowing what the Valley really thinks is through the mouthpieces of its lawyers, when summoned to a congressional hearing on content moderation. As you might expect, those lawyers don’t say much.

“The main approach of our film from the beginning, was to include [these companies], to have an open dialog on the topic. We didn’t want to bash companies. We wanted to think openly, what’s the problem of an open network?” says Block. “No one responded to our interview requests. That was very frustrating to us. Whenever we hear about Facebook, they present themselves very openly, to give everyone a voice. We experienced the opposite. . . . There’s something like a cone of silence in Silicon Valley.”

Recognize your company's culture of innovation by applying to this year's Best Workplaces for Innovators Awards before the final deadline, April 5.

PluggedIn Newsletter logo
Sign up for our weekly tech digest.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Privacy Policy

ABOUT THE AUTHOR

Mark Wilson is the Global Design Editor at Fast Company. He has written about design, technology, and culture for almost 15 years More


Explore Topics