Fast company logo
|
advertisement

STRONG FEMALE LEAD

This is the impact of Instagram’s accidental fat-phobic algorithm

The Visible Collective founder Jessica Richman explores the impact of Instagram’s unintentional algorithm that discriminates against fat bodies.

This is the impact of Instagram’s accidental fat-phobic algorithm

[Photo: AllGo – An App For Plus Size People/Unsplash]

BY Lydia Dishman6 minute read

I have heard for months that Instagram had an algorithm that disproportionately flagged images of fatter bodies. This September, my fears were confirmed.

According to Jennifer Allbaugh (performer name Ruby), vice president of the Adult Performers Actors Guild (APAG), a member of the Instagram team did, in fact, confirm that Instagram has an algorithm that detects and flags photos featuring over “60% skin.” [Editor’s note: a Facebook Company spokesperson said in a statement: “This is not true. We have proactive AI technology to find content that is likely violating our policies which don’t allow for nudity, however there are many signals that are taken into effect. We do not use an algorithm identifying a percentage of skin.” Facebook, which owns Instagram, has this policy regarding adult content.]

Although the original intention of the algorithm was to censor images that could possibly be seen as inappropriate, especially to younger audiences, the algorithm has disproportionately and adversely affected larger-bodied Instagram users and influencers.

Let’s say a smaller-bodied woman decides to wear a bathing suit that covers up 40% of her skin. Now let’s imagine a fat woman decides to wear the same bathing suit. That bathing suit may have slightly more fabric due to the larger size, but that individual’s body could have significantly more skin, causing Instagram’s algorithm to flag the image even though there is nothing inappropriate about it. Although Instagram likely did not set out to do this, they ended up creating an algorithm that discriminates against fat bodies.

As many major companies are addressing the fact that algorithms have adverse effects with regards to race and gender, I believe it’s important to point out that people of size are also affected right now, multiples times a day, on Instagram, by an inherently discriminatory algorithm.

I learned more about Instagram’s algorithm at CurvyCon, a “three-day event that brings plus size Brands, Fashionistas, Shopaholics, Bloggers, and YouTubers into one space, to chat curvy, shop curvy and embrace curvy.” That’s where Carina Shero, a plus-size influencer and lingerie model, asked actress and activist Jameela Jamil how she can stop Instagram from unfairly flagging her images and those of other plus-size influencers.

I followed up with Carina afterward to dive deeper into some of her concerns and learned additional details on how Instagram’s policies affected her. Carina said she has been shadowbanned for 2.5 years and is now on her seventh account (but still has 533K followers). According to Wikipedia, shadow banning is “the act of blocking or partially blocking a user or their content from an online community such that it will not be readily apparent to the user that they have been banned.”

Other larger-bodied influencers she knows had accounts that were disproportionately censored. That included having accounts deleted, posts deleted, photos removed, images not showing up in hashtags, and difficulty getting verified despite meeting the requirements for receiving a verified badge. Carina said, “The same content can be seen on a fat person and a skinny person, but on a fat person it is deemed pornographic, and on a skinny person it is allowed.”

Carina told me that the Adult Performers Actors Guild had recently protested against policies that were continually banning adult performers at Instagram. During the course of that protest, Carina said that members of the union had the opportunity to engage with representatives of Instagram. Through my research, I later discovered this meeting included Instagram’s head of public policy Karina Newton, public policy assistant manager Aparajitha Vadlamannati, communications officer Stephanie Otway, and Facebook associate manager of product policy Kim Malfacini.

When, according to Jennifer, “you can’t do business without being on Facebook or Instagram,” getting kicked off the site makes someone not only feel ostracized but can actually affect their income. Another plus-size influencer I spoke with, Lilith Fury, had many of her images flagged, including some that were sponsored posts. She is no longer able to use the branded content feature on Instagram, which removes her legitimacy as a possible partner for brands and thus decreases the amount of money she can make as an influencer. If I were to share every story about fat people (especially women) whose accounts have been affected, this article would never end.

This algorithm not only affects the emotional well-being and livelihood of the influencers but has large downstream effects.

According to Carina, “as a plus-size person, the only media we have is what we create for ourselves.” She’s right. When Refinery29 and Getty Images launched their 67% Project in 2017, they acknowledged that although 67% of women are plus-size, they make up “less than 2% of the images we see.” Carina described how she received messages from female fans who were on the verge of suicide because they never thought they could see themselves as beautiful until they saw images of her. She also received multiple emails from people’s partners saying that Carina’s images transformed their partners’ self-image, which helped positively change their relationships.

Carina’s fear of Instagram’s algorithm is that “if people can’t show themselves as they really are, and can’t live their own self-expressed lives, others will suffer as a result. Having people not being able to be fully expressed is unacceptable. We have the power to shift the way society perceives fat people.”

advertisement

As Jennifer (Ruby) from APAG reflected on her time with Instagram, she mentioned that they have an impossible job, and she feels for them. She detailed how, once the algorithm flags an image, users can appeal so these images get seen by a person. Unfortunately, she said the people who then review those images are not very diverse, which means that there may only be one point of view as to what is acceptable.

She believes that Instagram can increase fairness in their moderation by employing a varied and diverse group of people. She even suggested employing retired adult performers who may have more of a grasp for understanding what should be seen as appropriate and what shouldn’t. I suggested that hiring current plus-size influencers as moderators could also be beneficial to the moderation process. The more diverse backgrounds people have who are monitoring accounts, the better. We all have biases, but diversity in moderators allows for those biases to hopefully cancel each other out.

In the meantime, Jennifer suggested that plus-size people who feel they are continually facing discrimination on Instagram should continue to appeal. If they find that Instagram is not taking action, they should submit their information to APAG’s Instagram discrimination website. APAG is capturing this information and sending a list of flagged accounts to Instagram so they are aware of these accounts and can possibly assist.

APAG is especially keen and well-positioned to work with Instagram on this because the organization knows this is what the adult industry went through 40 years ago. If Instagram wants to avoid significantly increased government regulation, they need to figure out how to police themselves and protect everyone’s free speech at the same time.

Jennifer sounded positive and cooperative when speaking about Instagram. She even mentioned that the Instagram employees and APAG members hugged at the end of the deeply emotional meeting. She realizes Instagram has a challenging road ahead about but wants to help them as much as possible to create an algorithm and a moderation structure where all bodies are valued.

The same goes for The Visible Collective and the many influencers we connect with and support. Collaboration rather than fear is the answer. The more helpful and constructive feedback the fat community can give, the better chance we have of helping Instagram better serve us.

Jessica Richman is the founder and CEO of The Visible Collective. The Visible Collective advises companies on product development, marketing, and new business development to better serve the 71% of customers labeled overweight or obese by the Centers for Disease Control and Prevention.

PluggedIn Newsletter logo
Sign up for our weekly tech digest.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Privacy Policy

ABOUT THE AUTHOR

Lydia Dishman is the senior editor for Growth & Engagement for fastcompany.com. She has written for CBS Moneywatch, Fortune, The Guardian, Popular Science, and the New York Times, among others More


Explore Topics