Can You Tell How Dangerous A Neighborhood Is From Just A Picture?

People are surprisingly good at determining how safe and how rich a neighborhood is just by looking at an image on Google Street View. What does this say about how the built environment interacts with crime and income inequality?

Can You Tell How Dangerous A Neighborhood Is From Just A Picture?
[Image: Andrew F.Kamierski via Shutterstock]

In 1921, a young Swiss psychiatrist named Hermann Rorschach wrote a book called Psychodiagnostik containing 10 inkblot images he used to gauge mental patients’ emotional processes. Based on how the patients interpreted random visual cues, administrators of the test believed they were granted windows into patients’ psyches. In 2013, MIT researchers are using a similar approach to learn about cities, but with something much more sophisticated than inkblots.


In 2010, César A. Hidalgo, a professor and director of the Macro Connections Research Group at MIT’s Media Lab, started building a web tool to collect a multi-dimensional range of information about people’s feelings toward certain city neighborhoods. Users compared one Google Street View image against another, answering questions like, “Which place looks more boring?” or “Which place looks wealthier?” The team sourced images from four cities–Boston and New York in the United States, and Linz and Salzburg in Austria–then quantified the results in a new paper published in the journal PLoS One last month. Overall, researchers identified more clusters of strongly positive and negative views about neighborhoods in the American cities–showing that “emotional inequality” of neighborhoods was uniquely higher in Boston and New York.

This map shows how safe people perceive New York neighborhoods to be, based on how they look.

Some of the highest contrast was found in Brooklyn and Queens, which you can see in the team’s new results in the maps here. Researchers measured some 2 million clicks based on perceptions of safety and class, and generally, perception matched up with incidence of violent crime and income.

Surprisingly, users of the web tool ranked one of the most recently gentrified neighborhoods in Brooklyn–Greenpoint–below East New York in both categories. In the last year alone, East New York’s number of major crimes more than tripled compared to Greenpoint, but users still ranked Greenpoint streets as feeling less safe. The same held true for perceptions of class, despite the fact that Greenpoint’s median household income (in 2010, the latest year data is available) is higher than the total New York average at $58,311, while the median household in East New York makes $34,295.

This map shows which neighborhoods look richer (dark blue):

Hidalgo notes that the people ranking the images likely did not live in New York, and could have thought they were looking at, say, Philadelphia or Baltimore. But the results also potentially highlight New York anomalies–like, for example, gentrifiers deliberately moving to industrial-looking neighborhoods and seeking out a ruinous aesthetic (that’s my personal hypothesis). Hidalgo, meanwhile, suggested it could mean that gentrification in some specific areas is a largely internal process, while the built environment remains relatively static (i.e. Bushwick, where rich people now live in old warehouses, but not the Williamsburg waterfront, where giant new condos have sprung up to house a new population).

Either way, if the tool gets enough clicks for the data to take a stable shape in other cities, it could inform theories about how visual cues in urban environments interact with crime, income inequality, and a number of other qualities of city life.


For example, Broken Windows Theory (BWT), the notion that environmental disorder (like broken windows, trash, and tagging) paves the way for social disorder (like crime), informed an era of “zero tolerance” for minor offenses during the Giuliani administration. The crime rate fell dramatically in subsequent years, but whether BWT or the crackdown on “quality of life” crimes had anything to do with it is still debated today. With the growth of NYPD stop-and-frisks targeted toward minorities over suspected misdemeanors, some argue that BWT created a legacy of racial profiling–or distracted from other issues at hand.

Hidalgo’s team actually found a strong correlation between incidence of violent crime and perceptions of danger based on visual cues, which could support BWT, or perhaps the inverse–that people avoid unattractive neighborhoods, making them unsafe. “There’s not a unique explanation of the correlation that will serve,” Hidalgo explains. “Both explanations are perfectly reasonable. There could be a third variable that is causing this correlation,” he adds.

Hidalgo and his team are working towards creating a set of algorithms to determine perceptions–and then perhaps links to correlations in crime, income, along with a number of other factors–without the crowdsourcing. With enough clicks, Hidalgo hopes to make computers capable of analyzing inequality in an urban environment and drawing a number of conclusions, just based on the Street View.

“American society is known to be more unequal than European societies when we look at income and access to services,” he notes. “What the paper shows is not only that the cities are more unequal, but are unequal in a number of dimensions outside of income.”

Now that researchers have figured out how to usher the data into measures of emotion by neighborhood, their next step is to expand the clicks to 56 other cities. “What we published in the PLoS One paper was the pilot to see how many clicks were needed for this technique to converge,” Hidalgo explains. “If you look at rankings now, the more lively city with the data that we have is London. And the more wealthy-looking cities are Singapore and Washington D.C. You’re seeing global patterns and global differences.”

Maybe, one day, all we’d need to analyze the collective perceptions or qualities of Tulsa, Oklahoma, would be a handful of Street View images and a set of smart algorithms, Hidalgo adds. But he also cautions that we ought to take his initial results with a grain of salt. “It’s important to find ways to get as much out of each click that allows you to extrapolate the pattern,” he says. “It is like developing a photograph–like you’re exposing light until you see a pattern. We have exposed light that’s a small fraction of a millisecond. Everything is out of focus.”

In the meantime, check out what 2 million clicks have to say about different New York City neighborhoods in the full data below. Some neighborhoods didn’t accumulate enough clicks, but the ones listed show standard deviations away from neutral positions in terms of perceptions of safety and class. You can also add your feelings to the ongoing study by clicking through images here.


NYC Neighborhood Class Rank

NYC Neighborhood Safety Rank

About the author

Sydney Brownstone is a Seattle-based former staff writer at Co.Exist. She lives in a Brooklyn apartment with windows that don’t quite open, and covers environment, health, and data.