As the results came in for Florida on Tuesday night, many Democrats had a sinking feeling that Donald Trump was once again on the way to an upset win. They’d been hoping for a decisive Biden win on election night, and for Democrats to flip the Senate. Their hopes were by no means unfounded—they’d been consuming a steady diet of poll numbers pointing to those things for weeks, months.
The Washington Post-ABC News poll (rated A+ by FiveThirtyEight) had Biden up by 17 points—57% to 40%—over Trump in battleground state Wisconsin. The Cook Political Report said Texas, a traditional GOP stronghold, was a “toss-up” this year. The pollsters told us suburban women had abandoned Trump.
But for the second presidential election in a row, the polls got it wrong in the most important states and Senate races. Biden eked out a win in Wisconsin, but by less than a point, not 17. Texas stayed red. More suburban women voted for Trump than in 2016. And pollsters have found themselves in a credibility crisis. How can we ever trust them again?
First, it’s important to understand how polling works. Pollsters try to form a sample group of people that looks like a microcosm of the electorate—one that proportionately reflects all demographic and ideological groups. If some group is over- or under-represented, projections made using the sample may be skewed.
By most accounts, Trump won not by broadening his base so much as he did by drawing more votes from the same demographic groups he tapped in 2016. Chief among those groups is white Republicans without college degrees, who came out in unprecedented numbers and were once again underestimated by pollsters.
The Shy Trump Voter
One popular theory about how the pollsters may have missed that block of voters is “the shy Trump voter,” referring to Trump supporters who for one reason or other weren’t honest with phone surveyors about their candidate of choice.
The Shy Trump Voter does exist, says Columbia Business School professor Michael Slepian, who did a study of them after the 2016 election. He says these voters hide their intention to vote for Trump for two main reasons. They sometimes do so to avoid getting into arguments with friends or family. But the main reason is that they’re trying to control how people see them.
“It turns out that concern for their reputation is the one predominantly linked to the folks who did not want to admit that they were supporting Trump,” Slepian says. People fear that being outed as a Trump supporter could hurt their social standing or professional opportunities, Slepian says. And there’s every reason to believe that the political division and rancor around the 2016 election may have only increased in 2020.
Operating on this theory, a group of researchers from USC, MIT, and the Sante Fe Institute developed a different way of finding out if a person intended to vote for Trump. Instead of asking the question directly, they asked people to estimate the percentage of their social contacts who would vote for each of the candidates. The “social circle” question was added to this year’s USC Dornsife Daybreak poll, and the answers that came back suggested that Trump would get enough votes to win the Electoral College. While results are promising (the social-circle question responses predicted Biden would narrowly win the popular vote) the surveying approach needs further testing with larger samples.
While Shy Trump Voters probably did present a problem for pollsters this year, a larger problem may have clouded pollsters’ view of the electorate.
Why some voters are invisible in the polls
First, a little history. Opinion research has gotten much harder in the 21st century. Once upon a time, it was possible to generate a good random sample by just calling people on their home phones, veteran GOP pollster Scott Tranter tells me.
“In the ’60s, ’70s, and ’80s, they could call 100 people and get response rates of 30 or 40%,” says Tranter, who now works at the election data firm Decision Desk HQ. “Now they call a hundred people and get two or three.” That’s partly because we use cell phones now, and in the age of phone spam we’ve learned not to pick up calls from numbers we don’t recognize.
That raises questions about the two or three who do complete a phone survey, and how representative of large groups of voters they really are, Tranter says.
One of the voter groups that’s least likely to take a phone survey, or even pick up the call, is the block of non-college-degreed voters that make up a significant part of Donald Trump’s base, says Mike Greenfield of the opinion polling company Change Research.
The difficulty reaching those people once mattered less because they were more evenly distributed between the two major parties. However, white voters without college degrees have migrated to the Republican Party in high numbers over the past 20 years, Greenfield tells me, as they perceived that the modern Democratic Party no longer had blue-collar interests at heart.
One of the hard lessons pollsters learned in 2016 is that they can no longer make assumptions about white, non-degreed voters based on the political preferences of an adjacent demographic group. This concept works nicely for some population groups: The political preferences of Blacks with a college degree and Blacks without one aren’t so different, for example. Roughly 9 in 10 people in both those groups chose Biden for president.
But it works terribly for non-college-degreed whites. That group’s views don’t line up with those of college-degreed whites very well at all. Edison Research exit poll data shows that 50% of white males with degrees voted for Biden while 48% chose Trump. White males without college degrees, meanwhile, broke heavily for Trump: 67% to 30%. White females with degrees narrowly chose Trump over Biden, 50% to 49%, while non-degreed white women broke for Trump 60% to 39%.
Some pollsters acknowledge that they undersample these hard-to-reach non-college degreed whites, then try to compensate by giving them extra weight in projection numbers. But that calculation misses something. Without actually talking with a representative sample of those voters, it’s hard for pollsters to understand their level of enthusiasm and the likelihood that they’ll turn out to vote.
So, whites without a college degree are not only the hardest to reach, but their views also can’t be ascertained without direct sampling. Greenfield believes this is a more likely explanation for pollsters’ misleading projections in the months before this year’s election than the Shy Trump Voter.
It may be up to technologists to find ways of surveying hard-to-reach voters.
Research by Professor Emilio Ferrara, also of USC, suggests that non-degreed whites may have another reason for shunning pollsters. Believers in the QAnon conspiracy theory have a deep mistrust of establishment media outlets like the New York Times and the Washington Post—the same organizations that sponsor or conduct political polls—and so may hang up when they hear those names on the phone. Ferrara found that major polls underestimated Trump support the worst in states where Q-related Twitter activity was highest.
Tranter says it may be up to technologists to find ways of surveying hard-to-reach voters. Greenfield, who was the first data scientist at PayPal and then LinkedIn, co-founded Change Research with the idea of bringing a Silicon Valley approach to the problem—relying solely on the internet to recruit and interview voters. Others are hoping to leverage artificial intelligence to more accurately sample voter opinion.
A referendum on pollsters?
Not all pollsters got it wrong this year, and most mainstream pollsters got some big things right. Most picked Biden to win, and he did. They also measured Biden’s appeal correctly in most states. And while exit poll data provides some insight on what happened, pollsters and other researchers have only just begun digging into the real numbers.
None of that is likely to keep the public’s trust in political pollsters from eroding.
But the pollsters may not deserve all of the blame. Many consumers of polls have a vague understanding of pollsters’ disclaimers that the numbers are just probabilities and that margins of error matter. In addition, polling numbers are treated as if they were the running score of a sporting event in the months and weeks leading up to an election. Polling is a matter of data science, not infotainment.
So while pollsters’ reputations and methods take a beating, both pollsters and poll consumers have work to do before 2022.
“There’s going to need to be more work determining the polling error,” Tranter says. “And the public needs to understand what that polling error means.”
Correction: An earlier version of this story stated the “social circle” question was developed by “researchers at USC.” Actually, it was a team of researchers from USC, MIT, and the Sante Fe Institute. It also failed to mention that while the social-circle question responses predicted an electoral college win for Trump, it also predicted a narrow popular vote win for Biden.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.
