Nate Silver of FiveThirtyEight, the master statistician who predicted the outcome of the 2008 election down to the smallest detail (and who famously said politics is a far easier realm for statisticians than baseball, his usual stomping grounds), has a really interesting piece examining the problems with cellphone-only households and that effect on political polls.
The rise in cellphone-only households isn’t a new thing, but it’s also increasing in percentage. Half of Americans between the ages of 25 and 30 are cellphone-only, and two-thirds are cellphone-mostly (meaning they may have a landline for more expensive uses like long distance calling, but may not ever answer incoming calls). Even worse, sub-30-year-old Americans are more accustomed to screening calls than any other generation, making it tougher for an unknown pollster to get through. (Interestingly, the percent of landline-using households are slightly higher for those aged 18-24–probably since much of that demographic is either living in a college dorm or at home.)
Polling cellphones is expensive, often prohibitively so, for pollsters. The cost is around twice as high as it is to call landlines, so while many of the bigger organizations (Gallup, Pew, CBS/NYT, Quinnipiac) do call cellphones, the smaller pollsters often have to merely weight more heavily the demographics they assume cellphone users correspond to. And that’s a problem.
So the cellphone-only demographic, which Silver estimates could “be in the high 20s” by the November midterm election, is often underrepresented in polls. Silver describes this demographic thusly:
Cellphone-only households are different from their landline-using counterparts. They tend to be younger, poorer, more urban, less white, and more Internet-savvy. All of these characteristics are correlated with political viewpoints and voting behavior.
But some of the weighting used by pollsters that have inadequate cellphone polling results are inaccurate. Silver says it’s “somewhat rare” for pollsters to weight their polls “by characterists like urban/rural location or marital status, which are predictive of both cellphone usage and political beliefs.” Even if this weighting is assumed to be reasonable, which it may not be, the response rates for cellphone-only households are often so small that weighting them appropriately results in huge margins of error.
It’s a tough situation for pollsters. Calling cellphones is expensive and difficult, with drastically lower response rates even for those who can afford it. Robodialers are illegal. And there’s no easy solution–robotexts are likely to be heavily restricted as well, and even more costly for both the pollsters and the polled.
Silver suggests altering weighting techniques to focus on the “non-traditional” criteria like “urban/rural status, technology usage, or perhaps even media consumption habits” instead. But there’s no easy solution. We just have to remember that these polls may be inherently troubling, statistically speaking.