In the days and weeks ahead, there will be a lot of attention paid to the wide disparity between most preelection polls and the actual results. However, there’s a technology that could do a much better job of tracking and forecasting voter sentiment—artificial intelligence.
AI is already widely used to gauge consumer sentiment. Amazon mines huge amounts of data about search and online purchasing behavior to uncover insights about what customers buy and predict what they might want to buy in the future. Netflix bases its movie recommendations on what you’ve watched in the past, for how long, and how favorably you review the show.
In all of those cases, consumer sentiment is being gauged not by what consumers say they want but by their actual behavior. That’s a crucial difference, and one that gives AI a significant advantage over traditional polls. People don’t always tell the truth to a pollster who suddenly calls them on the phone at dinnertime. But what voters say to their friends on social media is usually a more reliable reflection of their true opinion.
AI can analyze far more data than standard opinion polls and can better track the momentum behind a candidate based on an analysis of everything from campaign contributions and volunteer activity to social media chatter. Already, several AI companies have created models to make election predictions. Expert.ai, a software company specializing in natural-language processing, used AI to analyze social media posts about Donald Trump and Joe Biden in the months leading up to the 2020 election. The company’s AI uses sentiment analysis to understand the emotions expressed in social media posts and then project how that could translate into votes.
Expert.ai’s system looked at nearly half a million tweets and other social media comments made before the election. Using natural language processing, the AI categorized the sentiment expressed in social media posts according to more than 80 different emotional labels. These labels were given an intensity ranking and then multiplied by the number of occurrences for each candidate.
The emotions were classified as either “positive” or “negative” and used to create an index that compared the two candidates. Trump was the subject of significantly more social media activity than Biden, accounting for almost 60% of all the posts Expert.ai analyzed, compared to about 17% for Biden (23% of the posts the startup analyzed didn’t mention either candidate). Biden ranked higher for emotions such as “success” and “hope,” while Trump scored higher on emotions such as “fear,” “action,” and “hatred.”
After crunching all of that voter sentiment data and taking into account the number of candidate mentions and how they related to emotions, Expert.ai made its prediction just before Election Day. The startup projected that Joe Biden would win 50.2% of the popular vote, and Donald Trump would get 47.3% of the vote, a 2.9% spread. That prediction turned out to be uncannily accurate—the latest figures show Biden winning 50.8% of the popular vote and Trump receiving 47.5%, a 3.3% margin. That prediction would be the envy of any traditional pollster.
Another AI company, Unanimous.ai, used its software to survey people in the U.S. in September, asking them to predict who they thought would win the presidential contest in 11 battleground states and by what margin. Unanimous.ai connects large groups of people over the internet, building a “swarm intelligence” that amplifies the participants’ combined knowledge and insights.
The results were impressive: Unanimous.ai correctly predicted the winner of the presidential vote in 11 of the 11 battleground states. (Unanimous.ai also forecast 94% odds that if Trump lost the election, he would blame election fraud and 82% odds that he would refuse to concede for at least two weeks after the vote is certified.)
Algorithms have the potential to capture a far more expansive picture of election dynamics than traditional polling because they draw on indirect signals rather than relying on what people tell a pollster. It’s not even necessary for these signals to explicitly mention a preference for a particular candidate. People who post on social media about “law and order” tend to lean Republican while people who post about global warming lean Democratic.
AI could also address one weakness in traditional polling that’s often overlooked: Polls are weighted based on academic metrics and are not risk-adjusted based on the consequences of an incorrect prediction. Artificial intelligence could do a much better job of tracking voter sentiment and identifying the degree of uncertainty about predictions. Using AI models that weigh the consequences of an incorrect prediction would give a risk-adjusted prediction that considers the cost of getting it wrong. For example, it would try to avoid choosing the wrong winner over getting the margin of the win exactly right.
Of course, AI isn’t infallible. As with polling, algorithm-driven forecasts can miss the mark due to methodological errors or failing to properly sample traditionally underrepresented groups, such as ethnic minorities. Younger voters and urban residents post on social media much more frequently than older and rural voters, which can skew any online sentiment analysis, and it’s not always easy to distinguish bots from real people online. In addition, these AI polls aren’t able to predict voter turnout, which typical polls incorporate. But it’s clear that traditional polling is no longer a reliable way to track what voters are really thinking. It’s time for AI to drag polling into the 21st century.
Arijit Sengupta is the founder and CEO of Aible.
Recognize your brand’s excellence by applying to this year’s Brands That Matter Awards before the early-rate deadline, May 3.
