Why Is Santorum Overperforming His Poll Numbers? Rick Santorum did something unusual on Tuesday: he won a state, Mississippi, in which no poll had shown him with a lead. This was not the first time. He did the same thing in Iowa, although the case could have been made that the polls there were understating his momentum. Mr. Santorum had also never led a survey of Colorado voters, although there was just one poll there. And he trailed either Mitt Romney or Newt Gingrich in the last seven polls of Alabama, although he did hold a lead in one survey of the state earlier on. The FiveThirtyEight forecast model is based on statewide polls and statewide polls alone in presidential primaries and caucuses. It doesn't do anything especially fancy. This approach has advantages and disadvantages. The disadvantage is that if the polls are wrong, the model will be wrong too. I will try to alert you when I think there's an especially large chance for a polling mistake, as I did before Alabama and Mississippi. Still, the Mississippi polls implied that Mr. Santorum had only about a 2 percent chance of winning the state, according to the model. He trailed both Mr. Romney and Mr. Gingrich by seven or eight points in the polling average there. Against one candidate, such a deficit is sometimes surmountable, but against two it has very rarely produced a victory. The advantage is that the model gives us a relatively clean and objective benchmark to evaluate the candidates" actual performance against the polls. So how has Mr. Santorum done against his forecasts? So far, 11 states have had robust polling, which I define as multiple polls in the field in the days immediately before a primary or caucus. These states provide the clearest measure of how a candidate did against his polls on primary day - cases in which the polling was sparse, or there was a late swing in the momentum that the polls were not current enough to capture, are excluded. In these states, Mr. Santorum has outperformed his forecasts by an average of 2.3 percentage points thus far. That isn't a huge difference, but it was enough to swing three states for him that the model predicted he would lose - Alabama, Iowa and Mississippi. In fact, the difference is statistically significant as measured by standard tests - unlikely to have occurred by chance alone. Mr. Santorum's overperformance against the polls has not necessarily equated to underperformance for Mr. Romney, who has beat his polls in some states but lagged them in others. Instead, Newt Gingrich and Ron Paul have tended to underperform their polls in these states. We could also look a broader set of states. I issued FiveThirtyEight forecasts for Massachusetts, Nevada, Oklahoma and Virginia even though there was limited polling there. And Public Policy Polling published polls on the weekend immediately before the Missouri "beauty contest" primary and the caucuses in Washington, Colorado and Minnesota. We can turn those Public Policy Polling polls into "forecasts" by removing the undecideds from the vote, as the FiveThirtyEight model does. Looking at this broader set of states does not change Mr. Santorum's numbers much. He is helped in states like Colorado and Minnesota, but harmed in Washington and Massachusetts. Over all, he has beaten his polling forecasts by an average of 2.1 percentage points. This broader set of states does show Mr. Romney slightly underperforming his polls, however, and it shows Mr. Gingrich doing especially poorly against them. But let's keep the focus on Mr. Santorum. Why might he be beating his numbers? I can think of four major hypotheses: Voter enthusiasm. It's possible that polls draw the likely voter universe too broadly, and do not sufficiently account for the greater propensity of Republican voters to turn out for Mr. Santorum. There is some circumstantial evidence to support this hypothesis: polls and turnout demographics have generally suggested that Mr. Santorum's voters are more enthusiastic than Mr. Romney's - and this advantage may be especially significant where turnout is relatively light, as in caucuses. There is also some history of very conservative candidates" outperforming their polls in Republican primaries and caucuses. This has been clearest in Iowa. In our database of past polls, moreover, the candidate who most consistently beat his polling numbers was Pat Buchanan, who would have beat his FiveThirtyEight forecasts (had they been issued at the time) by an average of four percentage points. Cellphone-only voters. Most primary and caucus polls have not surveyed voters who have a cellphone but no land line - a group that now includes about a quarter of the American population and is rising. This holds for almost all "robopolls" conducted by automated script, as well as some polls that use a more traditional methodology but haven't chosen to include cellphones in their samples, since doing so is more expensive. There is some pretty good research on the effects this might have in the context of a general election. The short story is that the failure to include cellphones probably biases the numbers slightly against Democratic candidates and liberal causes. But there has been very little about what it might do in the context of a party primary. So far, I haven't detected a big split in the polls in states in which some polls included calls to cellphones and some excluded them. But such cases have been fairly rare: in some states, including Alabama and Mississippi, we have had to rely almost entirely on polls that essentially disenfranchised cellphone-only voters. There has been a positive although weak (and not statistically significant) relationship between the fraction of cellphone-only voters in a state and Mr. Santorum's tendency to overperform or underperform the polls there. This trend is mostly driven by just two examples, however: New Hampshire, which contains an especially low number of cell-only voters and where Mr. Santorum underperformed his polls, and Mississippi, which contains an especially high number and where he overperformed them. It is quite possible that the discrepancies in these states where driven by other factors and that the exclusion of cellphones from the polls was not a major cause. Another strike against the hypothesis is that Ron Paul, whose demographics are probably especially cellphone-friendly, has not really outperformed his polls so far. Still, it's going to create problems in the long run when you are summarily excluding a quarter of the American population from your surveys, and these issues may manifest themselves in unpredictable ways. It would be helpful to have pollsters that have done more detailed research on how the exclusion of cellphone-only voters can skew samples - the three that come to mind here are Pew, SurveyUSA and Marist - examine this issue in more detail. Southern hospitality and social desirability bias. It could also be that the type of voter who is more likely to vote for Mr. Santorum is less likely to be forthcoming about his private choice to pollsters. One version of this theory - and the reason I thought polls in Alabama and Mississippi might go wrong - is what I call the "Southern hospitality" hypothesis. It holds that in parts of the country where etiquette codes are stronger, voters may tell pollsters that they plan to pick the most broadly acceptable choice (especially an establishment-backed front-runner like Mr. Romney) rather than disclosing their first preference. Some evidence for this is that Mr. Santorum has tended to outperform not just his pre-election polls but also the exit polls - especially in Iowa and Mississippi. Since exit polls are conducted in person, they should not have any problems with missing cellphone-only voters. And since they are conducted on primary day itself, they shouldn't have any problems with turnout models - they are surveying actual voters rather than "likely voters." In addition, almost all of Mr. Santorum's polling overperformance has come from the South. Between South Carolina, Florida, Georgia, Tennessee, Alabama and Mississippi, he outperformed his polls by an average of three and a half points - but he has run about evenly with them elsewhere. Tactical voting. A final hypothesis might be that some people are switching their votes on primary day toward Mr. Santorum and against candidates like Mr. Gingrich, whom they see as less viable. Voters quite rationally do not want to waste their votes, and there is some evidence that this played a role in Mr. Santorum's late surge in Iowa. But this theory does not do a good job of explaining the results in states like Tennessee, Alabama or Mississippi - places where Mr. Gingrich seemed to have a decent shot of winning. Nevertheless, it is worth remembering that the publication of polls can sometimes directly affect voter behavior, especially in multicandidate races. In the South, where Mr. Romney's favorability ratings are mixed even among Republican voters, tactical behavior on the part of voters could have especially meaningful effects. I don't think we're quite at the point where you should add a couple of points to Mr. Santorum's numbers as a default. Indeed, in high-population states that are expensive to advertise in - like Michigan and Florida - it is Mr. Romney rather than Mr. Santorum who has tended to beat his polling numbers. Still, there have been cases in the past where a candidate consistently outperformed or underperformed his primary polls. Barack Obama usually beat his numbers in 2008. This is a reason to look at demographic-based forecasts in addition to the polls. Particularly when the states resemble others that a candidate has already won, these forecasts may give you a hint about the direction that polls are likely to miss.