Final Tallies Minus Exit Polls = A Statistical Mystery! by John Allen Paulos
Why did the exit polls taken on election day in the battleground states differ so starkly from the final tallies in those states? As my crosstown colleague, Steven Freeman of the University of Pennsylvania has demonstrated in his paper, "The Unexplained Exit Poll Discrepancy," the pattern is unmistakable. In Ohio, Pennsylvania, and Florida, the differences between Bush's final tallies and his earlier exit poll percentages were, respectively, 6.7%, 6.5%, and 4.9%.
Similarly huge differences between the final tallies and the exit poll percentages occurred in 10 of the 11 battleground states, all of them in Bush's favor. If the people sampled in the exit polls were a random sample of voters, Freeman's standard statistical techniques show that these large discrepancies are way, way beyond the margins of error. Suffice it to say that the odds against them occuring by chance in just the three states mentioned above are almost a million to one.
Since exit polls historically have been quite accurate (there is no question about likely voters, for example) and the differences as likely to have been in one candidate's favor as the other's, we're confronted with the question of what caused them. Given the indefensible withholding of the full exit poll data by Edison Media Research, Mitofsky International, the Associated Press and various networks, we can only hazard guesses based on what was available election night. The obvious speculation, alluded to above, is that the exit samples were decidedly non-random.
Earlier voters across the country might have differed significantly from later voters. More women might have voted then or angrier partisans did or unemployed people walking their dogs wanted to cast their ballots sooner rather than later. This is hard to credit, however, without any supporting evidence for such an effect in other elections. Besides, the exit polls divide people along demographic lines, which is one of their primary functions, and weight repsonses accordingly if certain groups (e.g., blacks, males, 40-50 year olds) are over- or under-represented in the sample.
Another possible explanation for the discrepancy between the final tallies and the exit polls is that a fraction of the Bush voters were ashamed of their vote for him and lied to or avoided the exit pollsters. This happens regularly in polls on personal matters, but rarely in political polls. One example is David Duke, a former Ku Klux Klan wizard running for governor of Louisiana several years ago, who received many more votes than exit polls suggested he would because people didn't want to admit their preference for Duke and be labeled as racists. Bush is certainly no Duke and very few of his supporters seemed in the least shy, but an attenuated version of this phenomenon may be behind the difference. Or perhaps some evangelicals' aversion to exit pollsters as representatives of the "liberal media" is behind it. Who knows?
Absent any proof or compelling reasons for the differences between the final tallies and the exit polls in the swing states, I don't understand why these gross discrepancies are being so widely shrugged off. After all, the procuring of random samples is far more of a problem for ordinary telephone polls where the minority of people who cooperate with pollsters presumably differs in some way from the majority who don't. Still, these polls are not dismissed with the same impatient nonchalance as this year's exit polls.
Of course, what makes these discrepancies more than a technical problem in statistical methodology is that there is a much less likely, much more ominous explanation for them: massive fraud. Fraud is hard to believe for many reasons, one being the widespread nature, extending over different states and regions, of the shift to Bush. The difficulty of concealing a conspiracy grows very rapidly with the number of conspirators.
But another disturbing possibility is that there was no co- ordinated conspiracy, but rather many people working independently to subvert the election. The election has prompted extensive allegations of fraud, some of which have been debunked, but many of which have not. In several cases non-trivial errors have been established and official tallies changed. And there is one more scenario that doesn't require many conspirators: the tabulating machines and the software they run conceivably could have been dragooned into malevolent service by relatively few operatives. Without paper trails, this would be difficult, but probably not impossible, to establish.
Hard evidence? Definitely not. Nevertheless, the present system is such a creaky patchwork and angry suspicions are so prevalent that there is, despite the popular vote differential, a fear that the election was tainted and possibly stolen. (If 68,000 Ohio Bush supporters - only about a half dozen voters per precinct in the state - switched their votes, Kerry would be president-elect. Considerably fewer switches would be required if, as is likely, most provisional and spoiled ballots were good and went for Kerry.) A high-level commission should thoroughly examine the exit poll discrepancies and our electoral apparatus in general.
This is not a partisan issue. People differ about whom they want in the White House, but almost everybody wants whoever is there to be seen by all as having been rightfully elected.
---
Professor of mathematics at Temple University and winner of the 2003 American Association for the Advancement of Science award for the promotion of public understanding of science, John Allen Paulos is the author of several best-selling books, including Innumeracy and A Mathematician Plays the Stock Market.