[From his newsletter, compiled from various entries on his blog]
Gallup Poll Racially Biased
By this I don't mean that Gallup's pollsters are themselves racially biased. Rather I mean that their likely voter (LV) samples-whose results Gallup continues to promote above all others-tend to be racially biased because of the methodology Gallup employs to draw them.
Here's a basic sketch of how Gallup's methodology works:
Gallup asks each [RV] respondent seven LV screening questions, and gives each person an LV score of 0 to 7. [Assuming a turnout of 55 percent], the top 55% are classified as likely voters. In practice that typically means all of the "7"s-given full weight-plus some proportion of those with lower scores (usually the "6"s), who are weighted down so that the size of the likely voter sample matches the projected turnout for the year (apparently 55 percent this year). All other voters are discarded from the sample.
Note that the demographics of Gallup's LV sample are not adjusted in any way (as their overall samples are) and are simply allowed to fall where they may.
What this means is that if, say, minority voters are much less likely to answer the seven questions "right," they will be correspondingly under-represented in the LV sample-perhaps severely under-represented.
That is exactly what turns out to be the case. According to data obtained by Steve Soto over at the Left Coaster, Gallup's latest LV sample-the one that showed Bush with an eight-point lead-has only 14.5 percent minority representation and only 7.5 percent black representation.
How plausible is this as a representation of the election day electorate? Not remotely plausible. In 1996, minority representation among voters was 17 percent; in 2000, 19.4 percent. In 2004, the minority proportion of voters should be more than this, because minorities are growing, not declining, as a percentage of the U.S. population. So 14.5 percent for nonwhites as a prediction of the 2004 electorate is very, very unlikely. It would defy both recent history and powerful demographic trends.
As for 7.5 percent blacks? Come on. Blacks were 10.1 percent in 1996 and 9.7 percent in 2000. And they're 12 percent of the voting age population. There's just no way in the world blacks will only be 7.5 percent of voters in 2004.
So, in effect, Gallup's likely voter approach is disenfranchising minorities in assessing American voters' inclinations on the coming election. That's wrong and Gallup should stop doing it.
And speaking of disenfranchisement, how about America's young people? This group is also full of voters who are relatively unlikely to answer the seven LV questions right and thus qualify for admission into the exalted realm of the Gallup LV sample.
Sure enough, Gallup informs us that young voters (age eighteen to twenty-nine) only compose 11 percent of likely voters. Well, that would be quite a trick. In 1992, young voters were 21 percent of voters; in 1996, 17 percent of voters; and in 2000, 17 percent again. And we're supposed to believe that young voters are all of a sudden going to drop to 11 percent this year? Please, this doesn't pass the laugh test.
As it happens, minorities-no big surprise-lean very heavily toward Kerry this year. But young voters are also Kerry's best age group this year. Systematically under-representing these groups in Gallup's LV samples will therefore have an obvious, and fairly substantial, effect on their results, tilting them in the direction of Bush and the Republicans.
That's not right. Gallup should know better. And we should all know better than to trust results that are based on effective disenfranchisement of large numbers of minority and young voters.
Paul Krugman does know better. He had this to say in his October 22 column in the New York Times:
<quote>
If the election were held today and the votes were counted fairly, Senator John Kerry would probably win. But the votes won't be counted fairly, and the disenfranchisement of minority voters may determine the outcome.
Recent national poll results range from a three-percentage-point Kerry lead in the A.P.-Ipsos poll released yesterday to an eight-point Bush lead in the Gallup poll. But if you line up the polls released this week from the most to the least favorable to President Bush, the polls in the middle show a tie at about 47 percent.
This is bad news for Mr. Bush because undecided voters usually break against the incumbent-not always, but we're talking about probabilities. Those middle-of-the-road polls also show Mr. Bush with job approval around 47 percent, putting him very much in the danger zone.
Electoral College projections based on state polls also show a dead heat. Projections assuming that undecided voters will break for the challenger in typical proportions give Mr. Kerry more than 300 electoral votes.
<unquote>
But he goes on to point out that this picture is not the one you would get from watching cable news, where polls of Gallup and Fox News set the frame for the state of the horse race. And he specifically cites the data reported by myself and others exposing the bias of Gallup's likely voter samples against minority and young voters.
As Krugman emphasizes, this distorted view of the state of the campaign not only misleads voters and the public, it potentially can be used to cover up actual disenfranchisement of minority and young voters on election day. Perhaps Krugman's column will finally help get "the Gallup problem" the attention it so richly deserves.