Seth
----- Original Message ----- From: "Michael Pollak" <mpollak at panix.com> To: <lbo-talk at lbo-talk.org> Sent: Thursday, September 23, 2004 7:23 AM Subject: [lbo-talk] Ruy Teixeira on why most of the polls are crap
>
> [A powerful extension of the LV critique, plus an explanation of why the
> RV numbers are widely wrong as well: certifiable Republican oversampling.
> It's not just the CBS poll. Gallup's got the same problem. And although
> it seems like a million polls came out this last week, 95% have just been
> those two, quoted in a million different ways.]
>
> Public Opinion Watch - Sept. 22, 2004
> Ruy Teixeira
>
> Decoding the Gallup and New York Times Polls
>
> Gallup poll of 1,022 adults, released September 17, 2004 (conducted
> September 13-15, 2004)
>
> CBS News/New York Times poll of 1,287 adults, released September 18, 2004
> (conducted September 12-16, 2004)
>
> -----------------
>
> Here are Bush's leads in the three national polls released before Gallup's
> current poll (no registered voter [RV] data available for Democracy Corps
> and Harris, only likely voter [LV] data; Pew and Harris matchups include
> Nader):
>
> Democracy Corps, September 12-14 LVs: +1
>
> Pew Research Center, September 11-14 RVs: tied
>
> Harris Interactive: September 9-13 LVs: -1
>
> Looks like a tie ball game, right? But according to the Gallup poll
> conducted September 13-15 and released September 17, Bush is up . . . by
> thirteen points???
>
> Let's just say I'm just a wee bit skeptical of this one. First, Gallup's
> poll only includes one day (September 15) that the three other polls do
> not cover, so it can't be Gallup's survey dates that explain the big Bush
> lead.
>
> Second, this thirteen-point lead is an LV figure and, as I've repeatedly
> emphasized, Gallup's LV screening procedure produces completely
> untrustworthy measures of voter sentiment this far in advance of the
> election. Here is a summary of the case against Gallup's LV data:
>
> Sampling likely voters is a technique Gallup developed to measure voter
> sentiment on the eve of an election and predict the outcome, not to track
> voter sentiment weeks and months before the actual election. There is
> simply no evidence, and no good reason to believe, that it works well for
> the latter purpose. In fact, the evidence and compelling arguments are on
> the other side: that the registered voters are the more reliable gauge of
> voter sentiment during the course of the campaign.
>
> Here's why. Gallup decides who likely voters are based on seven questions
> about their interest in voting, attention to the campaign, and knowledge
> about how to vote (e.g., where their polling place is located). The
> interested/attentive/knowledgeable voters are designated "likely" and the
> rest are thrown out of the sample. But as a campaign progresses, the level
> of interest among voters tends to change, particularly among those with
> partisan inclinations whose interest level will rise when their party
> seems to be mobilized and doing well and fall when it is not. Because of
> this, partisans of the mobilized party (lately, Republicans) tend to be
> screened into the likely voter sample and partisans of the demobilized
> party (lately, Democrats) tend to get screened out. But tomorrow, of
> course, the Democrats could surge, in which case their partisans may be
> the ones over-represented in likely voter samples.
>
> That suggests the uncomfortable possibility that observed changes in the
> sentiments of "likely voters" represent not actual changes in voter
> sentiment, but rather changes in the composition of likely voter samples
> as political enthusiasm waxes and wanes among the different parties'
> supporters. And that is exactly what political scientists Robert Erikson,
> Costas Panagopoulos, and Christopher Wlezien find in their analysis of
> Gallup's 2000 RV/LV data in their forthcoming paper, "Likely (and
> Unlikely) Voters and the Assessment of Campaign Dynamics" in Public
> Opinion Quarterly: "shifts in voter classification as likely or unlikely
> account for more observed change in the preferences of likely voters than
> do actual changes in voters' candidate preferences."
>
> That means that, instead of giving you a better picture of voter sentiment
> and how it is changing than conventional registered voter data, likely
> voter data give you a worse one since true changes in voter sentiment are
> swamped by changes in who is classified as a likely voter.
>
> I think the case against the Gallup LV data looks rock solid. In my view,
> it's time for them to drop reporting these data because they are highly
> likely to give an inaccurate picture of the state of the race and, by
> doing so-especially given the high profile of Gallup's polls-unfairly pump
> up one side of the race and demoralize the other. That doesn't seem
> acceptable to me.
>
> Of course they'll reply: well, our data work so well right before the
> election, they must be the best data to use all the time. But, for the
> reasons outlined above, that reasoning is completely specious. And then
> there's this: the LV data haven't been working so well lately even right
> before the actual election. In three of the past four presidential
> elections (including the most recent one), Gallup's final RV reading was
> actually closer to the final result than their final LV reading!
>
> As I have repeatedly argued, it's time for a serious rethink down at
> Gallup headquarters.
>
> Throwing out the Gallup LV data, then, let's move on to their RV result:
> an eight-point Bush lead. Obviously pretty far off the results of the
> other contemporaneous polls summarized above, but . . . could be, I
> suppose.
>
> But then there's this: the Gallup internals show Kerry with a seven-point
> lead among independent RVs. Huh? Kerry's losing by eight points overall,
> yet leading among independents by seven. How is that possible? Only if
> there are substantially more Republicans than Democrats in the sample.
>
> That suggests that reweighting the sample to reflect the 2000 exit poll
> distribution (39 percent Democrats/35 percent Republicans/26 percent
> independents) would give a different result. It does: the race then
> becomes dead-even, instead of an eight-point Bush lead. (Note: Steve Soto
> of The Left Coaster got Gallup to give him their party identification
> distributions for this poll and confirms a five-point Republican party
> identification advantage in their RV sample.)
>
> One other note: I mentioned the Pew Research Center poll had the race
> dead-even just as in the reweighted Gallup data. And what was Pew's party
> identification distribution in their RV sample? You guessed it: a
> four-point lead (37 percent to 33 percent) for the Democrats, just as in
> in the 2000 exits.
>
> Right after the Gallup poll was released, the latest CBS News/New York
> Times (CBS/NYT) poll came out. That poll, conducted September 12-16, also
> gives Bush an eight-point lead (50 percnet to 42 percent) among RVs, but
> also, not coincidentally, gives the Republicans a four-point edge on party
> identification. Reweight their data to conform to an underlying Democratic
> four-point edge, as in the 2000 exit poll, and you get a nearly even race,
> 47 percent Bush/46 percent Kerry.
>
> Nearly even. That goes along with the 46 percent to 46 percent tie in the
> Pew Research Center poll (which gave the Democrats a four-point edge on
> party identification without weighting) and the 48 percent to 48 percent
> tie in the Gallup poll (once weighted to reflect an underlying Democratic
> four-point edge). Not to mention the two other recent national polls
> (Harris, Democracy Corps) that show the race within one point.
>
> Perhaps all this is just a coincidence, but the pattern seems striking.
> Once you adjust for the apparent over-representation of Republican
> identifiers in some samples, the polls all seem to be saying the same
> thing: the race is a tie or very close to it.
>
> Looking past the horse race result, there is another aspect of the new
> CBS/NYT poll that deserves emphasis. As Chris Bowers of MyDD points out in
> his insightful new essay, "Rapid Poll Movement Is a General Election
> Myth," (http://www.mydd.com/story/2004/9/18/454/00913#readmore) the new
> CBS/NYT poll is actually a lot worse for Bush than the CBS News poll
> released just a week earlier. That's because, since the current poll is
> substantially more Republican than the earlier poll (which actually had a
> slight Democratic edge), Bush should actually have performed better than
> the earlier poll on the horse race and on indicators like job approval and
> right/direction wrong track in this poll, instead of about the same. That
> also means that if we adjust the current poll to correct the apparent
> surplus of Republicans, Bush's performance on these indicators should
> actually decline below the measurements of the earlier poll.
>
> Since CBS News thoughtfully provides the overall result and the result
> broken down by party identification for each and every question in their
> survey, it is possible to estimate what Bush's ratings would have looked
> like if there weren't so many Republicans in the sample. Here are some
> examples, based on reweighting the current poll to the 2000 exit poll
> distribution of partisanship:
>
> Overall job approval: 49 percent approval/44 percent disapproval Economic
> job approval: 42 percent/52 percent Iraq job approval: 45 percent/51
> percent Campaign against terrorism job approval: 57 percent/37 percent
> Right direction/wrong track: 40 percent/53 percent
>
> In every case, these ratings are worse than they were a week ago, making
> the idea that the race is tightening up more plausible.
>
> Of course, Kerry needs not just a tight race, but to pull ahead. Given
> Bush's continued vulnerabilities, which these data highlight, Kerry's got
> the opening to do so. We shall see if he is able to take advantage of this
> opening.
>
> ====
>
> The Century Foundation is nonprofit and nonpartisan and was founded in
> 1919 by Edward A. Filene. [Filene, a Jewish Boston Brahmin who was one of
> the most prominent figures during the golden age of American aristocratic
> liberalism around the turn of the century, is credited with one of my
> favorite quotes. Inventor of the bargain basement, he reportedly said
> "Why shouldn't the American people take half my money from me? I took all
> of it from them!"]
> ___________________________________
> http://mailman.lbo-talk.org/mailman/listinfo/lbo-talk
>