On Fri, 16 Jan 2004, Doug Henwood wrote:
> The Zogby polling numbers in Iowa are probably useless. They don't
> disclose their methodology, and it's nearly impossible to predict who's
> going to show up at the caucuses.
A friend sent me this analysis from a professional pollster who prefers to remain anonymous
<quote>
As a pollster, I'm amazed at the way the Zogby's numbers seem to be
driving national coverage of Iowa for the last few days. It makes
me wonder how many reporters appreciate the basic math of caucus
turnout and the incredibly challenging task it creates for
pollsters.
Consider the first set of numbers: The previous record for
Democratic caucus turnout was 125,000 in 1988. "Democratic leaders"
seem to believe it will be higher this time (or so says the
oracle). Iowa has a voting age population of 2.2 million--125K is
about 6% of that. So even if turnout doubles over 1988, doing a
random digit dial (RDD) survey precisely means hanging up on
roughly nine of ten otherwise cooperative respondents.
So a pollster doing an RDD poll in Iowa can either spend a lot of
time and money screening or cut corners and interview anyone who
says they are "likely to vote." The latter approach yelds a
considerably higher percentage than 6-12% (you'll have to trust me
on that one, although Mark Mellman agrees).
The LA Times poll took the responsible approach, interviewing
3629 adults to get 640 likely Democratic caucus goers. Yes,
that's still 17%, higher than the likely turnout, but at least
in the ballpark. Given them credit for the expense involved and for
disclosing the details, something no other public pollster in Iowa
has matched.
There is another way--sampling from a list of past caucus goers--
which brings me to a second set of numbers: The Caucuses turnout
was only 60,000 four years ago, and far less in '96 and '92 when
they were uncontested. If turnout this time is 120K or more, then a
survey of past caucus goers could miss 50% or more of those who
actually show up on Caucus night. Odds are, those newcomers will be
different than the hard-core voters on the lists.
So what is Zogby doing? Who knows? He won't say. His web page
release provides no hint at how he samples caucus goers or how many
voters he screens.
Upon learning that Zogby told National Journal he makes 3,000 calls
per night, the Mystery Pollster sent a second message:
... the answer is incomplete. Given the number of calls, he seems
to be using either RDD or a telephone directory sample to start
with a "sample frame" of all adults in Iowa. So far so good.
Unfortunately, he's still not saying whether that's 3000 calls (or
what we call attempts) or whether he contacted 3000 otherwise
cooperative respondents who he screened down to 165 or so. If it's
the latter, the numbers are right.
If it's the former, his answer is a clever dodge. It is not at all
uncommon for a pollster doing a one night RDD sample to have to
make several thousand attempts to complete a few hundred
interviews. It's very plausible that 3000 RDD calls made on a
single night would result in....
1000 no answers
200 busy signals
600 non-working numbers
300 answering machines
400 refusals to be interviewed, and...
500 cooperative respondents, of whom he takes 165, which would be
33% of
Iowa and waaaay too big...
<endquote>