[lbo-talk] Polling Methods

[Cliff Staples]

At 02:42 PM 8/1/2003 -0400, you wrote:
>Jon Johanning wrote:
>
>>I'm not an expert in the science (?) of polling, but I think I've seen 
>>several stories recently noting that pollers are getting increasingly 
>>frustrated trying to find valid samples, given that they get a lot of 
>>answering machines when they try to call folks, or when they do reach 
>>them, folks refuse to participate, etc.
>
>Yup, the refusal rate is up, people are switching from land lines to cell 
>phones (which aren't as easily accessible), caller ID means people aren't 
>answering the phone, etc. Accuracy doesn't seem to have suffered all that 
>much, though.

I know a little about polling and telephone surveys, having conducted a few 
myself over the years, and I'm also affiliated with a research shop that 
does them by the bucket-load.  Things have gotten pretty awful over the 
last 20 years, the last 5 in particular.   For a while Random Digit Dialing 
(RDD) got past the problem of unlisted numbers (but not the problem of 
pissing off the people with the unlisted numbers that you got), but 
increasingly people are opting out of the sampling frame altogether by 
going cell phone only.  I, in fact, am one of them.  Once cable Internet 
was available to me about a 2 years ago I bailed on ATT.  So now there's no 
way I get included in anyone's telephone sample.  It's a gradual return to 
the 1940s when (for better or worse) it all got started.  You all recall 
the picture of Truman holding up the newspaper headline that says "Dewey 
Beats Truman."  That headline was based on polling that didn't include the 
democrats who didn't have phones but were able to vote.  Your sample is 
only as good as your sampling frame.

In grad school I worked for a while under Don Dillman, who wrote the bible 
on telephone and mail surveys (The Total Design Method for Mail and 
Telephone Surveys-- or TDM-- or TEDIUM, as we liked to refer to it).  He 
has a new edition of it out in which he discusses some of these issues and 
is likely to be on top of any research trying to overcome such problems (A 
Sociological Abstracts search will find the latest stuff).  Don is nothing 
if not persistent.

I did my last phone survey about 5 years ago and wouldn't trust one today-- 
at least if one is using a phone survey as a proxy for a household 
survey.  If I had to do a household survey today it would be back to 
pounding the pavement door to door with multiple return visits to get the 
"not home" folks.  If money was an issue, I'd take a smaller sample size, 
with tractable margins of error, versus the utterly unknown (and probably 
unknowable) error between the population and the sampling frame resulting 
from using telephone surveys today.

Cliff

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <../attachments/20030801/2ef56b9d/attachment.htm>