Junk Polling or Junk Press Release?

Jim Westrich westrich at miser.umass.edu
Thu May 3 14:41:22 PDT 2001


Some of you may have seen a press release about a FDIC/Gallup survey about deposit insurance. If you haven't I bet a lot of newspapers will run small stories tomorrow. Here is a chance to see bad journalism as it happens (Reuters has already filed a story that is just a rewrite of the FDIC press release repeating the "errors" noted below).

Anyway, in the official press release ( http://www.fdic.gov/news/news/press/2001/pr3101.html ) it says:

"Among the survey's other findings: -One in eight households keep more than $100,000 in the bank. -About one-third of all households reported having more than $100,000 in the bank at one time or another"

This is not even close to being true.

Luckily, there is a link to the technical aspects of the study and you can find out how they selected the sample. The long and short of this is that they selected a target income sample. There other aspects to the sample design that indicate to me that an even stronger income bias exists than there target $65,000 household income. (One particular pet peeve about "representative samples" formed by random digit dialing is that 6% do not have phones and multi-phone line homes are correlated with income).

They weighted the sample on income but only below $65,000 and above $65,000.

Further, (I tend to look for these kinds of "corrections") if someone indicated more $100,000 in one question and less than $100,000 in another question this was "corrected" up to indicate more than $100,000. That is, one answer is assumed correct and the other is assumed incorrect. This is actually a fairly common practice and is complete junk as science goes (but it does produce favorable results for people who want to force results out of surveys).

I first came across this practice in a health insurance survey and got a 7-page single spaced nasty letter from the survey designer for my efforts to call them on it. "Acquiescence Bias" is apparently a technique pollsters (ask a question more than once when hoping to get an affirmative answer) don't want people to know about.

Peace,

Jim



More information about the lbo-talk mailing list