On Tue, 9 Nov 2004, Wojtek Sokolowski wrote:
> The point I am trying to make here is that when we have a situation that a
> model explains a relatively small percent of variance (say, less than 20%)
> that indicates that the variables that can potentially explain most (80%) of
> the variance are not in the equation. This, in turn creates a strong
> possibility of spuriousness of the observed effects. That is to say, if we
> introduce those other variables that may potentially explain the hitherto
> unexplained 80% of the variance, the chances are the effects of the
> variables already in the equation may be "explained away" i.e. become
> statistically insignificant or even reverse its sign (i.e. from positive to
> negative).
Hmm. I realize only a few LBO wonks may be interested in this point, but Woj is confusing error variance and confound variance here. Whether the unexplained variance is 80% or 5%, the relationship between IQ and academic outcomes could be accounted for by an uncontrolled confound. Confounds--spurious relationships--contribute to the explained variance in r-squared, not the unexplained variance. Thus the amount of unexplained (error) variance is irrelevant to the question of whether or not spurious (confound) variables account for the relationship. --Typically, the "explained variance"-- r-squared--in this type of IQ research is about 10-20%. Note that this does not mean there is a strong likelihood that the IQ- academic achievement relationship is spurious; it just means that many factors contribute to variation in academic achievement (a factor can contribute to the unexplained variance here and not be a confound!).
>
> So unless someone shows me that IQ test alone explains more than, say, 60
> percent of the variance on academic achievement, or that IQ tests together
> with other variables explain that much variance and IQ still remains
> significant - I remain unimpressed by all those correlations claims.
This is an unrealistic criterion for psychological research. Human beings are complex, and human behavior is multiply determined. You will almost never find a single psychological variable that accounts for 60% of the variation in behavior (that's r = about .80!).
--Moreover, in the context of psychological research, this is a misrepresentation of the importance of "percentage of variation accounted for". Keep in mind that accounting for 15% of the variance is a large effect (e.g., the effect of smoking on risk of cancer); accounting for 6% is a moderate effect (e.g., the effect of many drug therapies on psychological disorders); and accounting for only 1% is a meaningful and small effect (e.g., the effect of asprin on risk of coronary heart disease). If IQ scores only accounted for 1% or 2% of the total variation in academic performance, that would be practically and theoretically important. As I mentioned, IQ scores typically account for 10-20% of the variation in academic outcomes.
Again, I want to emphasize that IQ tests do not measure general intelligence. However, we need to base our criticisms on scientific data and meaningful interpretations of those data.
Miles