The lead of their first response:
<http://credo.stanford.edu/reports/CREDO_Hoxby_Rebuttal.pdf>
> The memo, “A Serious Statistical Mistake in the CREDO Study of
> Charter Schools,” by Caroline
> Hoxby, does not provide any basis whatsoever for discounting the
> reliability of the CREDO study’s
> conclusions. The central element of Dr. Hoxby’s critique is a
> statistical argument that is quite unrelated
> to the CREDO analysis. The numerical elements of it are misleading
> in the extreme, even had the
> supporting logic been correct. Unfortunately, the memo is riddled
> with serious errors both in the
> structure of the underlying statistical models and in the derivation
> of any bias.
>
> The technical aspects of these issues are described below, but the
> overall description of the
> problem is straightforward. She considers estimation of models that
> are very different from those
> estimated by CREDO and derives conclusions that are completely
> irrelevant to the CREDO results.
Hoxby is an enthusiastic partisan of charter schools. But her NBER paper I posted earlier claimed results of "0.09 standard deviations per year of treatment in math and 0.04 standard deviations per year in reading." I.e., rather tiny.
<http://www.nber.org/papers/w14852>
On Nov 10, 2009, at 2:01 PM, Jordan Hayes wrote:
> So long as we're slinging links:
>
> ---
>
> http://www.nber.org/~schools/charterschoolseval/memo_on_the_credo_study.pdf
>
> "A Statistical Mistake in the CREDO Study of Charter Schools"
>
> Caroline M. Hoxby
> Stanford University and NBER
> August 2009*
>
> Abstract
>
> A recent study of charter schools' effect on student achievement
> has been published by CREDO (2009). It contains a statistical
> mistake that causes a biased estimate of how charter schools
> affect achievement. This paper explains that mistake. Essentially,
> the achievement of charter school students is measured with much
> more error than the achievement of the controls, which are not
> individual students but are group averages of students in the
> traditional public schools. By using the achievement data as
> both the dependent variable and (lagged) an independent variable,
> the CREDO study forces the estimated effect of charter schools
> to be biased, and the bias is negative unless the CREDO data
> are very atypical of administrative achievement data. This paper
> also notes that the CREDO study violates four rules for the
> empirically sound use of matching methods to evaluate charter
> schools' effects.
> ___________________________________
> http://mailman.lbo-talk.org/mailman/listinfo/lbo-talk