The results provide clear evidence of causality.
[WS:] It does not take much to convince you, apparently. As for me, the experiment suggests a connection but not causality, because it does not rule out spuriousness. To prove a connection, I would like to see more rigorous actual controls and fewer heroic assumptions about those controls.
But otoh, I understand the effects of publish or perish imperative on the quality of published papers. If you do not publish in sufficient quantities, your academic career (or funding) is in danger, and to publish you need to "find" something as opposed to "no relationship found" conclusions. So to increase the probability of "finding something" (and devising a fancy name for it so it is quotable later), flimsy standards or proof are adopted, like relaxing controls, increasing the margins of error (p < 0.1 is often quoted) etc. That is still better than straightforward cooking the numbers (as some occasionally do) - but it marks the trend of social science veering into the realms of social commentary and opinion journalism.
Wojtek
On Thu, Mar 18, 2010 at 2:44 PM, Miles Jackson <cqmv at pdx.edu> wrote:
> Wojtek S wrote:
>
>> the paper has a
>> more fundamental in my opinion error, especially in Experiment 3. It
>> fails
>> to separate the effect of experimental stimulus (shopping at conventional
>> or
>> green store) from other potential effects stemming from the general
>> propensity of the experimental subjects. That is, the authors should have
>> first done pre-testing i.e. measure all subjects on the lying and stealing
>> test, then divide them into 3 groups - one control (no shopping) and two
>> experimental (green and conventional shopping) and then do post-testing
>> i.e.
>> measure all subjects on the lying and stealing test again and look for
>> differences in changes between pre- and post-testing for all three groups.
>>
>>
> No, that's unnecessary. Participants were randomly assigned to the
> experimental levels, so the "individual propensities" of the participants
> are equally distributed among the groups. Eliminating the types of
> confounds you're concerned about here is the reason that psychologists
> prefer to use experimental rather than correlational designs whenever
> possible.
>
>
> The failure to do so is a fatal flaw in experimental design, because it
>> makes it impossible to isolate the effect under investigation (shopping
>> choice) form other possible effects. If I were asked to review this
>> paper
>> for potential publication, I would give it a negative review because the
>> paper does not produce sufficient evidence to support its conclusions. As
>> I
>> said before, it could be that the people who shopped green and cheated did
>> so because they were arrogant upper class pricks (or economists :)) and
>> both
>> behaviors caused by their uppity arrogance (green shopping - manifestation
>> of social status, cheating - feeling of entitlement and greed.) The paper
>> does not control for that possible effect.
>>
>
> Again, the experimental design does control for this possible effect. The
> results provide clear evidence of causality.
>
> Miles
>
> ___________________________________
> http://mailman.lbo-talk.org/mailman/listinfo/lbo-talk
>