[WS:] On the second thought, this "so-so science" is perhaps more insidious than outright cooking the numbers. When exposed, cooking the numbers is clearly recognized as fraud, whereas the so-so science can pass for truth behind flimsy covers of procedural correctness.
Wojtek
On Fri, Mar 19, 2010 at 7:11 AM, Wojtek S <wsoko52 at gmail.com> wrote:
> Miles: Again, the experimental design does control for this possible
> effect. The results provide clear evidence of causality.
>
> [WS:] It does not take much to convince you, apparently. As for me, the
> experiment suggests a connection but not causality, because it does not rule
> out spuriousness. To prove a connection, I would like to see more rigorous
> actual controls and fewer heroic assumptions about those controls.
>
> But otoh, I understand the effects of publish or perish imperative on the
> quality of published papers. If you do not publish in sufficient
> quantities, your academic career (or funding) is in danger, and to publish
> you need to "find" something as opposed to "no relationship found"
> conclusions. So to increase the probability of "finding something" (and
> devising a fancy name for it so it is quotable later), flimsy standards or
> proof are adopted, like relaxing controls, increasing the margins of error
> (p < 0.1 is often quoted) etc. That is still better than straightforward
> cooking the numbers (as some occasionally do) - but it marks the trend of
> social science veering into the realms of social commentary and opinion
> journalism.
>
> Wojtek
>
>
>
>
>
> On Thu, Mar 18, 2010 at 2:44 PM, Miles Jackson <cqmv at pdx.edu> wrote:
>
>> Wojtek S wrote:
>>
>>> the paper has a
>>> more fundamental in my opinion error, especially in Experiment 3. It
>>> fails
>>> to separate the effect of experimental stimulus (shopping at conventional
>>> or
>>> green store) from other potential effects stemming from the general
>>> propensity of the experimental subjects. That is, the authors should
>>> have
>>> first done pre-testing i.e. measure all subjects on the lying and
>>> stealing
>>> test, then divide them into 3 groups - one control (no shopping) and two
>>> experimental (green and conventional shopping) and then do post-testing
>>> i.e.
>>> measure all subjects on the lying and stealing test again and look for
>>> differences in changes between pre- and post-testing for all three
>>> groups.
>>>
>>>
>> No, that's unnecessary. Participants were randomly assigned to the
>> experimental levels, so the "individual propensities" of the participants
>> are equally distributed among the groups. Eliminating the types of
>> confounds you're concerned about here is the reason that psychologists
>> prefer to use experimental rather than correlational designs whenever
>> possible.
>>
>>
>> The failure to do so is a fatal flaw in experimental design, because it
>>> makes it impossible to isolate the effect under investigation (shopping
>>> choice) form other possible effects. If I were asked to review this
>>> paper
>>> for potential publication, I would give it a negative review because the
>>> paper does not produce sufficient evidence to support its conclusions.
>>> As I
>>> said before, it could be that the people who shopped green and cheated
>>> did
>>> so because they were arrogant upper class pricks (or economists :)) and
>>> both
>>> behaviors caused by their uppity arrogance (green shopping -
>>> manifestation
>>> of social status, cheating - feeling of entitlement and greed.) The
>>> paper
>>> does not control for that possible effect.
>>>
>>
>> Again, the experimental design does control for this possible effect.
>> The results provide clear evidence of causality.
>>
>> Miles
>>
>> ___________________________________
>> http://mailman.lbo-talk.org/mailman/listinfo/lbo-talk
>>
>
>