<story><title>50% of neuroscience papers suffer from a major statistical error.</title><url>http://www.badscience.net/2011/10/what-if-academics-were-as-dumb-as-quacks-with-statistics/</url></story><parent_chain><item><author>_delirium</author><text>The short version of the error: If in your data you find that A has no statistically significant effect, but B does have a statistically significant effect, this does &lt;i&gt;not&lt;/i&gt; automatically show that B has, with statistical significance, more effect than A. To do that you have to do a statistical test on the difference in the effects.&lt;p&gt;I believe (?) this is normally done correctly in medical studies with placebos, where the typical analysis is to show that a drug has a statistically significant effect compared to the placebo baseline; it&apos;s not sufficient to show that the drug has an effect compared to a no-drug baseline, and, separately, that a placebo doesn&apos;t.</text></item></parent_chain><comment><author>JoachimSchipper</author><text>You may be simplifying too much to be understood. In medical terms, if your drug is significantly more effective than doing nothing, but a placebo is not significantly more effective than doing nothing, your drug need not be significantly more effective than the placebo. (Yes, medicine typically avoids this particular error.)&lt;p&gt;(E.g. if doing nothing cures 10%, the placebo cures 11% and the drug cures 12%, only the difference drug/nothing may be significant.)</text></comment>
<story><title>50% of neuroscience papers suffer from a major statistical error.</title><url>http://www.badscience.net/2011/10/what-if-academics-were-as-dumb-as-quacks-with-statistics/</url></story><parent_chain><item><author>_delirium</author><text>The short version of the error: If in your data you find that A has no statistically significant effect, but B does have a statistically significant effect, this does &lt;i&gt;not&lt;/i&gt; automatically show that B has, with statistical significance, more effect than A. To do that you have to do a statistical test on the difference in the effects.&lt;p&gt;I believe (?) this is normally done correctly in medical studies with placebos, where the typical analysis is to show that a drug has a statistically significant effect compared to the placebo baseline; it&apos;s not sufficient to show that the drug has an effect compared to a no-drug baseline, and, separately, that a placebo doesn&apos;t.</text></item></parent_chain><comment><author>markkat</author><text>It&apos;s true. Many anti-depressant drugs have no significant effect when compared to placebo.&lt;p&gt;&lt;a href=&quot;http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0050045&quot; rel=&quot;nofollow&quot;&gt;http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fj...&lt;/a&gt;</text></comment>