<story><title>A bot crawled thousands of studies looking for simple math errors</title><url>http://www.vox.com/science-and-health/2016/9/30/13077658/statcheck-psychology-replication</url></story><parent_chain><item><author>c3534l</author><text>&amp;gt; and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve.&lt;p&gt;&amp;gt; Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.”&lt;p&gt;&amp;gt; The draft of the article said these online critics were engaging in “methodological terrorism.”&lt;p&gt;If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. There is a fundamental misunderstanding of how knowledge is created through the scientific process if the verification step is considered to be offensive or taboo. That anyone in the field of psychology would even be comfortable publically espousing a non-scientific worldview like that means that psychologists are not being properly educated in the scientific method and should not be in the business of producing research since they do not have a mature understanding of what &amp;quot;scientific&amp;quot; implies.</text></item></parent_chain><comment><author>dmd</author><text>I took a class with Susan Fiske in grad school, and the following is a quote from one of her comments on a homework where I had called into question a paper that was clearly not just statistically incorrectly done but was based on urban-legend-level assumptions:&lt;p&gt;&amp;quot;These results are from respected investigators, so it&amp;#x27;s inappropriate to question them.&amp;quot;</text></comment>
<story><title>A bot crawled thousands of studies looking for simple math errors</title><url>http://www.vox.com/science-and-health/2016/9/30/13077658/statcheck-psychology-replication</url></story><parent_chain><item><author>c3534l</author><text>&amp;gt; and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve.&lt;p&gt;&amp;gt; Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.”&lt;p&gt;&amp;gt; The draft of the article said these online critics were engaging in “methodological terrorism.”&lt;p&gt;If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. There is a fundamental misunderstanding of how knowledge is created through the scientific process if the verification step is considered to be offensive or taboo. That anyone in the field of psychology would even be comfortable publically espousing a non-scientific worldview like that means that psychologists are not being properly educated in the scientific method and should not be in the business of producing research since they do not have a mature understanding of what &amp;quot;scientific&amp;quot; implies.</text></item></parent_chain><comment><author>SilasX</author><text>Yes! Similarly, I remember a few years back being shocked at the attitude toward the replication crisis, where authors of irreproducible work were miffed that no one talked to them first.&lt;p&gt;I was like, &amp;quot;Um, the whole point of science is that it&amp;#x27;s knowledge &lt;i&gt;in the public record&lt;/i&gt;.  If you need to &amp;#x27;chill&amp;#x27; with the original researcher to get critical details, that&amp;#x27;s not science anymore.&amp;quot;&lt;p&gt;Remember, the studies might form the basis of an FDA approval.  If &lt;i&gt;only&lt;/i&gt; the original lab can produce the result, then is the FDA approval of the treatment going to be limited to &lt;i&gt;only&lt;/i&gt; that lab as well?  Kinda limits scalability...&lt;p&gt;I did a search and found the blog post that quoted the researcher [1] with this scary attitude:&lt;p&gt;&amp;gt;When researchers at Amgen, a pharmaceutical company in Thousand Oaks, California, failed to replicate many important studies in preclinical cancer research, they tried to contact the authors and exchange materials. They could confirm only 11% of the papers. I think that if more biotech companies had the patience to send someone to the original labs, perhaps the percentage of reproducibility would be much higher.&lt;p&gt;[1] &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;andrewgelman.com&amp;#x2F;2013&amp;#x2F;12&amp;#x2F;17&amp;#x2F;replication-backlash&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;andrewgelman.com&amp;#x2F;2013&amp;#x2F;12&amp;#x2F;17&amp;#x2F;replication-backlash&amp;#x2F;&lt;/a&gt; (thanks Google)</text></comment>