<story><title>Why the world of scientific research needs to be disrupted</title><url>http://gigaom.com/2011/10/31/why-the-world-of-scientific-research-needs-to-be-disrupted/?</url></story><parent_chain><item><author>mironathetin</author><text>Two comments:&lt;p&gt;1.
Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don&apos;t signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don&apos;t need more quickly published junk that stirs the world but is found to be inaccurate later.&lt;p&gt;Can we accelerate the process of peer reviewing? I doubt that. As I have written, the reviewers are other scientists who have to find the time to read and comment new articles. The more time they spend with this, the less time they spend doing research. So the small amount of experts usually is responsible for being slow.&lt;p&gt;You may be quicker by internet publishing, if (IF!) you work on a highly dynamic field where other scientists are hungry for your results. As the author writes, this is the exception. Most scientific results will be useful only a long time after the publication, if they will be useful at all. In this way, science is much faster than application already.&lt;p&gt;2. The author writes, we could make progress faster. Please: why? It seems to be a sign of the time that we want to accelerate everything. It is most likely that this will only produce more noise. We need to slow down our lives and our thinking to stay accurate and produce real value. I am glad that science results, at least in my field, are still reliably scrutinized by many. This way I know that reading and spending the time to understand the stuff is worth my time.</text></item></parent_chain><comment><author>gwern</author><text>Why do we need the process of peer review?&lt;p&gt;Peer review is not robust against even low levels of collusion (&lt;a href=&quot;http://arxiv.org/abs/1008.4324v1&quot; rel=&quot;nofollow&quot;&gt;http://arxiv.org/abs/1008.4324v1&lt;/a&gt;). Scientists who win the Nobel Prize find their &lt;i&gt;other&lt;/i&gt; work suddenly being heavily cited (&lt;a href=&quot;http://www.nature.com/news/2011/110506/full/news.2011.270.html&quot; rel=&quot;nofollow&quot;&gt;http://www.nature.com/news/2011/110506/full/news.2011.270.ht...&lt;/a&gt;), suggesting either that the community either badly failed in recognizing the work&apos;s true value or that they are now sucking up &amp;#38; attempting to look better by the halo effect. (A mathematician once told me that often, to boost a paper&apos;s acceptance chance, they would add citations to papers by the journal&apos;s editors - a practice that will surprise none familiar with Goodhart&apos;s law and the use of citations in tenure &amp;#38; grants.)&lt;p&gt;Physicist Michael Nielsen points out (&lt;a href=&quot;http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review/&quot; rel=&quot;nofollow&quot;&gt;http://michaelnielsen.org/blog/three-myths-about-scientific-...&lt;/a&gt;) that peer review is historically rare (just one of Einstein&apos;s 300 papers was peer reviewed! the famous _Nature_ did not institute peer review until 1967), has been poorly studied (&lt;a href=&quot;http://jama.ama-assn.org/cgi/content/abstract/287/21/2784&quot; rel=&quot;nofollow&quot;&gt;http://jama.ama-assn.org/cgi/content/abstract/287/21/2784&lt;/a&gt;) &amp;#38; not shown to be effective, is nationally biased (&lt;a href=&quot;http://jama.ama-assn.org/cgi/content/full/295/14/1675&quot; rel=&quot;nofollow&quot;&gt;http://jama.ama-assn.org/cgi/content/full/295/14/1675&lt;/a&gt;), erroneously rejects many historic discoveries (one study lists &quot;34 Nobel Laureates whose awarded work was rejected by peer review&quot; (&lt;a href=&quot;http://www.canonicalscience.org/publications/canonicalsciencereports/20082.html&quot; rel=&quot;nofollow&quot;&gt;http://www.canonicalscience.org/publications/canonicalscienc...&lt;/a&gt;); Horribin 1990 (&lt;a href=&quot;http://jama.ama-assn.org/content/263/10/1438.abstract&quot; rel=&quot;nofollow&quot;&gt;http://jama.ama-assn.org/content/263/10/1438.abstract&lt;/a&gt;) lists others like the discovery of quarks), and catches only a small fraction (&lt;a href=&quot;http://jama.ama-assn.org/cgi/content/abstract/280/3/237&quot; rel=&quot;nofollow&quot;&gt;http://jama.ama-assn.org/cgi/content/abstract/280/3/237&lt;/a&gt;) of errors. And fraud, like the one we just saw in psychology? Forget about it (&lt;a href=&quot;http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0005738&quot; rel=&quot;nofollow&quot;&gt;http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjourna...&lt;/a&gt;);&lt;p&gt;&amp;#62; &quot;A pooled weighted average of 1.97% (N = 7, 95%CI: 0.86–4.45) of scientists admitted to have fabricated, falsified or modified data or results at least once –a serious form of misconduct by any standard– and up to 33.7% admitted other questionable research practices. In surveys asking about the behaviour of colleagues, admission rates were 14.12% (N = 12, 95% CI: 9.91–19.72) for falsification, and up to 72% for other questionable research practices....When these factors were controlled for, misconduct was reported more frequently by medical/pharmacological researchers than others.&quot;&lt;p&gt;No, peer review is not the secret sauce of science. Replication is more like it.</text></comment>
<story><title>Why the world of scientific research needs to be disrupted</title><url>http://gigaom.com/2011/10/31/why-the-world-of-scientific-research-needs-to-be-disrupted/?</url></story><parent_chain><item><author>mironathetin</author><text>Two comments:&lt;p&gt;1.
Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don&apos;t signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don&apos;t need more quickly published junk that stirs the world but is found to be inaccurate later.&lt;p&gt;Can we accelerate the process of peer reviewing? I doubt that. As I have written, the reviewers are other scientists who have to find the time to read and comment new articles. The more time they spend with this, the less time they spend doing research. So the small amount of experts usually is responsible for being slow.&lt;p&gt;You may be quicker by internet publishing, if (IF!) you work on a highly dynamic field where other scientists are hungry for your results. As the author writes, this is the exception. Most scientific results will be useful only a long time after the publication, if they will be useful at all. In this way, science is much faster than application already.&lt;p&gt;2. The author writes, we could make progress faster. Please: why? It seems to be a sign of the time that we want to accelerate everything. It is most likely that this will only produce more noise. We need to slow down our lives and our thinking to stay accurate and produce real value. I am glad that science results, at least in my field, are still reliably scrutinized by many. This way I know that reading and spending the time to understand the stuff is worth my time.</text></item></parent_chain><comment><author>beambot</author><text>To add to that, 3. Scientists (particularly academics) should focus on a smaller number of solid, flushed-out papers rather than seeking out the minimum publishable unit.  The latter seems to be a byproduct of the tenure process (IMO) and just adds noise.</text></comment>