<story><title>Faster-than-light neutrino result may have been due to bad connection</title><url>http://news.sciencemag.org/scienceinsider/2012/02/breaking-news-error-undoes-faster.html</url></story><parent_chain><item><author>pessimist</author><text>The real issue is that experiments that give &quot;expected&quot; results are not subject to this kind of scrutiny. Thus experiments are much less trustworthy than one would assume. It sometimes takes decades for errors in experiments to come out - eg. from &quot;Surely your joking, Mr. Feynman&quot;:&lt;p&gt;Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It&apos;s a little bit off because he had the incorrect value for the viscosity of air. It&apos;s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan&apos;s, and the next one&apos;s a little bit bigger than that, and the next one&apos;s a little bit bigger than that, until finally they settle down to a number which is higher.&lt;p&gt;Why didn&apos;t they discover the new number was higher right away? It&apos;s a thing that scientists are ashamed of - this history - because it&apos;s apparent that people did things like this: When they got a number that was too high above Millikan&apos;s, they thought something must be wrong - and they would look for and find a reason why something might be wrong. When they got a number close to Millikan&apos;s value they didn&apos;t look so hard. And so they eliminated the numbers that were too far off, and did other things like that...</text></item></parent_chain><comment><author>splat</author><text>Actually, most major experiments in particle physics these days (including the OPERA experiment) avoid this sort of confirmation bias by being run &quot;blind.&quot;  The scientists write all of their data reduction pipelines before taking any actual data and test their pipelines on simulated data.  When they are confident that their pipeline is running as expected they run the experiment, put the data through their pipeline and publish the result, no matter how unexpected it is.&lt;p&gt;As the OPERA result showed, it has the problem that if you don&apos;t understand everything in your experiment perfectly (which is difficult to do in a very large, complicated experiment) you run the risk of embarrassing yourself by making some obvious-in-retrospect mistake and publishing an obviously absurd result.  But in the long run it&apos;s not so bad a price to pay to avoid the sort of confirmation bias that Feynman was talking about.</text></comment>
<story><title>Faster-than-light neutrino result may have been due to bad connection</title><url>http://news.sciencemag.org/scienceinsider/2012/02/breaking-news-error-undoes-faster.html</url></story><parent_chain><item><author>pessimist</author><text>The real issue is that experiments that give &quot;expected&quot; results are not subject to this kind of scrutiny. Thus experiments are much less trustworthy than one would assume. It sometimes takes decades for errors in experiments to come out - eg. from &quot;Surely your joking, Mr. Feynman&quot;:&lt;p&gt;Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It&apos;s a little bit off because he had the incorrect value for the viscosity of air. It&apos;s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan&apos;s, and the next one&apos;s a little bit bigger than that, and the next one&apos;s a little bit bigger than that, until finally they settle down to a number which is higher.&lt;p&gt;Why didn&apos;t they discover the new number was higher right away? It&apos;s a thing that scientists are ashamed of - this history - because it&apos;s apparent that people did things like this: When they got a number that was too high above Millikan&apos;s, they thought something must be wrong - and they would look for and find a reason why something might be wrong. When they got a number close to Millikan&apos;s value they didn&apos;t look so hard. And so they eliminated the numbers that were too far off, and did other things like that...</text></item></parent_chain><comment><author>siglesias</author><text>What you describe is more or less what Lakatos coined the &quot;Research Programme&quot; after. [1]&lt;p&gt;In a nutshell, we know that science more or less progresses from one theory to another after the existing theory has been falsified (Popper), &lt;i&gt;but&lt;/i&gt; this transition does not happen overnight, even after a falsifying result has been put forth. That is because science is really structured with a &quot;hard core&quot; hypothesis (say, in physics, that nothing moves faster than c), and various &quot;auxiliary hypotheses&quot; that &lt;i&gt;bear the brunt of criticism&lt;/i&gt; before the core belief is attacked (say those that posit the integrity of the measurement instrument, for example). And that is all that is going on here. There are many, many theories that are going to be dismantled before physics transitions to something post-relativity. It&apos;s just the way the assumptions are structured.&lt;p&gt;While this may seem trivial, we see this playing out in the battle of classical against behavioral economics. Are the results of experimental psychology admissible in the criticism of certain economic models? Models don&apos;t claim to be perfect, after all...&lt;p&gt;1. &lt;a href=&quot;http://en.wikipedia.org/wiki/Imre_Lakatos#Research_programmes&quot; rel=&quot;nofollow&quot;&gt;http://en.wikipedia.org/wiki/Imre_Lakatos#Research_programme...&lt;/a&gt;</text></comment>