<story><title>The incentives of academia naturally select for less reliable results (2016)</title><url>https://www.theatlantic.com/science/archive/2016/09/the-inevitable-evolution-of-bad-science/500609/?single_page=true</url></story><parent_chain><item><author>ethanwillis</author><text>This is exactly why I am aiming to bootstrap my own lab outside of academia and traditional research positions available private industry.&lt;p&gt;There&amp;#x27;s a lot of &amp;quot;BS&amp;quot; in these power structures that keep people from being able to dedicate resources and a proper amount of time to generate quality research.&lt;p&gt;I&amp;#x27;m fairly &amp;quot;green&amp;quot; in my research career, but I&amp;#x27;ve been in the game for a few years now in the Biology&amp;#x2F;Bioinformatics realm. I&amp;#x27;ve seen and heard stories from plenty of colleagues about how when they first got started and well into their first P.I. positions (and even until this day when they&amp;#x27;re 10-20 years into their work) where they get abused by their institutions. Aside from &amp;quot;publish or perish&amp;quot; there&amp;#x27;s also your institution skimming 50-60% of your hard won grant funding to let you &amp;quot;rent&amp;quot; their facilities where you&amp;#x27;re fighting over shelf space with other P.I.s Absolutely insane.&lt;p&gt;Seeing this I decided I can bootstrap my own lab for much less and have true autonomy. For example a rough breakdown of my costs for setting up a bio&amp;#x2F;bioinformatics lab for myself and a few friends:&lt;p&gt;Land Zoned for Light Industrial and Scientific Research?
about $1000&amp;#x2F;acre or less in my area. (hint: buy tax liened property)&lt;p&gt;CO2 incubators? Don&amp;#x27;t pay $4k-8k new. I can buy refurbed or liquidated lab equipment from the unsustainable institution labs for literally pennies on the dollar. Try $100-200 for this equipment.&lt;p&gt;High RPM centrifuges? Once again instead of $2k-5k I can spend $300 for a 15k RPM centrifuge. and something like $20 for desktop centrifuges.&lt;p&gt;PCR machines? Try &amp;lt; $100 each. And I&amp;#x27;ve even gotten &amp;quot;extras&amp;quot; with them in that price range.&lt;p&gt;Compute power&amp;#x2F;I.T.? I can get 256 core, 1tb ram blade centers for about $1k. Modern Core i5&amp;#x2F;Core i7 desktops for $80 each.&lt;p&gt;By my estimates I can buy the land, build a building(admittedly a very utilitarian one), and equip a new lab for sub $30k. Now my only costs are re-agents(if I&amp;#x27;m doing research that requires it), power, and cheap land taxes.</text></item><item><author>madhadron</author><text>Your solution is historically inaccurate. Statistical tests of significance were created as a tool to improve the objectivity of &lt;i&gt;evaluating&lt;/i&gt; results. Garbage in, garbage out still applies. Nor is it relevant. Statistical techniques are a symptom, not an underlying cause.&lt;p&gt;Indeed, everything in the article or in discussions of this that I see is always treated symptoms far from the causes. Let&amp;#x27;s do a quick five whys:&lt;p&gt;Why are publications not reproducible? Because positive results are being published as fast as possible with little checking on them.&lt;p&gt;Why are they publishing so? Because the people involved need publications to get out of their current transient position and secure another transient position, with hopes of someday being able to settle down somewhere.&lt;p&gt;Why do they need to secure their next transient position? Because they have been inculcated that this is the career to be in by their milieu and that is the only way to stay in it.&lt;p&gt;Why is that the only way to stay in it? Because there are no permanent positions that aren&amp;#x27;t at the very end of this gauntlet.&lt;p&gt;Why are there no permanent positions except at the end of the gauntlet? Because this is a pyramid scheme built on an apprenticeship system.&lt;p&gt;Don&amp;#x27;t get distracted by all the other stuff. It&amp;#x27;s the fundamental structure of academic science that leads to the rest of this.</text></item><item><author>nonbel</author><text>The proposed &amp;quot;solutions&amp;quot; of pre-registration and increase sample size are not going to work because they are not fixing the underlying problem of NHST use (where a researcher tests a nil null hypothesis of zero effect and then concludes their favorite explanation is correct if they reject this null hypothesis).&lt;p&gt;The solution is to go back to the old way of science (pre-1940):&lt;p&gt;1) Collect data and figure out what is repeatable and consistent (eg come up with &amp;quot;natural laws&amp;quot;).&lt;p&gt;2) Come up with theories that explain the observations from #1 that also make &lt;i&gt;precise&lt;/i&gt; predictions about some different type of data.&lt;p&gt;Reward researchers who figure out methods that lead to reliable, repeatable observations and those that make precise predictions (if your theory predicts nothing more than a positive&amp;#x2F;negative correlation between two variables it is too vague).</text></item></parent_chain><comment><author>analog31</author><text>You&amp;#x27;ll find that any large state university has dozens if not hundreds of little businesses that sprout up in exactly this way. You see them in the little business parks -- most of them lease rather than buy. You see them at the trade shows.&lt;p&gt;It goes without saying that the idea is just like any other business venture, with ways to succeed and to fail.&lt;p&gt;But a useful thing is to try and get a survey of what kinds of businesses are out there, what they are doing, and how well they are doing. Next time you&amp;#x27;re at a conference that has a trade show, go and look at the booths in the &amp;quot;low rent district,&amp;quot; because it will give you an idea of what people are doing.&lt;p&gt;Oddly enough I work for a F500, and we buy second hand gear. One reason is that if somebody is advertising a particular instrument, it means that they actually have it, and you can get it right away. With new gear, sometimes it can take weeks or even months. Also, if it&amp;#x27;s under a couple grand, I can buy it on my purchasing card without management approval.</text></comment>
<story><title>The incentives of academia naturally select for less reliable results (2016)</title><url>https://www.theatlantic.com/science/archive/2016/09/the-inevitable-evolution-of-bad-science/500609/?single_page=true</url></story><parent_chain><item><author>ethanwillis</author><text>This is exactly why I am aiming to bootstrap my own lab outside of academia and traditional research positions available private industry.&lt;p&gt;There&amp;#x27;s a lot of &amp;quot;BS&amp;quot; in these power structures that keep people from being able to dedicate resources and a proper amount of time to generate quality research.&lt;p&gt;I&amp;#x27;m fairly &amp;quot;green&amp;quot; in my research career, but I&amp;#x27;ve been in the game for a few years now in the Biology&amp;#x2F;Bioinformatics realm. I&amp;#x27;ve seen and heard stories from plenty of colleagues about how when they first got started and well into their first P.I. positions (and even until this day when they&amp;#x27;re 10-20 years into their work) where they get abused by their institutions. Aside from &amp;quot;publish or perish&amp;quot; there&amp;#x27;s also your institution skimming 50-60% of your hard won grant funding to let you &amp;quot;rent&amp;quot; their facilities where you&amp;#x27;re fighting over shelf space with other P.I.s Absolutely insane.&lt;p&gt;Seeing this I decided I can bootstrap my own lab for much less and have true autonomy. For example a rough breakdown of my costs for setting up a bio&amp;#x2F;bioinformatics lab for myself and a few friends:&lt;p&gt;Land Zoned for Light Industrial and Scientific Research?
about $1000&amp;#x2F;acre or less in my area. (hint: buy tax liened property)&lt;p&gt;CO2 incubators? Don&amp;#x27;t pay $4k-8k new. I can buy refurbed or liquidated lab equipment from the unsustainable institution labs for literally pennies on the dollar. Try $100-200 for this equipment.&lt;p&gt;High RPM centrifuges? Once again instead of $2k-5k I can spend $300 for a 15k RPM centrifuge. and something like $20 for desktop centrifuges.&lt;p&gt;PCR machines? Try &amp;lt; $100 each. And I&amp;#x27;ve even gotten &amp;quot;extras&amp;quot; with them in that price range.&lt;p&gt;Compute power&amp;#x2F;I.T.? I can get 256 core, 1tb ram blade centers for about $1k. Modern Core i5&amp;#x2F;Core i7 desktops for $80 each.&lt;p&gt;By my estimates I can buy the land, build a building(admittedly a very utilitarian one), and equip a new lab for sub $30k. Now my only costs are re-agents(if I&amp;#x27;m doing research that requires it), power, and cheap land taxes.</text></item><item><author>madhadron</author><text>Your solution is historically inaccurate. Statistical tests of significance were created as a tool to improve the objectivity of &lt;i&gt;evaluating&lt;/i&gt; results. Garbage in, garbage out still applies. Nor is it relevant. Statistical techniques are a symptom, not an underlying cause.&lt;p&gt;Indeed, everything in the article or in discussions of this that I see is always treated symptoms far from the causes. Let&amp;#x27;s do a quick five whys:&lt;p&gt;Why are publications not reproducible? Because positive results are being published as fast as possible with little checking on them.&lt;p&gt;Why are they publishing so? Because the people involved need publications to get out of their current transient position and secure another transient position, with hopes of someday being able to settle down somewhere.&lt;p&gt;Why do they need to secure their next transient position? Because they have been inculcated that this is the career to be in by their milieu and that is the only way to stay in it.&lt;p&gt;Why is that the only way to stay in it? Because there are no permanent positions that aren&amp;#x27;t at the very end of this gauntlet.&lt;p&gt;Why are there no permanent positions except at the end of the gauntlet? Because this is a pyramid scheme built on an apprenticeship system.&lt;p&gt;Don&amp;#x27;t get distracted by all the other stuff. It&amp;#x27;s the fundamental structure of academic science that leads to the rest of this.</text></item><item><author>nonbel</author><text>The proposed &amp;quot;solutions&amp;quot; of pre-registration and increase sample size are not going to work because they are not fixing the underlying problem of NHST use (where a researcher tests a nil null hypothesis of zero effect and then concludes their favorite explanation is correct if they reject this null hypothesis).&lt;p&gt;The solution is to go back to the old way of science (pre-1940):&lt;p&gt;1) Collect data and figure out what is repeatable and consistent (eg come up with &amp;quot;natural laws&amp;quot;).&lt;p&gt;2) Come up with theories that explain the observations from #1 that also make &lt;i&gt;precise&lt;/i&gt; predictions about some different type of data.&lt;p&gt;Reward researchers who figure out methods that lead to reliable, repeatable observations and those that make precise predictions (if your theory predicts nothing more than a positive&amp;#x2F;negative correlation between two variables it is too vague).</text></item></parent_chain><comment><author>mattb314</author><text>Could you link&amp;#x2F;explain how you get those hardware prices, specifically the servers and desktops? Are the available for the average consumer (ie not making a bulk purchase)? I have very little experience buying hardware, but those prices sound 4-8x lower than what I&amp;#x27;d expect (and I&amp;#x27;d be super happy to find that I&amp;#x27;m wrong).</text></comment>