How do academics control their own biases? As an academic, you are in the business of producing research. I know it might sound odd to call it a &quot;business&quot;, but frankly, you get paid to do it, and most of you wouldn't do it if you didn't get paid, hence it is a business and you're selling a product (your research) to your clients (the rest of society) and you get paid a price for that product (grants and aid and donations and other types of support from governments and other institutions).
This means that there is a danger that your research output is driven by a self-serving bias. After all, the 'better' research you produce (as in, research that draws a lot of attention), the more of it you will sell and the more money you will make.
How do academics controls themselves for this bias, that may be occurring consciously or subconsciously?
One might say that peer-reviewing solves this exact problem, but I fear that all peer-reviewing accomplishes is moving the goalpost. Okay, so now the bias of the lone researcher is being controlled, but who controls the biases of the peers? After all, this bias is an industry-wide phenomenon. Your peers, the ones that review your research, have the exact same bias as you do, and are also interested in seeing your research be validated if it brings more attention to your field, a field which is obviously shared between you and your peers.
Hence the bias affects you all at the same time. Who is controlling that? Is there a 3rd party? If not, how do academics, as a group, control themselves?
Obviously, this question is more pertinent for less fact-based fields, so not for example mathematics, where if a wrong theorem is approved, it would be catastrophic and probably discovered soon enough, to the shame of all people involved. But for more experimental fields such as physics, medicine, etc., how do academics ensure that there isn't a cartel of researchers some where all (sub)consciously agreeing to promote research that advances their careers?
Everything is in the end vetted by the community and by time.
Many researchers were fooled by their biases (including Galilei or Millikan who did systematic errors in their studies) and the biases were slowly uncovered by time. This is normal. Of course, a good researcher, as Feynman says, needs to take care not to fool themselves, and good researchers work hard to do so. In the end, however, it is time and the scientific community that filters out the wheat from the chaff.
One key thing to maximize your objectivity is to avoid p-hacking. Be aware what your hypothesis is before you run the experiment. Be aware of what parameters you extract from the experiment and how to factor them into your evaluation of the theoretical vs. experimental model.
Try to have at least two maximally independent ways of getting to a critical result. Understand the assumptions that got you a result.
Taking care of these various things will minimize the probability (it will never entirely go away) that you fool yourself and increase your confidence that you have done a proper job at doing good science.