How does peer review detect cheating when replicating a study isn't an option? Disclaimer: This is the third incarnation of my question because the previous two have met with silence.
Adapted to this site:
For a scientific study to be accepted as &quot;truth&quot; the standard process is peer review. When the subject is something like math where you can follow through all the steps and recreate the logic in your head, that works great.
But what about cases when that's not an option - for example, research into the efficiency of a vaccine or a new drug? After Phase 3 trials the pharmaceuticals company releases the data which is then scrutinized by (I'd expect) both peer researchers from rival companies and government agencies. But how can they detect that it's not all or in part a lie? There is after all a huge financial motivation for the company to do it. Even when the reviewers are highly motivated to find any traces of fraud, what is the actual process by which they do it, if they can't replicate the study itself and compare the results?
I take issue with your first statement, that for a &quot;...study to be accepted as &quot;truth&quot; the standard process is peer review&quot;. The purpose of peer review is not to be the judge of what is true or not, but to evaluate (simply put) whether a study is well-conducted, is using adequate methods, is acknowledging relevant previous research, and if the conclusions are supported by the data and analysis. Another way of putting this might be that peer review is about validation of the claims made, but not about validation of truth. Some other thoughts and guidelines on the purpose of peer-review can be found here; on scope and responsibilities, from PNAS (see &quot;Peer Reviewer Instructions&quot; and &quot;Reviewer responsibilities), and a relativly clear statement on the scope of peer-review, from The Royal Society (see &quot;Reviewing instructions&quot;).
In a scientific context, &quot;truth&quot; is something that follows from repeated studies that confirms previous results, and is based on a network of theory and observation that is (to a large extent) congruent. However, generally, I would say that it is more appropriate to define the scientific method as a way to search for &quot;truth&quot;, than a method to determine what is &quot;true&quot; (opinions will probably vary on this though).
When it comes to cheating, especially with regard to data, the possibilities to detect this during peer-review is limited, even in the ideal situation when data has been made fully available. If researchers for example fully fabricate data, or tamper with raw data, this will not be caught in peer-review, since only the modified data will be avaliable. Add to this that the time available to peer-review is very limited, so a full statistical re-analysis of the data is not possible. This is also one of the reasons for the need of replicated studies and other studies with supporting evidence before results are accepted as &quot;true&quot;.