A huge, collaborative research project attempted to recreate 100 studies that were recently published in major psychology journals, and it found that only 39 of those studies' results could be replicated. The mean effect size (r) of the replication effects (M r = 0.197, SD = 0.257) was half the magnitude of the mean effect size of the original effects (M r = 0.403, SD = 0.188), representing a substantial decline. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. Thirty-six percent of replications had significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. "And there has been a growing concern that reproducibility may be lower than expected or desired." Were replication standards to be put in place, these questions might be answered and lead to a clearer view of each finding's accuracy. Replication can increase certainty when findings are reproduced and promote innovation when they are not.
– A new study on psychology research is essentially bashing all other psychology studies. The research area has gotten a bad rap recently thanks to retracted research papers, so psychologists set out to discover what was going on. When research is valid, others should be able to duplicate the study and come up with the same results. But when more than 250 scientists tried to re-create 100 studies from three respected psychology journals—working closely with original researchers—the results matched in just 39 cases. Even among those, 83% were found to have smaller effects than first reported, reports the Verge. The studies covered topics like personality, relationships, learning, and memory, and were the kind trusted by therapists and educators, reports the New York Times. A scientist argues the reproducibility issue could apply across other fields, too. "To see it so clearly, on such a large scale—it's unprecedented," adds another. "There is little known about the reproducibility of research in general," says study author Brian Nosek. "And there has been a growing concern that reproducibility may be lower than expected or desired." He adds that scientific journals tend to focus on new research, so those hoping to get published don't waste time testing older studies. Nosek doesn't consider the results "negative or pessimistic," but he hopes they'll lead to self-correction and replication standards. So are these results reproducible? "Even this project itself is not a final word, a last word, a definitive word about reproducibility," Nosek says.