Reproducibility of research in psychology investigated
An international study presented in Science Magazine questions the reproducibility of much of the published findings in psychology journals. Researchers tried to replicate 100 studies from three prominent journals, and found that regardless of the analytic method or criteria used fewer than half of their replications produced the same findings as the original study.
“A failure to reproduce does not necessarily mean that the original report was incorrect, but this shows the challenges of reproducing research findings”, says study co-author Gustav Nilsonne, MD, PhD, affiliated to the Stress Research Institute at Stockholm University, and to Karolinska Institutet. “Scientific evidence should not depend on trusting the person that made the discovery, rather on credibility accumulating through independent replication and the elaboration of ideas and evidence.”
The so-called Reproducibility Project: Psychology was launched in 2011, and has since resulted in similar efforts within other research areas. The current study is the largest systematic investigation of reproducibility ever made in any research field. The team behind it consists of 270 researchers from all over the world, who have contributed through crowdsourcing, which is unique in itself.
“In the present academic culture, scientists’ main incentive is to publish as many scientific papers in high impact journals as possible”, says Gustav Nilsonne. "Research presenting new and surprising findings is more likely to be published, even at the cost of reproducibility of the findings. However, what is good for science and what is good for scientists, is not always the same thing, and vice versa. Therefore, we need more transparency and openness about methods and research data, so that independent review and replication is possible.”
- Press release from the Center for Open Science
- Editorial comment about the findings
- News article in The Atlantic
Publication
Estimating the reproducibility of psychological science – Open Science Collaboration
Science 28 August 2015: Vol. 349 no. 6251, DOI: 10.1126/science.aac4716