Failure to Reproduce: The Replication Crisis in Research — Can Librarians Help?

Failure to Reproduce: The Replication Crisis in Research — Can Librarians Help?

Carver Room

“It can be proven that most claimed research findings are false.” Those are the words of John Ioannidis in a highly-cited article from 2005. Ioannidis is referring to the “reproducibility crisis,” a phenomenon whereby researchers are not able to replicate published results in later experiments. A recent survey by Nature found that more than 70% of researchers have tried and failed to reproduce another scientist’s experiments and more than half have failed to reproduce their own.

In this presentation, we will introduce attendees to the replication crisis and provide real-life examples of reproducibility problems in the fields of psychology, economics, animal research, and biomedical research. We will outline the primary causes of the problem (the “file-drawer” problem, publication bias, poor experimental design, and the incentive structure for researchers) and will also note the unfortunate failure of peer review to weed out many false findings.

From there, we will discuss how librarians are assisting researchers in designing reproducible workflows that can help prevent research replication failure. These workflows include proper experimental design, proper management and documentation of research data and code, and the use of open-science tools for registering experiments, collaborating with colleagues, and sharing research outputs. We will conclude with a demonstration of one important tool in this area, the Open Science Framework from the Center for Open Science. We will cover how it works, how to use it to connect to outside services, and its support for versioning, collaboration, and sharing preprints.

Attendees will come away with a better understanding of the reproducibility crisis, the role librarians are playing in assisting researchers with reproducible workflows, and a popular tool they can use for doing so.