Type

Presentation

Description

“It can be proven that most claimed research findings are false.” Those are the words of John Ioannidis in a highly-cited article from 2005. Ioannidis is referring to the “reproducibility crisis,” a phenomenon whereby researchers are not able to replicate published results in later experiments. A recent survey by Nature found that more than 70% of researchers have tried and failed to reproduce another scientist’s experiments and more than half have failed to reproduce their own.

In this presentation, we will introduce attendees to the replication crisis and provide real-life examples of reproducibility problems in the fields of psychology, economics, animal research, and biomedical research. We will outline the primary causes of the problem (the “file-drawer” problem, publication bias, poor experimental design, and the incentive structure for researchers) and will also note the unfortunate failure of peer review to weed out many false findings.

From there, we will discuss how librarians are assisting researchers in designing reproducible workflows that can help prevent research replication failure. These workflows include proper experimental design, proper management and documentation of research data and code, and the use of open-science tools for registering experiments, collaborating with colleagues, and sharing research outputs. We will conclude with a demonstration of one important tool in this area, the Open Science Framework from the Center for Open Science. We will cover how it works, how to use it to connect to outside services, and its support for versioning, collaboration, and sharing preprints.

Attendees will come away with a better understanding of the reproducibility crisis, the role librarians are playing in assisting researchers with reproducible workflows, and a popular tool they can use for doing so.

More Information

This presentation fits into general conversations about the need to move the scholarly communication system toward greater openness and the opportunities provided by digital technologies and platforms to bring about systemic change.

Type of Library

University Library

Keywords

science, research, experiments, reproducibility, replication crisis, open science

Share

COinS
 
May 4th, 3:40 PM May 4th, 4:30 PM

Failure to Reproduce: The Replication Crisis in Research — Can Librarians Help?

Carver Room

“It can be proven that most claimed research findings are false.” Those are the words of John Ioannidis in a highly-cited article from 2005. Ioannidis is referring to the “reproducibility crisis,” a phenomenon whereby researchers are not able to replicate published results in later experiments. A recent survey by Nature found that more than 70% of researchers have tried and failed to reproduce another scientist’s experiments and more than half have failed to reproduce their own.

In this presentation, we will introduce attendees to the replication crisis and provide real-life examples of reproducibility problems in the fields of psychology, economics, animal research, and biomedical research. We will outline the primary causes of the problem (the “file-drawer” problem, publication bias, poor experimental design, and the incentive structure for researchers) and will also note the unfortunate failure of peer review to weed out many false findings.

From there, we will discuss how librarians are assisting researchers in designing reproducible workflows that can help prevent research replication failure. These workflows include proper experimental design, proper management and documentation of research data and code, and the use of open-science tools for registering experiments, collaborating with colleagues, and sharing research outputs. We will conclude with a demonstration of one important tool in this area, the Open Science Framework from the Center for Open Science. We will cover how it works, how to use it to connect to outside services, and its support for versioning, collaboration, and sharing preprints.

Attendees will come away with a better understanding of the reproducibility crisis, the role librarians are playing in assisting researchers with reproducible workflows, and a popular tool they can use for doing so.

 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.