Preventing the ends from justifying the means

withholding results to address publication bias in peer-review

Katherine S. Button, Liz Bal, Anna Clark, Tim Shipley

Research output: Contribution to journalEditorial

11 Citations (Scopus)

Abstract

The evidence that many of the findings in the published literature may be unreliable is compelling. There is an excess of positive results, often from studies with small sample sizes, or other methodological limitations, and the conspicuous absence of null findings from studies of a similar quality. This distorts the evidence base, leading to false conclusions and undermining scientific progress. Central to this problem is a peer-review system where the decisions of authors, reviewers, and editors are more influenced by impressive results than they are by the validity of the study design. To address this, BMC Psychology is launching a pilot to trial a new 'results-free' peer-review process, whereby editors and reviewers are blinded to the study's results, initially assessing manuscripts on the scientific merits of the rationale and methods alone. The aim is to improve the reliability and quality of published research, by focusing editorial decisions on the rigour of the methods, and preventing impressive ends justifying poor means.

Original languageEnglish
Article number59
JournalPsychology
Volume4
Issue number1
DOIs
Publication statusPublished - 1 Dec 2016

Fingerprint

Publication Bias
Peer Review
Manuscripts
Reproducibility of Results
Sample Size
Psychology
Research

Keywords

  • Peer review
  • Publication bias
  • Results-free review
  • Transparency

Cite this

Preventing the ends from justifying the means : withholding results to address publication bias in peer-review. / Button, Katherine S.; Bal, Liz; Clark, Anna; Shipley, Tim.

In: Psychology, Vol. 4, No. 1, 59, 01.12.2016.

Research output: Contribution to journalEditorial

@article{c95f05315ac84031b544c4caa6b65d81,
title = "Preventing the ends from justifying the means: withholding results to address publication bias in peer-review",
abstract = "The evidence that many of the findings in the published literature may be unreliable is compelling. There is an excess of positive results, often from studies with small sample sizes, or other methodological limitations, and the conspicuous absence of null findings from studies of a similar quality. This distorts the evidence base, leading to false conclusions and undermining scientific progress. Central to this problem is a peer-review system where the decisions of authors, reviewers, and editors are more influenced by impressive results than they are by the validity of the study design. To address this, BMC Psychology is launching a pilot to trial a new 'results-free' peer-review process, whereby editors and reviewers are blinded to the study's results, initially assessing manuscripts on the scientific merits of the rationale and methods alone. The aim is to improve the reliability and quality of published research, by focusing editorial decisions on the rigour of the methods, and preventing impressive ends justifying poor means.",
keywords = "Peer review, Publication bias, Results-free review, Transparency",
author = "Button, {Katherine S.} and Liz Bal and Anna Clark and Tim Shipley",
year = "2016",
month = "12",
day = "1",
doi = "10.1186/s40359-016-0167-7",
language = "English",
volume = "4",
journal = "Psychology",
issn = "2152-7180",
number = "1",

}

TY - JOUR

T1 - Preventing the ends from justifying the means

T2 - withholding results to address publication bias in peer-review

AU - Button, Katherine S.

AU - Bal, Liz

AU - Clark, Anna

AU - Shipley, Tim

PY - 2016/12/1

Y1 - 2016/12/1

N2 - The evidence that many of the findings in the published literature may be unreliable is compelling. There is an excess of positive results, often from studies with small sample sizes, or other methodological limitations, and the conspicuous absence of null findings from studies of a similar quality. This distorts the evidence base, leading to false conclusions and undermining scientific progress. Central to this problem is a peer-review system where the decisions of authors, reviewers, and editors are more influenced by impressive results than they are by the validity of the study design. To address this, BMC Psychology is launching a pilot to trial a new 'results-free' peer-review process, whereby editors and reviewers are blinded to the study's results, initially assessing manuscripts on the scientific merits of the rationale and methods alone. The aim is to improve the reliability and quality of published research, by focusing editorial decisions on the rigour of the methods, and preventing impressive ends justifying poor means.

AB - The evidence that many of the findings in the published literature may be unreliable is compelling. There is an excess of positive results, often from studies with small sample sizes, or other methodological limitations, and the conspicuous absence of null findings from studies of a similar quality. This distorts the evidence base, leading to false conclusions and undermining scientific progress. Central to this problem is a peer-review system where the decisions of authors, reviewers, and editors are more influenced by impressive results than they are by the validity of the study design. To address this, BMC Psychology is launching a pilot to trial a new 'results-free' peer-review process, whereby editors and reviewers are blinded to the study's results, initially assessing manuscripts on the scientific merits of the rationale and methods alone. The aim is to improve the reliability and quality of published research, by focusing editorial decisions on the rigour of the methods, and preventing impressive ends justifying poor means.

KW - Peer review

KW - Publication bias

KW - Results-free review

KW - Transparency

UR - http://www.scopus.com/inward/record.url?scp=85000645628&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1186/s40359-016-0167-7

UR - http://dx.doi.org/10.1186/s40359-016-0167-7

U2 - 10.1186/s40359-016-0167-7

DO - 10.1186/s40359-016-0167-7

M3 - Editorial

VL - 4

JO - Psychology

JF - Psychology

SN - 2152-7180

IS - 1

M1 - 59

ER -