The Perceived Feasibility of Methods to Reduce Publication Bias

Harriet Amy Carroll, Zoi Toumpakari, Laura Johnson, James Betts

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Publication bias (PB) is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce PB, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce PB by gauging opinions of journal editors (n=73) and other academics/researchers (n=160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing PB for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6 % of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce PB.
Original languageEnglish
Article numbere0186472
JournalPLoS ONE
Volume12
Issue number10
DOIs
Publication statusPublished - 24 Oct 2017

Fingerprint

Publication Bias
Research Personnel
Publications
researchers
methodology
Gaging
Accreditation
peers
Research
Literature
Peer Review
questionnaires

Cite this

The Perceived Feasibility of Methods to Reduce Publication Bias. / Carroll, Harriet Amy; Toumpakari, Zoi; Johnson, Laura; Betts, James.

In: PLoS ONE, Vol. 12, No. 10, e0186472, 24.10.2017.

Research output: Contribution to journalArticle

Carroll, Harriet Amy ; Toumpakari, Zoi ; Johnson, Laura ; Betts, James. / The Perceived Feasibility of Methods to Reduce Publication Bias. In: PLoS ONE. 2017 ; Vol. 12, No. 10.
@article{32ae264c442c479db3062ca100b9361f,
title = "The Perceived Feasibility of Methods to Reduce Publication Bias",
abstract = "Publication bias (PB) is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce PB, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce PB by gauging opinions of journal editors (n=73) and other academics/researchers (n=160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing PB for editors (25{\%}), and was the third most popular choice for academics/researchers (14{\%}). The most common selection for academics/researchers was two-stage review (26{\%}), but fewer editors prioritised this (11{\%}). Negative results journals/articles were the second and third most common choices for academics/researchers (21{\%}) and editors (16{\%}), respectively. Editors more commonly chose research registration as ‘most effective’ (21{\%}), which was favoured by only 6 {\%} of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce PB.",
author = "Carroll, {Harriet Amy} and Zoi Toumpakari and Laura Johnson and James Betts",
year = "2017",
month = "10",
day = "24",
doi = "10.1371/journal.pone.0186472",
language = "English",
volume = "12",
journal = "PLoS ONE",
issn = "1932-6203",
publisher = "Public Library of Science (PLOS)",
number = "10",

}

TY - JOUR

T1 - The Perceived Feasibility of Methods to Reduce Publication Bias

AU - Carroll, Harriet Amy

AU - Toumpakari, Zoi

AU - Johnson, Laura

AU - Betts, James

PY - 2017/10/24

Y1 - 2017/10/24

N2 - Publication bias (PB) is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce PB, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce PB by gauging opinions of journal editors (n=73) and other academics/researchers (n=160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing PB for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6 % of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce PB.

AB - Publication bias (PB) is prevalent within the scientific literature. Whilst there are multiple ideas on how to reduce PB, only a minority of journals have made substantive changes to address the problem. We aimed to explore the perceived feasibility of strategies to reduce PB by gauging opinions of journal editors (n=73) and other academics/researchers (n=160) regarding nine methods of publishing and peer-reviewing research: mandatory publication, negative results journals/articles, open reviewing, peer-review training and accreditation, post-publication review, pre-study publication of methodology, published rejection lists, research registration, and two-stage review. Participants completed a questionnaire asking both quantitative (multiple choice or Likert scales) and qualitative (open-ended) questions regarding the barriers to implementing each suggestion, and their strengths and limitations. Participants were asked to rate the nine suggestions, then choose the method they felt was most effective. Mandatory publication was most popularly selected as the ‘most effective’ method of reducing PB for editors (25%), and was the third most popular choice for academics/researchers (14%). The most common selection for academics/researchers was two-stage review (26%), but fewer editors prioritised this (11%). Negative results journals/articles were the second and third most common choices for academics/researchers (21%) and editors (16%), respectively. Editors more commonly chose research registration as ‘most effective’ (21%), which was favoured by only 6 % of academics/researchers. Whilst mandatory publication was generally favoured by respondents, it is infeasible to trial at a journal level. Where suggestions have already been implemented (e.g. negative results journals/articles, trial registration), efforts should be made to objectively assess their efficacy. Two-stage review should be further trialled as its popularity amongst academics/researchers suggests it may be well received, though editors may be less receptive. Several underlying barriers to change also emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce PB.

UR - http://dx.doi.org/10.1371/journal.pone.0186472

U2 - 10.1371/journal.pone.0186472

DO - 10.1371/journal.pone.0186472

M3 - Article

VL - 12

JO - PLoS ONE

JF - PLoS ONE

SN - 1932-6203

IS - 10

M1 - e0186472

ER -