Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Jessie R. Baldwin; Jean Baptiste Pingault; Tabea Schoeler; Hannah M. Sallis; Marcus R. Munafò

doi:10.1007/s10654-021-00839-0

Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Jessie R. Baldwin, Jean Baptiste Pingault, Tabea Schoeler, Hannah M. Sallis, Marcus R. Munafò

Vice Chancellor's Office

Research output: Contribution to journal › Article › peer-review

99 Citations (SciVal)

Abstract

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

Original language	English
Journal	European Journal of Epidemiology
Volume	37
Issue number	1
Early online date	13 Jan 2022
DOIs	https://doi.org/10.1007/s10654-021-00839-0
Publication status	Published - 13 Jan 2022

Acknowledgements

The authors are grateful to Professor George Davey for his helpful comments on this article.

Funding

J.R.B is funded by a Wellcome Trust Sir Henry Wellcome fellowship (grant 215917/Z/19/Z). J.B.P is a supported by the Medical Research Foundation 2018 Emerging Leaders 1st Prize in Adolescent Mental Health (MRF-160–0002-ELP-PINGA). M.R.M and H.M.S work in a unit that receives funding from the University of Bristol and the UK Medical Research Council (MC_UU_00011/5, MC_UU_00011/7), and M.R.M is also supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at the University Hospitals Bristol National Health Service Foundation Trust and the University of Bristol.

Keywords

Open science
Pre-registration
Researcher bias
Secondary data analysis

ASJC Scopus subject areas

Epidemiology

Access to Document

10.1007/s10654-021-00839-0Licence: CC BY

Cite this

@article{cabe3e7c76534c21ae7b014a601eb191,

title = "Protecting against researcher bias in secondary data analysis: challenges and potential solutions",

abstract = "Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society{\textquoteright}s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.",

keywords = "Open science, Pre-registration, Researcher bias, Secondary data analysis",

author = "Baldwin, \{Jessie R.\} and Pingault, \{Jean Baptiste\} and Tabea Schoeler and Sallis, \{Hannah M.\} and Munaf{\`o}, \{Marcus R.\}",

year = "2022",

month = jan,

day = "13",

doi = "10.1007/s10654-021-00839-0",

language = "English",

volume = "37",

journal = "European Journal of Epidemiology",

issn = "0393-2990",

publisher = "Springer Science and Business Media B.V.",

number = "1",

}

TY - JOUR

T1 - Protecting against researcher bias in secondary data analysis

T2 - challenges and potential solutions

AU - Baldwin, Jessie R.

AU - Pingault, Jean Baptiste

AU - Schoeler, Tabea

AU - Sallis, Hannah M.

AU - Munafò, Marcus R.

PY - 2022/1/13

Y1 - 2022/1/13

N2 - Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

AB - Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

KW - Open science

KW - Pre-registration

KW - Researcher bias

KW - Secondary data analysis

UR - https://www.scopus.com/pages/publications/85123078664

U2 - 10.1007/s10654-021-00839-0

DO - 10.1007/s10654-021-00839-0

M3 - Article

C2 - 35025022

AN - SCOPUS:85123078664

SN - 0393-2990

VL - 37

JO - European Journal of Epidemiology

JF - European Journal of Epidemiology

IS - 1

ER -

Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Abstract

Acknowledgements

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this