Updates to data versions and analytic methods influence the reproducibility of results from epigenome-wide association studies

Alexandre A. Lussier; Yiwen Zhu; Brooke J. Smith; Andrew J. Simpkin; Andrew D.A.C. Smith; Matthew J. Suderman; Esther Walton; Kerry J. Ressler; Erin C. Dunn

doi:10.1080/15592294.2022.2028072

Updates to data versions and analytic methods influence the reproducibility of results from epigenome-wide association studies

Alexandre A. Lussier, Yiwen Zhu, Brooke J. Smith, Andrew J. Simpkin, Andrew D.A.C. Smith, Matthew J. Suderman, Esther Walton, Kerry J. Ressler, Erin C. Dunn

Research output: Contribution to journal › Article › peer-review

8 Citations (SciVal)

52 Downloads (Pure)

Abstract

Biomedical research has grown increasingly cooperative through the sharing of consortia-level epigenetic data. Since consortia preprocess data prior to distribution, new processing pipelines can lead to different versions of the same dataset. Similarly, analytic frameworks evolve to incorporate cutting-edge methods and best practices. However, it remains unknown how different data and analytic versions alter the results of epigenome-wide analyses, which could influence the replicability of epigenetic associations. Thus, we assessed the impact of these changes using data from the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. We analysed DNA methylation from two data versions, processed using separate preprocessing and analytic pipelines, examining associations between seven childhood adversities or prenatal smoking exposure and DNA methylation at age 7. We performed two sets of analyses: (1) epigenome-wide association studies (EWAS); (2) Structured Life Course Modelling Approach (SLCMA), a two-stage method that models time-dependent effects. SLCMA results were also compared across two analytic versions. Data version changes impacted both EWAS and SLCMA analyses, yielding different associations at conventional p-value thresholds. However, the magnitude and direction of associations was generally consistent between data versions, regardless of p-values. Differences were especially apparent in analyses of childhood adversity, while smoking associations were more consistent using significance thresholds. SLCMA analytic versions similarly altered top associations, but time-dependent effects remained concordant. Alterations to data and analytic versions influenced the results of epigenome-wide analyses. Our findings highlight that magnitude and direction are better measures for replication and stability than p-value thresholds.

Original language	English
Pages (from-to)	1373-1388
Number of pages	16
Journal	Epigenetics
Volume	17
Issue number	11
Early online date	14 Feb 2022
DOIs	https://doi.org/10.1080/15592294.2022.2028072
Publication status	Published - 31 Dec 2022

Bibliographical note

Funding Information:
Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk . Dr. Walton is also supported by the European Union’s Horizon 2020 research and innovation programme (grant n° 848158).

Funding Information:
This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website ( http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf ); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper.

Funding Information:
This work was supported by the Economic and Social Research Council [ES/K000357/1]; Horizon 2020 [848158]; National Institute of Child Health and Human Development [R01HD068437]; National Institute of Mental Health [R01MH113930]; National Institutes of Health [5RO1AI121226-02]; UK Medical Research Council and Wellcome [217065/Z/19/Z]; CONTAMED EU [212502]; IEU [MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8]; OneMind [OneMind Rising Star Award]; Biotechnology and Biological Sciences Research Council [BBI025751/1; BB/I025263/1]. This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website (http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper. Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk. Dr. Walton is also supported by the European Union’s Horizon 2020 research and innovation programme (grant n° 848158).

Keywords

adversity
ALSPAC
analytic versions
DNA methylation
epigenetic data versions
reproducibility
updates/revised

ASJC Scopus subject areas

Molecular Biology
Cancer Research

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1080/15592294.2022.2028072

2021.04.23.441014v1.full.PDF
This is an Accepted Manuscript of an article published by Taylor & Francis in Epigenetics on 14/02/2022, available online: http://www.tandfonline.com/10.1080/15592294.2022.2028072
Accepted author manuscript, 8.1 MB

Cite this

@article{2c92bfbb67d840f28e30144de95a41f0,

title = "Updates to data versions and analytic methods influence the reproducibility of results from epigenome-wide association studies",

abstract = "Biomedical research has grown increasingly cooperative through the sharing of consortia-level epigenetic data. Since consortia preprocess data prior to distribution, new processing pipelines can lead to different versions of the same dataset. Similarly, analytic frameworks evolve to incorporate cutting-edge methods and best practices. However, it remains unknown how different data and analytic versions alter the results of epigenome-wide analyses, which could influence the replicability of epigenetic associations. Thus, we assessed the impact of these changes using data from the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. We analysed DNA methylation from two data versions, processed using separate preprocessing and analytic pipelines, examining associations between seven childhood adversities or prenatal smoking exposure and DNA methylation at age 7. We performed two sets of analyses: (1) epigenome-wide association studies (EWAS); (2) Structured Life Course Modelling Approach (SLCMA), a two-stage method that models time-dependent effects. SLCMA results were also compared across two analytic versions. Data version changes impacted both EWAS and SLCMA analyses, yielding different associations at conventional p-value thresholds. However, the magnitude and direction of associations was generally consistent between data versions, regardless of p-values. Differences were especially apparent in analyses of childhood adversity, while smoking associations were more consistent using significance thresholds. SLCMA analytic versions similarly altered top associations, but time-dependent effects remained concordant. Alterations to data and analytic versions influenced the results of epigenome-wide analyses. Our findings highlight that magnitude and direction are better measures for replication and stability than p-value thresholds.",

keywords = "adversity, ALSPAC, analytic versions, DNA methylation, epigenetic data versions, reproducibility, updates/revised",

author = "Lussier, {Alexandre A.} and Yiwen Zhu and Smith, {Brooke J.} and Simpkin, {Andrew J.} and Smith, {Andrew D.A.C.} and Suderman, {Matthew J.} and Esther Walton and Ressler, {Kerry J.} and Dunn, {Erin C.}",

note = "Funding Information: Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk . Dr. Walton is also supported by the European Union{\textquoteright}s Horizon 2020 research and innovation programme (grant n° 848158). Funding Information: This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website ( http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf ); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper. Funding Information: This work was supported by the Economic and Social Research Council [ES/K000357/1]; Horizon 2020 [848158]; National Institute of Child Health and Human Development [R01HD068437]; National Institute of Mental Health [R01MH113930]; National Institutes of Health [5RO1AI121226-02]; UK Medical Research Council and Wellcome [217065/Z/19/Z]; CONTAMED EU [212502]; IEU [MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8]; OneMind [OneMind Rising Star Award]; Biotechnology and Biological Sciences Research Council [BBI025751/1; BB/I025263/1]. This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website (http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper. Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk. Dr. Walton is also supported by the European Union{\textquoteright}s Horizon 2020 research and innovation programme (grant n° 848158). ",

year = "2022",

month = dec,

day = "31",

doi = "10.1080/15592294.2022.2028072",

language = "English",

volume = "17",

pages = "1373--1388",

journal = "Epigenetics",

issn = "1559-2294",

publisher = "Landes Bioscience",

number = "11",

}

TY - JOUR

T1 - Updates to data versions and analytic methods influence the reproducibility of results from epigenome-wide association studies

AU - Lussier, Alexandre A.

AU - Zhu, Yiwen

AU - Smith, Brooke J.

AU - Simpkin, Andrew J.

AU - Smith, Andrew D.A.C.

AU - Suderman, Matthew J.

AU - Walton, Esther

AU - Ressler, Kerry J.

AU - Dunn, Erin C.

N1 - Funding Information: Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk . Dr. Walton is also supported by the European Union’s Horizon 2020 research and innovation programme (grant n° 848158). Funding Information: This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website ( http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf ); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper. Funding Information: This work was supported by the Economic and Social Research Council [ES/K000357/1]; Horizon 2020 [848158]; National Institute of Child Health and Human Development [R01HD068437]; National Institute of Mental Health [R01MH113930]; National Institutes of Health [5RO1AI121226-02]; UK Medical Research Council and Wellcome [217065/Z/19/Z]; CONTAMED EU [212502]; IEU [MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8]; OneMind [OneMind Rising Star Award]; Biotechnology and Biological Sciences Research Council [BBI025751/1; BB/I025263/1]. This work was supported by the National Institute of Mental Health of the National Institutes of Health (grant number R01MH113930 awarded to ECD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Dunn and Dr. Lussier were also supported by a grant from One Mind. We are extremely grateful to all the families who took part in the ALSPAC study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and Wellcome (Grant ref: 217065/Z/19/Z) and the University of Bristol provide core support for ALSPAC. A comprehensive list of grants funding is available on the ALSPAC website (http://www.bristol.ac.uk/alspac/external/documents/grant-acknowledgements.pdf); This research was specifically funded by grants from the BBSRC (BBI025751/1; BB/I025263/1), IEU (MC_UU_12013/1; MC_UU_12013/2; MC_UU_12013/8), National Institute of Child and Human Development (R01HD068437), NIH (5RO1AI121226-02), and CONTAMED EU (212502). This publication is the work of the authors, whom will serve as guarantors for the contents of this paper. Dr. Walton is funded by CLOSER, whose mission is to maximise the use, value and impact of longitudinal studies. CLOSER was funded by the Economic and Social Research Council (ESRC) and the Medical Research Council (MRC) between 2012 and 2017. Its initial five-year grant has since been extended to March 2021 by the ESRC (grant reference: ES/K000357/1). The funders took no role in the design, execution, analysis or interpretation of the data or in the writing up of the findings. www.closer.ac.uk. Dr. Walton is also supported by the European Union’s Horizon 2020 research and innovation programme (grant n° 848158).

PY - 2022/12/31

Y1 - 2022/12/31

N2 - Biomedical research has grown increasingly cooperative through the sharing of consortia-level epigenetic data. Since consortia preprocess data prior to distribution, new processing pipelines can lead to different versions of the same dataset. Similarly, analytic frameworks evolve to incorporate cutting-edge methods and best practices. However, it remains unknown how different data and analytic versions alter the results of epigenome-wide analyses, which could influence the replicability of epigenetic associations. Thus, we assessed the impact of these changes using data from the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. We analysed DNA methylation from two data versions, processed using separate preprocessing and analytic pipelines, examining associations between seven childhood adversities or prenatal smoking exposure and DNA methylation at age 7. We performed two sets of analyses: (1) epigenome-wide association studies (EWAS); (2) Structured Life Course Modelling Approach (SLCMA), a two-stage method that models time-dependent effects. SLCMA results were also compared across two analytic versions. Data version changes impacted both EWAS and SLCMA analyses, yielding different associations at conventional p-value thresholds. However, the magnitude and direction of associations was generally consistent between data versions, regardless of p-values. Differences were especially apparent in analyses of childhood adversity, while smoking associations were more consistent using significance thresholds. SLCMA analytic versions similarly altered top associations, but time-dependent effects remained concordant. Alterations to data and analytic versions influenced the results of epigenome-wide analyses. Our findings highlight that magnitude and direction are better measures for replication and stability than p-value thresholds.

AB - Biomedical research has grown increasingly cooperative through the sharing of consortia-level epigenetic data. Since consortia preprocess data prior to distribution, new processing pipelines can lead to different versions of the same dataset. Similarly, analytic frameworks evolve to incorporate cutting-edge methods and best practices. However, it remains unknown how different data and analytic versions alter the results of epigenome-wide analyses, which could influence the replicability of epigenetic associations. Thus, we assessed the impact of these changes using data from the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort. We analysed DNA methylation from two data versions, processed using separate preprocessing and analytic pipelines, examining associations between seven childhood adversities or prenatal smoking exposure and DNA methylation at age 7. We performed two sets of analyses: (1) epigenome-wide association studies (EWAS); (2) Structured Life Course Modelling Approach (SLCMA), a two-stage method that models time-dependent effects. SLCMA results were also compared across two analytic versions. Data version changes impacted both EWAS and SLCMA analyses, yielding different associations at conventional p-value thresholds. However, the magnitude and direction of associations was generally consistent between data versions, regardless of p-values. Differences were especially apparent in analyses of childhood adversity, while smoking associations were more consistent using significance thresholds. SLCMA analytic versions similarly altered top associations, but time-dependent effects remained concordant. Alterations to data and analytic versions influenced the results of epigenome-wide analyses. Our findings highlight that magnitude and direction are better measures for replication and stability than p-value thresholds.

KW - adversity

KW - ALSPAC

KW - analytic versions

KW - DNA methylation

KW - epigenetic data versions

KW - reproducibility

KW - updates/revised

UR - http://www.scopus.com/inward/record.url?scp=85125287957&partnerID=8YFLogxK

U2 - 10.1080/15592294.2022.2028072

DO - 10.1080/15592294.2022.2028072

M3 - Article

C2 - 35156895

AN - SCOPUS:85125287957

SN - 1559-2294

VL - 17

SP - 1373

EP - 1388

JO - Epigenetics

JF - Epigenetics

IS - 11

ER -

Updates to data versions and analytic methods influence the reproducibility of results from epigenome-wide association studies

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this