Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking

Sophie Graham; Elise Tessier; Julia Stowe; Jamie Lopez Bernal; Edward P.K. Parker; Dorothea Nitsch; Elizabeth Miller; Nick Andrews; Jemma L. Walker; Helen I. McDonald

doi:10.1038/s41467-023-39674-0

Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking

Sophie Graham, Elise Tessier, Julia Stowe, Jamie Lopez Bernal, Edward P.K. Parker, Dorothea Nitsch, Elizabeth Miller, Nick Andrews, Jemma L. Walker, Helen I. McDonald

Research output: Contribution to journal › Article › peer-review

Abstract

National test-negative-case-control (TNCC) studies are used to monitor COVID-19 vaccine effectiveness in the UK. A questionnaire was sent to participants from the first published TNCC COVID-19 vaccine effectiveness study conducted by the UK Health Security Agency, to assess for potential biases and changes in behaviour related to vaccination. The original study included symptomatic adults aged ≥70 years testing for COVID-19 between 08/12/2020 and 21/02/2021. A questionnaire was sent to cases and controls tested from 1–21 February 2021. In this study, 8648 individuals responded to the questionnaire (36.5% response). Using information from the questionnaire to produce a combined estimate that accounted for all potential biases decreased the original vaccine effectiveness estimate after two doses of BNT162b2 from 88% (95% CI: 79–94%) to 85% (95% CI: 68–94%). Self-reported behaviour demonstrated minimal evidence of riskier behaviour after vaccination. These findings offer reassurance to policy makers and clinicians making decisions based on COVID-19 vaccine effectiveness TNCC studies.

Original language	English
Article number	3984
Journal	Nature Communications
Volume	14
Issue number	1
DOIs	https://doi.org/10.1038/s41467-023-39674-0
Publication status	Published - 6 Jul 2023
Externally published	Yes

Bibliographical note

Funding Information:
This work uses data provided by patients and collected by the NHS as part of their care and support ( http://www.usemydata.org ). In addition, we would like to thank participants of the questionnaire who provided valuable data that enabled the conduct of this study. S.G., E.M., N.A., J.L.W. and N.A. and H.I.M. are funded by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit in Vaccines and Immunisation (grant reference NIHR200929), a partnership between UK Health Security Agency and London School of Hygiene & Tropical Medicine. EPKP received funding from the UKRI COVID-19 Longitudinal Health and Wellbeing National Core Study (Phase 1 LHW-NCS, MC_PC-20059). The views expressed are those of the author(s) and not necessarily those of the NIHR, UK Health Security Agency or the Department of Health and Social Care.

Data availability
Access to pseudonymised national datasets used in this study (National Immunisation Management Service and Second Generation Surveillance System) is managed by NHS England through the NHS COVID-19 Data Store: https://www.england.nhs.uk/contact-us/privacy-notice/how-we-use-your-information/covid-19-response/nhs-covid-19-data-store/. Questionnaire data was collected for the purposes of public health service evaluation and consent was not obtained for further sharing for research. To discuss a request for UKHSA data you would like to submit, contact [email protected].

ASJC Scopus subject areas

General Chemistry
General Biochemistry,Genetics and Molecular Biology
General Physics and Astronomy

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s41467-023-39674-0Licence: CC BY

Cite this

@article{0f92192a8cc74d77a51adf5f55a8efb4,

title = "Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking",

abstract = "National test-negative-case-control (TNCC) studies are used to monitor COVID-19 vaccine effectiveness in the UK. A questionnaire was sent to participants from the first published TNCC COVID-19 vaccine effectiveness study conducted by the UK Health Security Agency, to assess for potential biases and changes in behaviour related to vaccination. The original study included symptomatic adults aged ≥70 years testing for COVID-19 between 08/12/2020 and 21/02/2021. A questionnaire was sent to cases and controls tested from 1–21 February 2021. In this study, 8648 individuals responded to the questionnaire (36.5% response). Using information from the questionnaire to produce a combined estimate that accounted for all potential biases decreased the original vaccine effectiveness estimate after two doses of BNT162b2 from 88% (95% CI: 79–94%) to 85% (95% CI: 68–94%). Self-reported behaviour demonstrated minimal evidence of riskier behaviour after vaccination. These findings offer reassurance to policy makers and clinicians making decisions based on COVID-19 vaccine effectiveness TNCC studies.",

author = "Sophie Graham and Elise Tessier and Julia Stowe and Bernal, {Jamie Lopez} and Parker, {Edward P.K.} and Dorothea Nitsch and Elizabeth Miller and Nick Andrews and Walker, {Jemma L.} and McDonald, {Helen I.}",

note = "Funding Information: This work uses data provided by patients and collected by the NHS as part of their care and support ( http://www.usemydata.org ). In addition, we would like to thank participants of the questionnaire who provided valuable data that enabled the conduct of this study. S.G., E.M., N.A., J.L.W. and N.A. and H.I.M. are funded by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit in Vaccines and Immunisation (grant reference NIHR200929), a partnership between UK Health Security Agency and London School of Hygiene & Tropical Medicine. EPKP received funding from the UKRI COVID-19 Longitudinal Health and Wellbeing National Core Study (Phase 1 LHW-NCS, MC_PC-20059). The views expressed are those of the author(s) and not necessarily those of the NIHR, UK Health Security Agency or the Department of Health and Social Care. Data availability Access to pseudonymised national datasets used in this study (National Immunisation Management Service and Second Generation Surveillance System) is managed by NHS England through the NHS COVID-19 Data Store: https://www.england.nhs.uk/contact-us/privacy-notice/how-we-use-your-information/covid-19-response/nhs-covid-19-data-store/. Questionnaire data was collected for the purposes of public health service evaluation and consent was not obtained for further sharing for research. To discuss a request for UKHSA data you would like to submit, contact [email protected].",

year = "2023",

month = jul,

day = "6",

doi = "10.1038/s41467-023-39674-0",

language = "English",

volume = "14",

journal = "Nature Communications",

issn = "2041-1723",

publisher = "Nature Research",

number = "1",

}

TY - JOUR

T1 - Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking

AU - Graham, Sophie

AU - Tessier, Elise

AU - Stowe, Julia

AU - Bernal, Jamie Lopez

AU - Parker, Edward P.K.

AU - Nitsch, Dorothea

AU - Miller, Elizabeth

AU - Andrews, Nick

AU - Walker, Jemma L.

AU - McDonald, Helen I.

N1 - Funding Information: This work uses data provided by patients and collected by the NHS as part of their care and support ( http://www.usemydata.org ). In addition, we would like to thank participants of the questionnaire who provided valuable data that enabled the conduct of this study. S.G., E.M., N.A., J.L.W. and N.A. and H.I.M. are funded by the National Institute for Health and Care Research (NIHR) Health Protection Research Unit in Vaccines and Immunisation (grant reference NIHR200929), a partnership between UK Health Security Agency and London School of Hygiene & Tropical Medicine. EPKP received funding from the UKRI COVID-19 Longitudinal Health and Wellbeing National Core Study (Phase 1 LHW-NCS, MC_PC-20059). The views expressed are those of the author(s) and not necessarily those of the NIHR, UK Health Security Agency or the Department of Health and Social Care. Data availability Access to pseudonymised national datasets used in this study (National Immunisation Management Service and Second Generation Surveillance System) is managed by NHS England through the NHS COVID-19 Data Store: https://www.england.nhs.uk/contact-us/privacy-notice/how-we-use-your-information/covid-19-response/nhs-covid-19-data-store/. Questionnaire data was collected for the purposes of public health service evaluation and consent was not obtained for further sharing for research. To discuss a request for UKHSA data you would like to submit, contact [email protected].

PY - 2023/7/6

Y1 - 2023/7/6

N2 - National test-negative-case-control (TNCC) studies are used to monitor COVID-19 vaccine effectiveness in the UK. A questionnaire was sent to participants from the first published TNCC COVID-19 vaccine effectiveness study conducted by the UK Health Security Agency, to assess for potential biases and changes in behaviour related to vaccination. The original study included symptomatic adults aged ≥70 years testing for COVID-19 between 08/12/2020 and 21/02/2021. A questionnaire was sent to cases and controls tested from 1–21 February 2021. In this study, 8648 individuals responded to the questionnaire (36.5% response). Using information from the questionnaire to produce a combined estimate that accounted for all potential biases decreased the original vaccine effectiveness estimate after two doses of BNT162b2 from 88% (95% CI: 79–94%) to 85% (95% CI: 68–94%). Self-reported behaviour demonstrated minimal evidence of riskier behaviour after vaccination. These findings offer reassurance to policy makers and clinicians making decisions based on COVID-19 vaccine effectiveness TNCC studies.

AB - National test-negative-case-control (TNCC) studies are used to monitor COVID-19 vaccine effectiveness in the UK. A questionnaire was sent to participants from the first published TNCC COVID-19 vaccine effectiveness study conducted by the UK Health Security Agency, to assess for potential biases and changes in behaviour related to vaccination. The original study included symptomatic adults aged ≥70 years testing for COVID-19 between 08/12/2020 and 21/02/2021. A questionnaire was sent to cases and controls tested from 1–21 February 2021. In this study, 8648 individuals responded to the questionnaire (36.5% response). Using information from the questionnaire to produce a combined estimate that accounted for all potential biases decreased the original vaccine effectiveness estimate after two doses of BNT162b2 from 88% (95% CI: 79–94%) to 85% (95% CI: 68–94%). Self-reported behaviour demonstrated minimal evidence of riskier behaviour after vaccination. These findings offer reassurance to policy makers and clinicians making decisions based on COVID-19 vaccine effectiveness TNCC studies.

UR - http://www.scopus.com/inward/record.url?scp=85164100334&partnerID=8YFLogxK

U2 - 10.1038/s41467-023-39674-0

DO - 10.1038/s41467-023-39674-0

M3 - Article

AN - SCOPUS:85164100334

SN - 2041-1723

VL - 14

JO - Nature Communications

JF - Nature Communications

IS - 1

M1 - 3984

ER -

Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking

Abstract

Bibliographical note

ASJC Scopus subject areas

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this