Low statistical power in biomedical science

a review of three human research domains

Estelle Dumas-Mallet, Katherine S. Button, Thomas Boraud, Francois Gonon, Marcus R. Munafò

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

Original languageEnglish
Article number160254
JournalRoyal Society Open Science
Volume4
Issue number2
DOIs
Publication statusPublished - 1 Feb 2017

Fingerprint

Meta-Analysis
Research
Psychiatry
Research Design
Genes

Keywords

  • Neurology
  • Psychiatry
  • Reproducibility
  • Somatic disease
  • Statistical power

Cite this

Low statistical power in biomedical science : a review of three human research domains. / Dumas-Mallet, Estelle; Button, Katherine S.; Boraud, Thomas; Gonon, Francois; Munafò, Marcus R.

In: Royal Society Open Science, Vol. 4, No. 2, 160254, 01.02.2017.

Research output: Contribution to journalArticle

Dumas-Mallet, Estelle ; Button, Katherine S. ; Boraud, Thomas ; Gonon, Francois ; Munafò, Marcus R. / Low statistical power in biomedical science : a review of three human research domains. In: Royal Society Open Science. 2017 ; Vol. 4, No. 2.
@article{aa073e28fb274f54ae922005c4bd894c,
title = "Low statistical power in biomedical science: a review of three human research domains",
abstract = "Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5{\%}, we found that approximately 50{\%} of studies have statistical power in the 0–10{\%} or 11–20{\%} range, well below the minimum of 80{\%} that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.",
keywords = "Neurology, Psychiatry, Reproducibility, Somatic disease, Statistical power",
author = "Estelle Dumas-Mallet and Button, {Katherine S.} and Thomas Boraud and Francois Gonon and Munaf{\`o}, {Marcus R.}",
year = "2017",
month = "2",
day = "1",
doi = "10.1098/rsos.160254",
language = "English",
volume = "4",
journal = "Royal Society Open Science",
issn = "2054-5703",
publisher = "The Royal Society",
number = "2",

}

TY - JOUR

T1 - Low statistical power in biomedical science

T2 - a review of three human research domains

AU - Dumas-Mallet, Estelle

AU - Button, Katherine S.

AU - Boraud, Thomas

AU - Gonon, Francois

AU - Munafò, Marcus R.

PY - 2017/2/1

Y1 - 2017/2/1

N2 - Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

AB - Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

KW - Neurology

KW - Psychiatry

KW - Reproducibility

KW - Somatic disease

KW - Statistical power

UR - http://www.scopus.com/inward/record.url?scp=85011579953&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1098/rsos.160254

UR - http://dx.doi.org/10.1098/rsos.160254

U2 - 10.1098/rsos.160254

DO - 10.1098/rsos.160254

M3 - Article

VL - 4

JO - Royal Society Open Science

JF - Royal Society Open Science

SN - 2054-5703

IS - 2

M1 - 160254

ER -