Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

Paul Hanel, David M. A. Mehler

Research output: Contribution to journalArticle

Abstract

Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition of p-value based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g., differences or similarities).
LanguageEnglish
JournalPublic Understanding of Science
StatusAccepted/In press - 5 Feb 2019

Cite this

@article{b36cf4730a2a43dfaf0ae47e890d683c,
title = "Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication",
abstract = "Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition of p-value based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69{\%}) found U3 more informative than Cohen’s d while 95 (15{\%}) found d more informative than U3, with 99 participants (16{\%}) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g., differences or similarities).",
author = "Paul Hanel and Mehler, {David M. A.}",
year = "2019",
month = "2",
day = "5",
language = "English",
journal = "Public Understanding of Science",
issn = "0963-6625",
publisher = "Sage Publications",

}

TY - JOUR

T1 - Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

AU - Hanel, Paul

AU - Mehler, David M. A.

PY - 2019/2/5

Y1 - 2019/2/5

N2 - Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition of p-value based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g., differences or similarities).

AB - Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition of p-value based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g., differences or similarities).

M3 - Article

JO - Public Understanding of Science

T2 - Public Understanding of Science

JF - Public Understanding of Science

SN - 0963-6625

ER -