Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

Paul Hanel, David M. A. Mehler

Research output: Contribution to journalArticle

3 Citations (Scopus)
63 Downloads (Pure)

Abstract

Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d, while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).

Original languageEnglish
Pages (from-to)468-485
Number of pages18
JournalPublic Understanding of Science
Volume28
Issue number4
Early online date8 Mar 2019
DOIs
Publication statusPublished - 1 May 2019

Keywords

  • Cohen’s U3
  • Cohen’s d
  • effect size
  • scientific communication
  • statistical communication
  • statistical significance

ASJC Scopus subject areas

  • Communication
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)

Cite this

Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication. / Hanel, Paul; Mehler, David M. A. .

In: Public Understanding of Science, Vol. 28, No. 4, 01.05.2019, p. 468-485.

Research output: Contribution to journalArticle

@article{b36cf4730a2a43dfaf0ae47e890d683c,
title = "Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication",
abstract = "Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69{\%}) found U3 more informative than Cohen’s d, while 95 (15{\%}) found d more informative than U3, with 99 participants (16{\%}) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).",
keywords = "Cohen’s U3, Cohen’s d, effect size, scientific communication, statistical communication, statistical significance",
author = "Paul Hanel and Mehler, {David M. A.}",
year = "2019",
month = "5",
day = "1",
doi = "10.1177/0963662519834193",
language = "English",
volume = "28",
pages = "468--485",
journal = "Public Understanding of Science",
issn = "0963-6625",
publisher = "Sage Publications",
number = "4",

}

TY - JOUR

T1 - Beyond Reporting Statistical Significance: Identifying Informative Effect Sizes to Improve Scientific Communication

AU - Hanel, Paul

AU - Mehler, David M. A.

PY - 2019/5/1

Y1 - 2019/5/1

N2 - Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d, while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).

AB - Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d, while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).

KW - Cohen’s U3

KW - Cohen’s d

KW - effect size

KW - scientific communication

KW - statistical communication

KW - statistical significance

UR - http://www.scopus.com/inward/record.url?scp=85062728405&partnerID=8YFLogxK

U2 - 10.1177/0963662519834193

DO - 10.1177/0963662519834193

M3 - Article

VL - 28

SP - 468

EP - 485

JO - Public Understanding of Science

JF - Public Understanding of Science

SN - 0963-6625

IS - 4

ER -