Credible impact evaluation in complex contexts

Confirmatory and exploratory approaches

Research output: Contribution to journalArticle

15 Citations (Scopus)
109 Downloads (Pure)

Abstract

Debate continues over how best international development agencies can evaluate the impact of actions intended to reduce poverty, insecurity and vulnerability in diverse and complex contexts. There are strong ethical grounds for simply asking those intended to benefit what happened to them, but it is not obvious how to do so in a way that is sufficiently free from bias in favour of confirming what is expected. This article considers scope for addressing this problem by minimising the prior knowledge participants have of what is being evaluated. The tensions between more confirmatory and exploratory methodological approaches are reviewed in the light of experience of designing and piloting a qualitative impact assessment protocol for evaluating NGO interventions in complex rural livelihood transformations. The paper concludes that resolving these tensions entails using mixed methodologies, and that the importance attached to exploratory (nested within confirmatory) approaches depends on contextual complexity, the type of evidence sought and the level of trust between stakeholders.
Original languageEnglish
Pages (from-to)412-427
Number of pages16
JournalEvaluation
Volume20
Issue number4
Early online date30 Sep 2014
DOIs
Publication statusPublished - 1 Oct 2014

Fingerprint

evaluation
nongovernmental organization
non-governmental organization
livelihood
poverty
vulnerability
stakeholder
methodology
trend
knowledge
evidence
experience
impact assessment
protocol

Keywords

  • Impact evaluation
  • Impact assessment
  • ART
  • qualitative methods
  • development practice
  • exploratory and confirmatory analysis
  • pro-project bias

Cite this

Credible impact evaluation in complex contexts : Confirmatory and exploratory approaches. / Copestake, J G.

In: Evaluation, Vol. 20, No. 4, 01.10.2014, p. 412-427.

Research output: Contribution to journalArticle

@article{1f304a45fbf748f7bde4c9fc94622885,
title = "Credible impact evaluation in complex contexts: Confirmatory and exploratory approaches",
abstract = "Debate continues over how best international development agencies can evaluate the impact of actions intended to reduce poverty, insecurity and vulnerability in diverse and complex contexts. There are strong ethical grounds for simply asking those intended to benefit what happened to them, but it is not obvious how to do so in a way that is sufficiently free from bias in favour of confirming what is expected. This article considers scope for addressing this problem by minimising the prior knowledge participants have of what is being evaluated. The tensions between more confirmatory and exploratory methodological approaches are reviewed in the light of experience of designing and piloting a qualitative impact assessment protocol for evaluating NGO interventions in complex rural livelihood transformations. The paper concludes that resolving these tensions entails using mixed methodologies, and that the importance attached to exploratory (nested within confirmatory) approaches depends on contextual complexity, the type of evidence sought and the level of trust between stakeholders.",
keywords = "Impact evaluation, Impact assessment, ART, qualitative methods, development practice, exploratory and confirmatory analysis, pro-project bias",
author = "Copestake, {J G}",
year = "2014",
month = "10",
day = "1",
doi = "10.1177/1356389014550559",
language = "English",
volume = "20",
pages = "412--427",
journal = "Evaluation",
issn = "1356-3890",
publisher = "Sage Publications",
number = "4",

}

TY - JOUR

T1 - Credible impact evaluation in complex contexts

T2 - Confirmatory and exploratory approaches

AU - Copestake, J G

PY - 2014/10/1

Y1 - 2014/10/1

N2 - Debate continues over how best international development agencies can evaluate the impact of actions intended to reduce poverty, insecurity and vulnerability in diverse and complex contexts. There are strong ethical grounds for simply asking those intended to benefit what happened to them, but it is not obvious how to do so in a way that is sufficiently free from bias in favour of confirming what is expected. This article considers scope for addressing this problem by minimising the prior knowledge participants have of what is being evaluated. The tensions between more confirmatory and exploratory methodological approaches are reviewed in the light of experience of designing and piloting a qualitative impact assessment protocol for evaluating NGO interventions in complex rural livelihood transformations. The paper concludes that resolving these tensions entails using mixed methodologies, and that the importance attached to exploratory (nested within confirmatory) approaches depends on contextual complexity, the type of evidence sought and the level of trust between stakeholders.

AB - Debate continues over how best international development agencies can evaluate the impact of actions intended to reduce poverty, insecurity and vulnerability in diverse and complex contexts. There are strong ethical grounds for simply asking those intended to benefit what happened to them, but it is not obvious how to do so in a way that is sufficiently free from bias in favour of confirming what is expected. This article considers scope for addressing this problem by minimising the prior knowledge participants have of what is being evaluated. The tensions between more confirmatory and exploratory methodological approaches are reviewed in the light of experience of designing and piloting a qualitative impact assessment protocol for evaluating NGO interventions in complex rural livelihood transformations. The paper concludes that resolving these tensions entails using mixed methodologies, and that the importance attached to exploratory (nested within confirmatory) approaches depends on contextual complexity, the type of evidence sought and the level of trust between stakeholders.

KW - Impact evaluation

KW - Impact assessment

KW - ART

KW - qualitative methods

KW - development practice

KW - exploratory and confirmatory analysis

KW - pro-project bias

U2 - 10.1177/1356389014550559

DO - 10.1177/1356389014550559

M3 - Article

VL - 20

SP - 412

EP - 427

JO - Evaluation

JF - Evaluation

SN - 1356-3890

IS - 4

ER -