A foundational observation method for studying design situations

Philip Cash, Ben Hicks, Steve Culley, Tim Adlam

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.

Original languageEnglish
Pages (from-to)187-219
Number of pages33
JournalJournal of Engineering Design
Volume26
Issue number7-9
Early online date23 Mar 2015
DOIs
Publication statusPublished - 2 Sep 2015

Fingerprint

Agglomeration
Standardization

Keywords

  • descriptive models of the design process
  • empirical study
  • ethnography

Cite this

A foundational observation method for studying design situations. / Cash, Philip; Hicks, Ben; Culley, Steve; Adlam, Tim.

In: Journal of Engineering Design, Vol. 26, No. 7-9, 02.09.2015, p. 187-219.

Research output: Contribution to journalArticle

@article{0b97abe52a464fe8b8d083620b5369d3,
title = "A foundational observation method for studying design situations",
abstract = "Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.",
keywords = "descriptive models of the design process, empirical study, ethnography",
author = "Philip Cash and Ben Hicks and Steve Culley and Tim Adlam",
year = "2015",
month = "9",
day = "2",
doi = "10.1080/09544828.2015.1020418",
language = "English",
volume = "26",
pages = "187--219",
journal = "Journal of Engineering Design",
issn = "0954-4828",
publisher = "Taylor and Francis",
number = "7-9",

}

TY - JOUR

T1 - A foundational observation method for studying design situations

AU - Cash, Philip

AU - Hicks, Ben

AU - Culley, Steve

AU - Adlam, Tim

PY - 2015/9/2

Y1 - 2015/9/2

N2 - Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.

AB - Observational studies of designers play an important role in engineering design research, yet there is currently no accepted standard approach for comparing, combining, or contrasting studies. Consequentially, reuse, reanalysis, replication, and aggregation of data are limited and the potential impact of individual studies is severely constrained. This paper begins to address this issue by introducing and developing a foundational method for observational design research to improve replicability, reuse, and overall comparability of empirical studies. A three-step foundational method is proposed that covers capture, coding, and analysis. The capture step defines overall and situational context as well as multiple capture streams, generating a broad data-set that can be examined from multiple perspectives. The coding step employs a multi-level approach that seeks to minimise workload while describing both detailed and high-level information. The analysis step builds on the multi-level approach to provide for a flexible yet standardised examination of the data-set. The overall approach is introduced theoretically and illustrated using a comparison of an industrial study and an experimental study. Finally, it is argued that the proposed method promotes rigour, reliability, and standardisation; and could provide one means for improving comparison and aggregation, ultimately increasing impact in academia and practice.

KW - descriptive models of the design process

KW - empirical study

KW - ethnography

UR - http://www.scopus.com/inward/record.url?scp=84925433307&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1080/09544828.2015.1020418

U2 - 10.1080/09544828.2015.1020418

DO - 10.1080/09544828.2015.1020418

M3 - Article

VL - 26

SP - 187

EP - 219

JO - Journal of Engineering Design

JF - Journal of Engineering Design

SN - 0954-4828

IS - 7-9

ER -