Utility evaluation of models

Jean Scholtz; Oriana Love; Mark Whiting; Duncan Hodges; Lia Emanuel; Danae Stanton Fraser

doi:10.1145/2669557.2669562

Utility evaluation of models

Jean Scholtz, Oriana Love, Mark Whiting, Duncan Hodges, Lia Emanuel, Danae Stanton Fraser

Department of Psychology

Research output: Chapter or section in a book/report/conference proceeding › Chapter or section

244 Downloads (Pure)

Abstract

In this paper, we present three case studies of utility evaluations of underlying models in software systems: a user-model, technical and social models both singly and in combination, and a research-based model for user identification. Each of the three cases used a different approach to evaluating the model and each had challenges to overcome in designing and implementing the evaluation. We describe the methods we used and challenges faced in designing the evaluation procedures, summarize the lessons learned, enumerate considerations for those undertaking such evaluations, and present directions for future work.

Original language	English
Title of host publication	BELIV '14 Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization
Publisher	Association for Computing Machinery
Pages	160-167
DOIs	https://doi.org/10.1145/2669557.2669562
Publication status	Published - 2014

Keywords

Utility evaluation
abstract models
evaluation design

Access to Document

10.1145/2669557.2669562

Scholtz et al. BELIV 2014 Utility evaluation of models (pre-print)
© ACM, 2014. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in BELIV '14 Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, http://doi.acm.org/10.1145/10.1145/2669557.2669562"
Submitted manuscript, 919 KB

Cite this

@inbook{aeb49ef27a144c4da14019f6547f5b67,

title = "Utility evaluation of models",

abstract = "In this paper, we present three case studies of utility evaluations of underlying models in software systems: a user-model, technical and social models both singly and in combination, and a research-based model for user identification. Each of the three cases used a different approach to evaluating the model and each had challenges to overcome in designing and implementing the evaluation. We describe the methods we used and challenges faced in designing the evaluation procedures, summarize the lessons learned, enumerate considerations for those undertaking such evaluations, and present directions for future work.",

keywords = "Utility evaluation, abstract models, evaluation design",

author = "Jean Scholtz and Oriana Love and Mark Whiting and Duncan Hodges and Lia Emanuel and {Stanton Fraser}, Danae",

year = "2014",

doi = "10.1145/2669557.2669562",

language = "English",

pages = "160--167 ",

booktitle = "BELIV '14 Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization",

publisher = "Association for Computing Machinery",

address = "USA United States",

}

TY - CHAP

T1 - Utility evaluation of models

AU - Scholtz, Jean

AU - Love, Oriana

AU - Whiting, Mark

AU - Hodges, Duncan

AU - Emanuel, Lia

AU - Stanton Fraser, Danae

PY - 2014

Y1 - 2014

N2 - In this paper, we present three case studies of utility evaluations of underlying models in software systems: a user-model, technical and social models both singly and in combination, and a research-based model for user identification. Each of the three cases used a different approach to evaluating the model and each had challenges to overcome in designing and implementing the evaluation. We describe the methods we used and challenges faced in designing the evaluation procedures, summarize the lessons learned, enumerate considerations for those undertaking such evaluations, and present directions for future work.

AB - In this paper, we present three case studies of utility evaluations of underlying models in software systems: a user-model, technical and social models both singly and in combination, and a research-based model for user identification. Each of the three cases used a different approach to evaluating the model and each had challenges to overcome in designing and implementing the evaluation. We describe the methods we used and challenges faced in designing the evaluation procedures, summarize the lessons learned, enumerate considerations for those undertaking such evaluations, and present directions for future work.

KW - Utility evaluation

KW - abstract models

KW - evaluation design

UR - http://dx.doi.org/10.1145/2669557.2669562

U2 - 10.1145/2669557.2669562

DO - 10.1145/2669557.2669562

M3 - Chapter or section

SP - 160

EP - 167

BT - BELIV '14 Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization

PB - Association for Computing Machinery

ER -

Utility evaluation of models

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this