How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

Joanna Bergström; Tor-salve Dalsgaard; Jason Alexander; Kasper Hornbæk

doi:10.1145/3411764.3445193

How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

Joanna Bergström, Tor-salve Dalsgaard, Jason Alexander, Kasper Hornbæk

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

75 Citations (SciVal)

993 Downloads (Pure)

Abstract

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefts over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate fndings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them.We also identify research directions for improving evaluation methods and ofer ideas for how to make studies more ecologically valid and rigorous.

Original language	English
Title of host publication	CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Subtitle of host publication	Making Waves, Combining Strengths
Publisher	Association for Computing Machinery
Pages	1-20
Number of pages	20
Volume	May 2021
ISBN (Electronic)	9781450380966
DOIs	https://doi.org/10.1145/3411764.3445193
Publication status	Published - 7 May 2021

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Bibliographical note

Funding Information:
This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No. 853063).

Publisher Copyright:
© 2021 ACM.

Keywords

Experiments
Object selection and manipulation
Virtual reality

ASJC Scopus subject areas

Human-Computer Interaction
Computer Graphics and Computer-Aided Design
Software

Access to Document

10.1145/3411764.3445193

CHI21_Object_Selection_VR
© ACM, 2021. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing systems. http://doi.acm.org/10.1145/3411764.3445193
Accepted author manuscript, 941 KB

https://dl.acm.org/doi/10.1145/3411764.3445193

Cite this

Bergström, J., Dalsgaard, T., Alexander, J., & Hornbæk, K. (2021). How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. In CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths (Vol. May 2021, pp. 1-20). Article 533 (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3411764.3445193

How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. / Bergström, Joanna; Dalsgaard, Tor-salve; Alexander, Jason et al.
CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths. Vol. May 2021 Association for Computing Machinery, 2021. p. 1-20 533 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Bergström, J, Dalsgaard, T, Alexander, J & Hornbæk, K 2021, How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. in CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths. vol. May 2021, 533, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, pp. 1-20. https://doi.org/10.1145/3411764.3445193

Bergström J, Dalsgaard T, Alexander J, Hornbæk K. How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. In CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths. Vol. May 2021. Association for Computing Machinery. 2021. p. 1-20. 533. (Conference on Human Factors in Computing Systems - Proceedings). doi: 10.1145/3411764.3445193

Bergström, Joanna ; Dalsgaard, Tor-salve ; Alexander, Jason et al. / How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies. CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths. Vol. May 2021 Association for Computing Machinery, 2021. pp. 1-20 (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{379a030120bb4e7caffaa6c175951960,

title = "How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies",

abstract = "The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefts over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate fndings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them.We also identify research directions for improving evaluation methods and ofer ideas for how to make studies more ecologically valid and rigorous.",

keywords = "Experiments, Object selection and manipulation, Virtual reality",

author = "Joanna Bergstr{\"o}m and Tor-salve Dalsgaard and Jason Alexander and Kasper Hornb{\ae}k",

note = "Funding Information: This project has received funding from the European Research Council (ERC) under the European Union{\textquoteright}s Horizon 2020 research and innovation programme (Grant agreement No. 853063). Publisher Copyright: {\textcopyright} 2021 ACM.",

year = "2021",

month = may,

day = "7",

doi = "10.1145/3411764.3445193",

language = "English",

volume = "May 2021",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

pages = "1--20",

booktitle = "CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems",

address = "USA United States",

}

TY - GEN

T1 - How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

AU - Bergström, Joanna

AU - Dalsgaard, Tor-salve

AU - Alexander, Jason

AU - Hornbæk, Kasper

N1 - Funding Information: This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant agreement No. 853063). Publisher Copyright: © 2021 ACM.

PY - 2021/5/7

Y1 - 2021/5/7

N2 - The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefts over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate fndings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them.We also identify research directions for improving evaluation methods and ofer ideas for how to make studies more ecologically valid and rigorous.

AB - The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefts over the state-of-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate fndings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them.We also identify research directions for improving evaluation methods and ofer ideas for how to make studies more ecologically valid and rigorous.

KW - Experiments

KW - Object selection and manipulation

KW - Virtual reality

UR - http://www.scopus.com/inward/record.url?scp=85106717866&partnerID=8YFLogxK

U2 - 10.1145/3411764.3445193

DO - 10.1145/3411764.3445193

M3 - Chapter in a published conference proceeding

VL - May 2021

T3 - Conference on Human Factors in Computing Systems - Proceedings

SP - 1

EP - 20

BT - CHI 2021 - Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

ER -

How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

Abstract

Publication series

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this