Motion Correlation: Selecting Objects by Matching Their Movement: Selecting Objects by Matching Their Movement

Eduardo Velloso, Marcus Carter, Joshua Newn, Augusto Emanuel Abreu Esteves, Christopher Clarke, Hans-Werner Georg Gellersen

Research output: Contribution to journalArticlepeer-review


Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking its specific motion, and use the correlation between the system’s output with the user’s input to determine the selection. The resulting interaction has compelling properties, as users are guided by motion feedback, and only need to copy a presented motion. Motion correlation has been explored in earlier work but only recently begun to feature in holistic interface designs. We provide a first comprehensive review of the principle, and present an analysis of five previously published works, in which motion correlation underpinned the design of novel gaze and gesture interfaces for diverse application contexts. We derive guidelines for motion correlation algorithms, motion feedback, choice of modalities, overall design of motion correlation interfaces, and identify opportunities and challenges identified for future research and design.
Original languageEnglish
Article number22
JournalACM Transactions on Computer-Human Interaction
Issue number3
Early online date30 Apr 2017
Publication statusPublished - 22 Jul 2017

Cite this