Understanding Freehand Cursorless Pointing Variability and Its Impact on Selection Performance

Freehand pointing is a fundamental gesture commonly used for cursorless interactions. Prior work in HCI often elicits the same pointing behaviour– facing the target with an outstretched dominant arm and index finger. However, freehand pointing outside of HCI shows more variability across hand pose, usage, and coordination with gaze. To understand what variability exists and how it affects pointing performance, we collected data (N=23) using a hybrid motion capture system. To elicit a wide variety of pointing behaviours we included different levels of user effort and attention, as well as the widest range of target placements studied. We systematically characterised and described three distinct pointing behaviours, each with three different traits, ranging from accurate stereotypical pointing observed in prior works to more casual hip fire-style pointing. Our analysis demonstrates how different pointing behaviours affect pointing performance and highlights their importance when designing interactive systems for more naturalistic freehand pointing.