Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays

This research aims to provide an interaction mechanism for large displays that allow users to interact with the display both from a distance and up close. They have come up with a method to use the hand to control the cursor by attaching reflective markers to a couple fingers and points on the hand and wrist and tracking those points.

There are several "modes" of interaction. There is a clicking mode in which a click can be performed by either pushing the index finger down in the air or bringing the thumb up to the side of the hand. Feedback is introduced with some on-screen animation and sound since there is no tactile feedback on the index finger and little feedback with the thumb.

There are a couple different pointing modes they have come up with. First, they use a ray-casting mode that basically acts as if a pointer is coming out of the index finger. There is also a relative mode of movement as well as a combination of pointing and relative movement.

They found that their methods worked pretty well and had high accuracy, though the clicking and pointing modes cannot be used together.

------------

I think this type of interaction is really interesting and cool. I really like touch interaction, and this extension of touch to control from a distance is interesting, especially because the interaction at a distance works up close when the display is touched. No other systems allow an easy transition from far away to up close.

This research made me think of the CyberTouch gloves (link) I am currently working on with Manoj. We have the ability to get fingertip and wrist locations already, and we have the added benefit of vibrotactile feedback with this particular glove. We are currently working with 2D and 3D tracking, so we have the ability to do things like this here in our lab, which is exciting to me.

Comments: Kevin, Josh, Manoj, Franck, Paul, Sashi