This paper talks about gesture recognition using vision-based techniques to recognize 6 hand gestures for use in augmented reality.
Because the gestures are recognized using a vision based approach, the only hardware needed is a standard camera. The user does not need to wear a glove.
The hand gestures are recognized by manipulating the image of the hand's pixels in such a way that the hand becomes white and the background becomes solid black, even if there are other objects in the background.
To count the number of fingers that are outstretched, concentric circles about the hand are examined.
One extended finger is recognized as a pointing gesture, and a transition between three states is used for a click gesture, in which the thumb is used to "click."
The generality of this algorithm means people with varying hands can use it. Also, it works for both left and right hands.
The author claims that this gesturing method was easily understood and used in a simple AR application, which I don't feel like describing...
------------
I like the vision-based approach used in this method. The CyberGlove is not easy to wear for a long period of time, so I like any solution that works with a bare hand as input. I have not studied any computer-vision methods, so this is new to me. I was wondering what the resolution was of the camera they use, because it seems that doing per-pixel calculations would take a long time.
I also wonder how sensitive the glove is to different skin tones. They mentioned an initialization step, but I don't remember if they specified if that included setting the skin color.
I'm sure we have all made hand puppets in which we use two fingers as legs. Lam, et al. have used this idea to map this motion to 3D models to control animation. Specifically, they use the finger-leg motions to manipulate previously-captured motion data. This can potentially turn an existing walking animation into a running, dancing, or hopping animation.
The finger motion data was captured using a P5 glove. This glove does not provide as good or as much data as our CyberGloves, but are adequate for this research, especially when considering the limitations of human fingers themselves when compared to legs.
The human body has many more degrees of motion than the fingers, which the researchers acknowledge and try to compensate for in their mapping function. The mapping function they have implemented converts the finger-leg motion into full body motion. To control things like arms, simple, common rules are used, such as the opposite movement of the arms to the legs when walking or running. As expected, this method produces unnatural animations. The researchers propose some methods of remedying this for future work.
A small experiment was described in which an animator manipulated the motion-capture data by performing walking, running, and hopping motions with the hand. A complete analysis was not presented, nor was a complete description of the users.
------------
I think this is an interesting concept that can be expanded on. I like the fact that Lam tries to tackle the problem of key-frame, static animation by introducing the dynamic, real-time finger controlled animation. I think this method would be good for establishing timing and rhythm, but I think the animations could be more defined by physics-based calculations rather than a direct mapping of the hand to the body.
EyePoint™ is a gaze-based system for mouse cursor control that aims to provide a worthwhile alternative to the mouse to the degree that the average computer user might choose this system over a mouse, depending on the task at hand. Kumar wants to move eye tracking systems out of the domain of disabled users and into everyday computing with normal people.
Design of the system
EyePoint's design incorporates three principles:
1. They do not want to "overload the visual channel for pointing," i.e. don't map interaction directly to eye movement.
2. They aim to increase selection accuracy by including zooming/magnification in such a way that it enhances accuracy but does not inhibit performance. The solution to this is to magnify a square around the gaze area and overlay a grid of orange dots, called "focus points," to enhance fine selection. The magnification avoids distortion of the interface that some techniques, such as fish-eye magnification, introduce. The magnified area is partially transparent as well, to avoid obscuring the interface, as some zooming techniques do.
3. They want to reduce jitter, which they do with "fixation detection and [a] smoothing algorithm."
4. The selection method must be "fluid" and simple in order to make fast selections while maintaining usability by both disabled and normal functioning people. The solution to this is to perform selections with some "hotkeys" (corresponding to click, double click, right click, mouse over, start click-and-drag, and end click-and-drag) on a standard keyboard. The thought is that pressing keys is faster and better than dwell and zooming selection methods.
This system uses a Tobii 1750 eye tracking system.
Usage of the EyePoint software
1. The user looks at the target on the screen.
2. The user presses down one of the hotkeys (click, double click, right click, mouse over, start click-and-drag, and end click-and-drag)
3. A square around the estimated gaze point is magnified in the manner described above.
4. The user looks at the target in the magnified area (this helps enhance the selection)
5. The user releases the hotkey
6. The action of the hotkey is performed
The researchers think that this system can equally be used by able-bodied and disabled users. They propose changing the hotkeys to some other input device depending on the situations of the users. No testing was done in this area, however.
User Study and evaluation
A user study was conducted with 20 normal, able-bodied, participants who were very experienced computer users.
A quantitative evaluation was performed which had the users perform specific tasks, including navigating through web pages and clicking targets. Data including timing and error rates were collected. The tasks were varied to account for any learning curves.
A qualitative evaluation was also performed using a questionnaire to get users opinions of the EyePoint system compared to a standard mouse input.
Results
Web browsing:
The "focus points" did not help with timing or accuracy.
The average time to click a target with EyePoint was about 400 ms slower than a mouse.
The mouse had a lower error rate by about 10% in the normal EyePoint tests.
User's were divided on which system they thought was faster and easier to use (mouse or EyePoint).
75% of the users said they might choose EyePoint in certain situations.
Most users preferred the focus points.
Target selection:
EyePoint is only marginally slower than the mouse for selection.
EyePoint has much higher error rates than the mouse.
Users generally preferred EyePoint for its ease of use despite its lower performance.
Analysis
It was found that EyePoint is similar in speed to the mouse. The error rates for EyePoint vary from user to user. The study participants preferred EyePoint to the mouse.
The researchers conclude that it is possible and practical for an eye tracking system to be used by the common computer user in place of the mouse.
Watch the video if you want:
------------
I like this eye tracking system the best of the papers we have read so far this semester. I have thought of using eye tracking to augment the mouse in some way, which would mean a system is mostly mouse-based with some eye help. This system is the opposite. It is a an eye-based system with some hand click help. This is pretty cool, and the speed and accuracy results look a lot better than some of the other work we looked at.
I am most intrigued by the magnification solution to enhance accuracy and reduce error by introducing a second focusing saccade at the cost of speed. I do think emphasizing accuracy over speed was a good move by Kumar, et al. since the mouse is already pretty darn fast and the time to actually focus on a target doesn't seem that it could be any faster than the mouse, at least for experienced computer users.
The EyePoint system might increase selection speed for computer novices that might not be used to using a mouse and therefore use it slower than us experienced users. After all, not all people are experience computer users, but nearly all people can see.
