Subject: simple code to get simple gestures

From: [email-masked] (Greg Alt)
Date: Sat, 12 Oct 91 22:40:41 -0600

------------- cut here for pwrglove.h ----------------
typedef struct _glove_data
    {
    signed char  dum0,x,y,z,rot,fingers,keys,dum7,dum8,dum9,dumA,dumB;
    } glove_data;

void init_glove (void);
void query_glove (glove_data *);
------------- end of pwrglove.h ----------------------
------------- cut here for fingers.c -----------------
#include <stdio.h>
#include "pwrglove.h"
#include "pwrglove.c"
 
#define F1(x)   ((x>>6)&3)
#define F2(x)   ((x>>4)&3)
#define F3(x)   ((x>>2)&3)
#define F4(x)   (x&3)
#define HAND(x) (((x&2)>>1)|((x&8)>>2)|((x&32)>>3)|((x&128)>>4))
 
static char  gesture[16][8]={
       "Open   ",
       "Dbl Gun",
       "0010   ",
       "Gun    ",
       "0100   ",
       "0101   ",
       "0110   ",
       "Thumb  ",
       "Three  ",
       "Dbl Pnt",
       "1010   ",
       "Point  ",
       "O.K.   ",
       "Birdie ",
       "1110   ",
       "Fist   "};
 
void main ()
{
	glove_data   glov;
	int          hand,dir;
 
	init_glove();
	while(1)
	{
	query_glove(&glov);
	hand=HAND(glov.fingers);
	printf("%s  (%x)  (%d,%d,%d)\n",
	       gesture[hand],glov.rot,glov.x,glov.y,glov.z);
	}
}
 
------------- end of fingers.c -----------------------
I'm working on something that will strip some of the
noise out of the positional data by possibly predicting
movement. does anyone know if there are any books/papers
that discuss a decent way of doing this without having
a lag most of the time?  My first goal is to make a game
of rock/paper/scissors (once I get vogle up and running).
Anyway, have fun with this code...
    Greg

---

From: Lance Norskog <[email-masked]>
Date: 14 Oct 91 11:11:36 PDT (Mon)

Yes, Fred Brooks of the Pixel-Planes project said that predictive
tracking should work best.  

Graphics Gems II has a thing on predictive coding as compression technique.
Here's the idea: you have a state machine which reads a sample stream and 
guesses the next sample, and output the difference between your guess
and the actual sample.  This should give you lots of samples with low
values, suitable for Huffman or dictionary compression.  I've been
itching to try this on sound samples.  The article does it on pictures,
and shows the error output as a separate picture.  It's an excellent
demonstration of the principle.

A first attempt would track the slope of the last few samples, thus
extending the first derivative out.  If you assume that the input
stream is delayed by X milliseconds, and it samples at half your
update rate, you can create a separate one-dimensional tracker
for each of (X, Y, Z, rot) and do a better job than drawing
from raw input.  You can also reject weird inputs, and average
noisy ones.

If you move your hand fast and then stop, the predictive tracking
will overshoot and come back to the resting place.  Cest la vie.

I'd say it's time to sample movements, run the output through
a statistics package, and figure out just what kind of error
you need to deal with.

Lance Norskog

---

From: Dave Stampe-Psy+Eng <[email-masked]>
Date: Mon, 14 Oct 91 18:35:47 -0400

> From [email-masked] Mon Oct 14 15:44:19 1991
> To: [email-masked]
> Subject: Re: simple code to get simple gestures
> Cc: [email-masked]
> From: Lance Norskog <[email-masked]>
> 
> 
Sounds like a Kalman predictive filter.  The question is, how much will
the overshoots affect operator performance?
If you're going to develop the code for such a filter, I hope you make
several versions, using different delays.  This would allow users to
match the filter to their own system's video display, rendering and
processing times.  Isuggest multiples of 33 mS.
Also, it would be interesting to set up a simple system using a "bar"
cursor or other easy-to-draw symbol, and try out the effect of changing
the filter coefficients.  After all, we're getting into psychophysics
here, and that is *not* an easy field to predict.  I've noticed weird
"viscous" effects using head-position to cursor feedback with a system
delay of 50 mS or less.  Addition of hysterisis of 1/6 visual degree
stopped that, and reduced noise, but at the cost of some of the 
"reality" feel of the system.  It's amazing how little it takes to 
make such feedback feel like a "tool" rather than an extension of
the body.

Dave Stampe

---

From: Lance Norskog <[email-masked]>
Date: 14 Oct 91 17:15:36 PDT (Mon)

Sounds like you know what you're talking about and I don't.

Yes, this needs a lot of experimentation, and preferably a
menu and calibration system.  I don't believe in canned software.
"I know what you need.  Trust me!"

References, please?  Should I look for books by Kalman at the
Stanford CS library?  References in CACM year-end indices?

Thanks,

Lance Norskog

---

From: Dave Stampe-Psy+Eng <[email-masked]>
Date: Mon, 14 Oct 91 21:58:04 -0400

A "Kalman filter" refers to any of a class of predictive filters.  The simplest predictive filter is just a "highpass" type.  What I gather from your talk
of "states" is that you are referring to a "state-variable" type of filter.

OK, enough of the technical terms.  Basically, what you need to do is to
look up some signal-processing or control textbooks.  Alternatively,
there should be some software available to generate your filter code
given delays, etc.

Now the bad news.  This type of filter will always have overshoots
and delays at any change of velocity of the glove.  This means that
the image of the glove will play catch-up when motion starts, and
will continue moving after motion ceases, then go backwars to correct
itself.  This can be disconcerting to the user!

The predictive filters also tend to increse noise in position.  The
"jitter" in the hi-res mode (if it is just a quick jump to the wrong
position and back) can be removed at the cost of more delay-- just add
a glitch detector to the glove data output.  Alternatively, the 
derivative or highpass terms of the predictive filter can be limited,
but this might cause weird effects during fast motions.

From what I've read about flight simulators, the best way to find the
correct filter coefficients is to compute them as usual, then get into
an interactive program (using the glove to move a cursor), and test the
"feel" of the system.  Try several different sets of coefficients, then
use the intuition you'll develop to tweak the program.

Hope you can find some references that are more CS based, or sample code.
All the stuff I have assumes familiarity with Laplace transforms and other
scary stuff (typical engineering overkill).

Dave Stampe

---

From: Lance Norskog <[email-masked]>
Date: 15 Oct 91 10:53:17 PDT (Tue)

This follows the signal-processing paradigm.  An extensions
of the mouse-cursor paradigm is another possibility.

If you're willing to forgo the direct mapping concept,
prediction from the glove inputs can just "push" a 3D 
position around.  Changes in direction and speed can
be handled as special cases to avoid the overshoot/correction
effect.

Being 31 (as of this weekend) instead of 12 years old,
I don't plan to hold my hand up to the screen 
for several hours at a time anyway.
I've been testing with the sensors arranged on the
floor and letting my hand hang down.  I don't see that
I need direct mapping to use the glove in this mode,
and so the 3D-cursor-pushing scheme should work fine.

Lance Norskog