|
Someday,
your computer may respond to a hand gesture
By
Anne Eisenberg
New York Times
Without
a doubt, there are uncounted millions of people who have
made rude gestures to their computer screens in moments
of desperation. And the computers never respond, which doesn't
help matters.
That may change. One day soon, computers may understand
and respond to many common human gestures, from simple pointing
and grasping to the complex motions used to conduct an orchestra.
In laboratories in the United States and abroad, researchers
are experimenting with computer systems that use tiny cameras,
fast processors and smart software to identify and respond
quickly and accurately to a range of hand gestures natural
to humans.
Such systems promise to simplify the way people and computers
interact. For instance, people may one day walk through
their offices, living rooms or backyards and point out the
chores that need to be done to video-based interfaces that
will pass on the information to computers that can process
motions as well as words.
"Hand gestures - and eventually full body and facial information
- are the natural complement to speech communication with
a computer," said Dr. Jakub Segen, a researcher who has
worked in this field for 10 years. Segen is a member of
the technical staff at Lucent Technologies' Bell Labs in
Murray Hill, N.J.
He and Dr. Senthil Kumar reported their work on gesture
recognition technology, including tests of its accuracy,
in the July issue of Communications of the ACM, a journal
of the Association for Computing Machinery.
Segen said he had been teaching the computer to recognize
and respond accurately to a number of gestures made by the
hand. His current project is aimed at making video conferences
work better.
"Video conferences don't really cut it in their present
form," Segen said. "That's why I end up flying to California
for meetings. So this seemed like a promising area for improvement."
People attend Segen's virtual meetings by logging onto computers
equipped with cameras that observe them and with special
software that extracts important elements in their gestures.
The computers translate participants' gestures into the
movements of animated figures he calls varionettes, articulated,
marionettelike characters that imitate on the computer screen
just what the people at their desks are doing.
"If you raise your hand to make a point," Segen said, "the
figure raises its hand, too."
He has also demonstrated his technology in a livelier spot
than a corporate meeting: The conductor's podium. Last year
he conducted a digital orchestra - as cameras tracked the
position of Segen's hand, which was holding a baton, his
software translated that motion into tempos for the digitally
sampled instruments.
Such applications of gesture technology are not as far in
the future as they might seem.
Dr. Frederick Bianchi, a professor at Worcester Polytechnic
Institute and one of the founders of the Virtual Orchestra,
the computerized orchestra simulation that Segen conducted,
has also conducted the Virtual Orchestra by using Segen's
system.
"We plan to have this technology on Broadway and on the
road in the next few years," Bianchi said. Without Segen's
system, someone keeps the beat for the Virtual Orchestra
by tapping out a rhythm on a computer keyboard. With the
new software system, Bianchi said, a conductor will be able
to use the baton to set the dynamics and speed of the orchestra.
The system is set up for the one-handed conductor. What
about the conductor who uses two hands, one to keep the
pace and the other to signal crescendos or diminuendos or
to tell players when to come in?
Richard Campbell, a colleague of Bianchi's on the Virtual
Orchestra project and an electrical engineering professor
at Worcester Polytechnic Institute, has supervised a graduate
student's dissertation on a gesture controller that can
understand two hands.
"One performance was the overture to Tchaikovsky's 'Nutcracker'
with a full orchestral score," he said. "The tempo tracked
beautifully."
While baton movements interest Campbell, Dr. Charles Cohen,
vice president of research and development at the Cybernet
Systems Corp. in Ann Arbor, Mich., has been concentrating
on circles - specifically, repeated circular motions. The
work is being done for NASA and the Army.
The Army is using Cohen's programs to train scouts in the
secret hand signals they must transmit stealthily but accurately
to troops. "These are exactly the gestures our system can
recognize," he said.
The system promises to train scouts at least as well as
the most remorseless sergeant, delivering relentless feedback
about errors.
For the National Aeronautics and Space Administration, Cohen
is developing kiosk screens that respond to gesture rather
than touch. Such screens would, for instance, let an observer
wave a hand in a circle to rotate a display of the space
station.
Gesture-driven screens sidestep the problems of wear and
tear that plague standard touch screens, problems like broken
joysticks and dirt on the screens. They also make life easier
for users.
"We want to create kiosks where people can use their hands,
not a keyboard or mouse," said Stacey Morrison, the deputy
chief information officer for the Space and Life Sciences
Directorate at the Johnson Space Center in Houston. "It's
simpler if people can just point their hand to go left or
right."
Segen sees enormous potential for gesture technology. "The
ultimate wireless interface will be optical," he said. And
optical devices can be very precise, he said, pointing out
that cameras can read license plates from space.
So will computers someday be able to understand the kind
of unfriendly gestures that show that a user is reaching
a point of uncontrollable rage? The researchers have not
tackled that problem yet. But computers that evolve in that
direction could be the ones with the best chances of survival.
|
