Computer scientists of the University of Washington have built a working prototype of a gesture control interface named AllSee. AllSee can operate without a battery and, unlike most gesture recognition systems, does not rely on a camera to record gestural input. Instead the device measures the wireless signals that are all around us like TV transmissions and interprets specific changes in that continuous flow as input.

The two main components of the AllSee are an ultra-low-power receiver and a microcontroller. Gestures cause changes in the amplitude of the ambient wireless signals. Hand gestures made near the receiver are detected by the device and treated as an input value. The AllSee can distinguish the unique amplitude variations of eight different hand gestures.

The findings were published in the paper Bringing Gesture Recognition To All Devices. The technology will be demonstrated at the Usenix Symposium on Networked Systems design and Implementation in Seattle, in early April.



Always-on

To demonstrate the possible applications of the interface the UW team integrated it with several devices. They mounted it on a Samsung Galaxy Nexus smartphone to control a music app and successfully stopped the music and skipped to the next song with hand gestures. Because there's no need for line of sight the phone can be controlled while it's tucked away in a pocket or bag. Since AllSee hardly consumes any power gesture control can be always-on. That's a significant step forward from the air gesture feature currently available on the Samsung Galaxy S4. Its reliance on a camera requires line of sight and drains the battery quickly.

Internet of Things

Gesture interfaces based on ambient signals aren't new. The novel contribution of the academics is the system's ability to operate on battery-free devices. The AllSee can harvest the power it needs from RF signals it picks up. To accomplish that, the team developed a set of algorithms that require a minimum amount of computational power to detect and classify the amplitude variations of wireless signals.

Its ultra-low-power requirements makes AllSee suitable for integration with power-constrained Internet of Things devices such as infrastructural sensors. Also, many IoT devices will be too small to incorporate graphical or touch user interfaces. In such cases gesture recognition offers a solution.

For now AllSee only works with TV and RFID transmissions but the scientists are confident they can make the device work with other kinds of signals like WiFi and cellular.

Image: University of Washington