As the virtual world seeps out into reality, we also find ourselves getting more immersed in the digital domain. Gesture controls continue to be at the forefront of user interfaces (UI), and Israel-based PointGrab believes its software-based approach may be the key to bringing ubiquitous gesture control to the masses.
“Gesture is not a gimmick. It’s an input mechanism,” PointGrab’s VP of marketing and product, Assaf Gad says. “And the best solution will be the one that is unified.” Gesture control is nothing new, but it’s something that no one has been able to get right – yet. Remember the excitement around Nintendo Wii or even the controller-less Kinect? How many Xboxers use the Kinect to navigate the system today?
Microsoft recently unveiled the new Xbox One with an updated Kinect that the company hopes users will adapt beyond their gaming console and compete with PC gesture interfaces like Leap Motion. PointGrab differentiates itself from those interfaces in that it’s purely software-based, so it works with the camera already in your device. Any device that comes with a camera (or can be attached to a camera) can become your virtual marionette.
The software underneath PointGrab’s hood registers individual body movements – right down to the orientation of individual fingers – and translates them as command prompts. For example, placing a finger over pursed lips (the universal signal for shhhhh) can prompt a device to mute, and swiping through the air can register the same as swiping a touchscreen. It can even register the distance and depth of your motions through 3D space.
PointGrab’s camera-based technology has already been utilised by Samsung in its latest generation of smart TVs, and in laptops by Acer and Fujitsu. However, the company sees the biggest opportunities in the mobile field. Gad says that the software can register individual hand motions on an iPhone 5’s front camera up to nearly seven metres away, and slightly further on the beefier back camera – further than either Leap Motion or Kinect.
The company is hoping the tech will be embraced by developers and find its way into apps across the smartphone spectrum. While the company has little interest in creating apps themselves, they did release the selfie-aide app CamMe for free on iOS as a proof-of-concept. CamMe allows users to de-forearm their self portraits by controlling their phone’s camera from afar using only hand gestures. The app has been downloaded more than 200,000 times in three months.
The bigger picture
Touchscreens didn’t make the keyboard or touchpad obsolete. Rather, they complemented them. Likewise, as gesture control develops, it won’t replace touchscreens either. “The complete solution will be a combination of touchscreens with touch and also voice control,” says Gad. For example, a gesture UI might register that a hand is approaching the screen and pop up a hidden control menu or make certain buttons bigger.
Gad is quick to show off how the new generation of Intel Snapdragon processors will allow gadgets to read hand gestures caught on a back camera and process them for information on the front screen, adding a new dimension to augmented reality. The next generation of technology will feature various UIs working in concert to provide new dimensions of control.
Before Apple introduced the iPhone in 2007, there were already touchscreen UIs aimed at consumers (going as far back as far as the early 1980s), but it took the right mix of hardware and software to make the technology viable for the masses. In the five-plus years since the iPhone exploded the way we interface with the digital world, manufacturers have attempted various takes at alternative touch-free UIs, but they have yet to take off with mass adaptation.
PointGrab is not a flawless interface, but it shows as much promise as sensor-based solutions like Kinect. And if the technology can become more popular with app developers, it will unleash a lot of new ideas and help the industry think further outside the box – literally.
by Evan Dashevsky, Techhive