en by fabric or another obstructions—and other technologies like radar that require both custom sensor hardware and greater computing power, says senior author Shyam Gollakota, assistant professor of computer science and engineering.
“Acoustic signals are great—because sound waves travel much slower than the radio waves used in radar, you don’t need as much processing bandwidth so everything is simpler. And from a cost perspective, almost every device has a speaker and microphones so you can achieve this without any special hardware.”
But sonar echoes are weak and typically not accurate enough to track finger motion at a high resolution. Errors of a few centimeters make it impossible to differentiate between writing individual letters or subtle hand gestures.
Researchers employed a type of signal typically used in wireless communication—called Orthogonal Frequency Division Multiplexing—and demonstrated that it can be used to achieve high-resolution finger tracking using sound. Their algorithms leverage the properties of OFDM signals to track phase changes in the echoes and correct for any errors in the finger location to achieve sub-centimeter finger tracking.
To test the approach, the researchers created a FingerIO prototype app for Android devices and downloaded it to an off-the-shelf Samsung Galaxy S4 smartphone and a smartwatch customized with two microphones, which are needed to track finger motion in two dimensions. Today’s smartwatches typically only have one, which can be used to track a finger in one dimension.
Testers drew shapes such as stars, squiggles, or figure 8s on a touchpad next to a smartphone or smartwatch running FingerIO. They then compared the touchpad tracings to the shapes created by FingerIO’s tracking. The average difference between the drawings and the FingerIO tracings was 0.8 centimeters for the smartphone and 1.2 centimeters for the smartwatch.
“Given that your finger is already a centimeter thick, that’s sufficient to accurately interact with the devices,” says coauthor and electrical engineering graduate student Vikram Iyer.
Next steps for the research team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time, and extending its tracking abilities into three dimensions by adding additional microphones to the devices.
The National Science Foundation and Google funded the work.
Source: University of Washington