Serendipity: Finger Gesture Recognition using an Off the Shelf Smartwatch
Previous work on muscle activity sensing has leveraged specialized sensors such as electromyography and force sensitive resistors. While these sensors show great potential for detecting finger/hand gestures, they require additional hardware that adds to the cost and user discomfort. Past research has utilized sensors on commercial devices, focusing on recognizing gross hand gestures. In this work we present Serendipity, a new technique for recognizing unremarkable and fine motor finger gestures using integrated motion sensors (accelerometer and gyroscope) in off the shelf smartwatches. Our system demonstrates the potential to distinguish 5 fine motor gestures like pinching, tapping and rubbing fingers with an average f1 score of 87% Our work is the first to explore the feasibility of using solely motion sensors on everyday wearable devices to detect fine grained gestures. This promising technology can be deployed today on current smartwatches and has the potential to be applied to cross device interactions, or as a tool for research in fields involving finger and hand motion.
CHI