Hand segmentation and fingertip detection play an indispensable role in hand gesture-based human-machine interaction systems. In this study, we propose a method to discriminate hand components and to locate fingertips in RGB-D images. The system consists of three main steps: hand detection using RGB images providing regions which are considered as promising areas for further processing, hand segmentation, and fingertip detection using depth image and our modified SegNet, a single lightweight architecture that can process two independent tasks at the same time. The experimental results show that our system is a promising method for hand segmentation and fingertip detection which achieves a comparable performance while model complexity is suitable for real-time applications.
This paper presents a low-cost hand gesture human-computer interaction system for remote controlling of TV and Set-Top-Box (STB). The proposed system adds only a little to the hardware cost as just a webcam is used, and can run on mainstream and even low-end TV and STB without any software and hardware upgrading. Gesture recognition has been a research focus with the popularity of depth sensing device. In this paper, we propose a new fingertip detection method based on a novel definition of fingers. This method consists of two steps. Firstly, finger bases are detected and estimated as prior information. Secondly, the finger regions and fingertips are located. In the second module, the point cloud of hand is represented as a graph to obtain all geodesic paths originated from palm center. If one path travels through a finger base, then its terminal point is defined as a finger point. The fingertips are determined within these finger points by utilizing geodesic distances. To our knowledge, such definition has never been applied in gesture recognition before and its performance surpasses the definition of geodesic maxima. Experimental results demonstrates the effectiveness of our method even when hand is not parallel to camera. Compared with state-of-the-art approach, our method shows much less error. We propose a real-time finger writing character recognition system using depth information. This system allows user to input characters by writing freely in the air with the Kinect. During the writing process, it is reasonable to assume that the finger and hand are always holding in front of torso. Firstly, we compute the depth histogram of human body and use a switch mixture Gaussian model to characterize it. Since the hand is closer to camera, a model-based threshold can segment the hand-related region out. Then, we employ an unsupervised clustering algorithm, K-means, to classify the segmented region into two parts, the finger-hand part and hand-arm part. By identifying the arm direction, we can determine the finger-hand cluster and locate the fingertip as the farthest point from the other cluster. We collected over 8000 frames writing-in-the-air sequences including two different subjects writing numbers, strokes, pattern, English and Chinese characters from two different distances. From our experiments, the proposed algorithm can provide robust and accurate fingertip detection, and achieve encouraging character recognition result. Human-Computer Interaction (HCI) exists ubiquitously in our daily lives. It is usually achieved by using a physical controller such as a mouse, keyboard or touch screen. It hinders Natural User Interface (NUI) as there is a strong barrier between the user and computer. There are various hand tracking systems available on the market, but they are complex and expensive. In this paper, we present the design and development of a robust marker-less hand finger tracking and gesture recognition system using low-cost hardware. We propose a simple but efficient method that allows robust and fast hand tracking despite complex background and motion blur. Our system is able to translate the detected hands or gestures into different functional inputs and interfaces with other applications via several methods. It enables intuitive HCI and interactive motion gaming. We also developed sample applications that can utilize the inputs from the hand tracking system. Our results show that an intuitive HCI and motion gaming system can be achieved with minimum hardware requirements.