Robust recognition of hand gestures in real-world applications is still an unaccomplished goal due to many remaining challenges, such as cluttered backgrounds and unconstrained environmental factors. In most existing methods in this field, hand segmentation is a critical step to reduce redundant information in the scene background, in preparation for the hand gesture recognition stage. Therefore, we propose a new two-stage convolutional neural network (CNN) architecture, called HGR-Net, where the first stage performs accurate pixel-level semantic segmentation to determine the hand regions, and the second stage identifies the hand gesture. The segmentation stage architecture is based on the combination of fully convolutional residual network and atrous spatial pyramid pooling. Although the segmentation sub-network is trained without depth information, it is significantly robust against challenges such as illumination variations and complex backgrounds. The recognition stage deploys a two-stream CNN which fuses the information from the RGB and segmented images by combining their deep representations in a new fully connected layer before classification. Extensive experiments on public hand gesture datasets show that our deep architecture achieves almost as good as state-of-the-art performance in segmentation and recognition of static hand gestures, at a fraction of training time, run time, and model size.
Natural communication between humans involves hand gestures, which has an impact on research in human-robot interaction. In a real-world scenario, understanding human gestures by a robot is hard due to several challenges like hand segmentation. To recognize hand postures this paper proposes a novel convolutional implementation. The model is able to recognize hand postures recorded by a robot camera in real-time, in a real-world application scenario. The proposed model was also evaluated with a benchmark database and showed better results than the ones reported in the benchmark paper.