Robust recognition of hand gestures in real-world applications is still an unaccomplished goal due to many remaining challenges, such as cluttered backgrounds and unconstrained environmental factors. In most existing methods in this field, hand segmentation is a critical step to reduce redundant information in the scene background, in preparation for the hand gesture recognition stage. Therefore, we propose a new two-stage convolutional neural network (CNN) architecture, called HGR-Net, where the first stage performs accurate pixel-level semantic segmentation to determine the hand regions, and the second stage identifies the hand gesture. The segmentation stage architecture is based on the combination of fully convolutional residual network and atrous spatial pyramid pooling. Although the segmentation sub-network is trained without depth information, it is significantly robust against challenges such as illumination variations and complex backgrounds. The recognition stage deploys a two-stream CNN which fuses the information from the RGB and segmented images by combining their deep representations in a new fully connected layer before classification. Extensive experiments on public hand gesture datasets show that our deep architecture achieves almost as good as state-of-the-art performance in segmentation and recognition of static hand gestures, at a fraction of training time, run time, and model size.
This paper introduces a new method for skin regions segmentation which consists in spatial analysis of skin probability maps obtained using pixel-wise detectors. There are a number of methods which use various techniques of skin color modeling to classify every individual pixel or transform input color images into skin probability maps, but their performance is limited due to high variance and low specificity of the skin color. Detection precision can be enhanced based on spatial analysis of skin pixels, however this direction has been little explored so far. Our contribution lies in using the distance transform for propagating the “skinness” across the image in a combined domain of luminance, hue and skin probability. In the paper we explain theoretical advantages of the proposed method over alternative skin detectors that also perform spatial analysis. Finally, we present results of an extensive experimental study which clearly indicate high competitiveness of the proposed method and its relevance to gesture recognition. Problem statement: Skin color detection is used as a preliminary step in numerous computer vision applications like face detection, nudity recognition, hand gesture detection and person identification. In this study we present a pixel based skin color classification approach, for detecting skin pixels and non skin pixels in color images, using a novel neural network symmetric classifier. The neural classifiers used in the literature either uses a symmetric model with single neuron in the output layer or uses two separate neural networks (asymmetric model) for each of the skin and non-skin classes. The novelty of our approach is that it has two output layer neurons; one each for skin and non-skin class, instead of using two separate classifiers. Thus by using a single neural network classifier we have improved the separability between these two classes, eliminating additional time complexity that is needed in asymmetric classifier. Approach: Skin samples from web images of people from different ethnic groups were collected and used for training. Ground truth skin segmented images were obtained by using semiautomatic skin segmentation tool developed by the authors. The ground truth database of skin segmented images, thus obtained was used to evaluate the performance of our NN based classifier. Results: With proper selection of optimum classification threshold that varies from image to image the classifier gave the detection rate of more than 90 with 7 false positives on an average, Conclusion Recommendations: It is observed that the neural network is capable of detecting skin in complex lighting and background environments. The classifier has the ability to classify the skin pixels belonging to people from different ethnic groups even when they are present simultaneously in an image. The proper choice of optimum classification threshold that varies from image to image is an issue here. Automatic computation of this optimum threshold for each image is desired in practical skin detection applications. This issue can be taken up as a future study, which will enable us to perform fully automatic skin segmentation with reduced false positives. The skin probability maps are refined using discriminative features (DSPFs).Spatial analysis of skin pixels is more successful when performed in DSPF skin maps.The textural features are extracted from skin maps rather than from the grayscale. In this paper we propose a new method for skin detection in color images which consists in spatial analysis using the introduced texture-based discriminative skin-presence features. Color-based skin detection has been widely explored and many skin color modeling techniques were developed so far. However, efficacy of the pixel-wise classification is limited due to an overlap between the skin and non-skin pixels reported in many color spaces. To increase the discriminating power of the skin classification schemes, textural and spatial features are often exploited for skin modeling. Our contribution lies in using the proposed discriminative feature space as a domain for spatial analysis of skin pixels. Contrary to existing approaches, we extract the textural features from the skin probability maps rather than from the luminance channel. Presented experimental study confirms that the proposed method outperforms alternative skin detection techniques, which also involve analysis of textural and spatial features. Skin segmentation is the cornerstone of many applications such as gesture recognition, face detection, and objectionable image filtering. In this paper, we attempt to address the skin segmentation problem for gesture recognition. Initially, given a gesture video sequence, a generic skin model is applied to the first couple of frames to automatically collect the training data. Then, an SVM classifier based on active learning is used to identify the skin pixels. Finally, the results are improved by incorporating region segmentation. The proposed algorithm is fully automatic and adaptive to different signers. We have tested our approach on the ECHO database. Comparing with other existing algorithms, our method could achieve better performance.  