With the rapid development of augmented reality (AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is one of the important interactive methods for HCI applications such as in the Google Soli and Microsoft Kinect projects. However, the progress in this research is slow due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality micro-gesture images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. The principle of the holoscopic 3D camera is based on the fly viewing system to see the objects. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of micro-gesture recognition, HoMG has a video subset with 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in this area.
The Kinect and RGB-D camera are very popular in recent years due to the benefits of Kinect and RGB-D camera that have low cost and wide availability as a sensor @cite_18 to capture gestures. However, RGB-D camera suffers from the underside artefacts such as the edge inaccuracies, low object remission and holes owning @cite_5 . The Kinect sensor offers the information of the depth measurement and create coordinates of the 3D objects. Although the abundant development toolkits can support the human body recognition, the weakness is its lacking ability to capture the flexible and robust mechanism to perform high-level gesture @cite_20 .
@cite_5 @cite_18 @cite_20
2156222070 2743552164 1588788171 2151047074
Over the last decade, the availability of public image repositories and recognition benchmarks has enabled rapid progress in visual object category and instance detection. Today we are witnessing the birth of a new generation of sensing technologies capable of providing high quality synchronized videos of both color and depth, the RGB-D (Kinect-style) camera. With its advanced sensing capabilities and the potential for mass adoption, this technology represents an opportunity to dramatically increase robotic object recognition, manipulation, navigation, and interaction capabilities. In this paper, we introduce a large-scale, hierarchical multi-view object dataset collected using an RGB-D camera. The dataset contains 300 objects organized into 51 categories and has been made publicly available to the research community so as to enable rapid progress based on this promising technology. This paper describes the dataset collection procedure and introduces techniques for RGB-D based object recognition and detection, demonstrating that combining color and depth information substantially improves quality of results. Commercial RGB-D cameras provide the possibility of fast, accurate, and cost-effective 3-D scanning solution in a single package. These economical depth cameras provide several advantages over conventional depth sensors, such as sonars and lidars, in specific usage scenarios. In this paper, we analyze the performance of Kinect v2 time-of-flight camera while operating fully submerged underwater in a customized waterproof housing. Camera calibration has been performed for Kinect’s RGB and NIR cameras, and the effect of calibration on the generated 3-D mesh is discussed in detail. To overcome the effect of refraction of light due to the sensor housing and water, we propose a time-of-flight correction method and a fast, accurate and intuitive refraction correction method that can be applied to the acquired depth images, during 3-D mesh generation. Experimental results show that the Kinect v2 can acquire point cloud data up to 650 mm. The reconstruction results have been analyzed qualitatively and quantitatively, and confirm that the 3-D reconstruction of submerged objects at small distances is possible without the requirement of any external NIR light source. The proposed algorithms successfully generated 3-D mesh with a mean error of ±6 mm at a frame rate of nearly 10 fps. We acquired a large data set of RGB, IR and depth data from a submerged Kinect v2. The data set covers a large variety of objects scanned underwater and is publicly available for further use, along with the Kinect waterproof housing design and correction filter codes. The research is aimed toward small-scale research activities and economical solution for 3-D scanning underwater. Applications such as coral reef mapping and underwater SLAM in shallow waters for ROV’s can be a viable application area that can benefit from results achieved. Recently, the low-cost Microsoft Kinect sensor, which can capture real-time high-resolution RGB and depth visual information, has attracted increasing attentions for a wide range of applications in computer vision. Existing techniques extract hand-tuned features from the RGB and the depth data separately and heuristically fuse them, which would not fully exploit the complementarity of both data sources. In this paper, we introduce an adaptive learning methodology to automatically extract (holistic) spatio-temporal features, simultaneously fusing the RGB and depth information, from RGB-D video data for visual recognition tasks. We address this as an optimization problem using our proposed restricted graph-based genetic programming (RGGP) approach, in which a group of primitive 3D operators are first randomly assembled as graph-based combinations and then evolved generation by generation by evaluating on a set of RGB-D video samples. Finally the best-performed combination is selected as the (near-)optimal representation for a pre-defined task. The proposed method is systematically evaluated on a new hand gesture dataset, SKIG, that we collected ourselves and the public MSR Daily Activity 3D dataset, respectively. Extensive experimental results show that our approach leads to significant advantages compared with state-of-the-art hand-crafted and machine-learned features. With the invention of the low-cost Microsoft Kinect sensor, high-resolution depth and visual (RGB) sensing has become available for widespread use. The complementary nature of the depth and visual information provided by the Kinect sensor opens up new opportunities to solve fundamental problems in computer vision. This paper presents a comprehensive review of recent Kinect-based computer vision algorithms and applications. The reviewed approaches are classified according to the type of vision problems that can be addressed or enhanced by means of the Kinect sensor. The covered topics include preprocessing, object tracking and recognition, human activity analysis, hand gesture analysis, and indoor 3-D mapping. For each category of methods, we outline their main algorithmic contributions and summarize their advantages differences compared to their RGB counterparts. Finally, we give an overview of the challenges in this field and future research trends. This paper is expected to serve as a tutorial and source of references for Kinect-based computer vision researchers.