Extracting key information from documents, such as receipts or invoices, and preserving the interested texts to structured data is crucial in the document-intensive streamline processes of office automation in areas that includes but not limited to accounting, financial, and taxation areas. To avoid designing expert rules for each specific type of document, some published works attempt to tackle the problem by learning a model to explore the semantic context in text sequences based on the Named Entity Recognition (NER) method in the NLP field. In this paper, we propose to harness the effective information from both semantic meaning and spatial distribution of texts in documents. Specifically, our proposed model, Convolutional Universal Text Information Extractor (CUTIE), applies convolutional neural networks on gridded texts where texts are embedded as features with semantical connotations. We further explore the effect of employing different structures of convolutional neural network and propose a fast and portable structure. We demonstrate the effectiveness of the proposed method on a dataset with up to @math labelled receipts, without any pre-training or post-processing, achieving state of the art performance that is much higher than BERT but with only @math parameters and without requiring the @math M word dataset for pre-training. Experimental results also demonstrate that the CUTIE being able to achieve state of the art performance with much smaller amount of training data.
CloudScan is a work attempting to extract key information with learning based models @cite_3 . Firstly, N-grams features are formed by connecting expert designed rules calculated results on texts of each document line. Then, the features are feed to train a RNN-based or a logistic regression based classifier for key information extraction. Certain post-processing are added to further enhance extraction results. However, the line-based feature extraction method can not achieve its best performance if document texts are not perfectly aligned in line. Moreover, the RNN-based classifier, bi-directional LSTM model in CloudScan, has limited ability to learn relationship among distant words.
@cite_3
1533693043 2963336905 2132761823 2001613026
Automatic cloud extraction from satellite imagery is an important task for many applications in remote sensing. Humans can easily identify various clouds from satellite images based on the visual features of cloud. In this study, a method of automatic cloud detection is proposed based on object classification of image features. An image is first segmented into superpixels so that the descriptor of each superpixel can be computed to form a feature vector for classification. The support vector machine algorithm is then applied to discriminate cloud and noncloud regions. Thereafter, the GrabCut algorithm is used to extract more accurate cloud regions. The key of the method is to deal with the highly varying patterns of clouds. The bag-of-words (BOW) model is used to construct the compact feature vectors from densely extracted local features, such as dense scale-invariant feature transform (SIFT). The algorithm is tested using 101 RapidEye and 86 Landsat images with many cloud patterns. These images achieve 89.2 of precision, 87.8 of recall for RapidEye, 85.8 of precision, and 83.9 of recall for Landsat. The experiments show that the method is insensitive to the number of codewords in the codebook construction of the BOW. Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately, which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN)for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object's class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmentation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods. In this paper we address the topic of feature extraction in 3D point cloud data for object recognition and pose identification. We present a novel interest keypoint extraction method that operates on range images generated from arbitrary 3D point clouds, which explicitly considers the borders of the objects identified by transitions from foreground to background. We furthermore present a feature descriptor that takes the same information into account. We have implemented our approach and present rigorous experiments in which we analyze the individual components with respect to their repeatability and matching capabilities and evaluate the usefulness for point feature based object detection methods. This paper reports a novel mutual information (MI) based algorithm for automatic registration of unstructured 3D point clouds comprised of co-registered 3D lidar and camera imagery. The proposed method provides a robust and principled framework for fusing the complementary information obtained from these two different sensing modalities. High-dimensional features are extracted from a training set of textured point clouds (scans) and hierarchical k-means clustering is used to quantize these features into a set of codewords. Using this codebook, any new scan can be represented as a collection of codewords. Under the correct rigid-body transformation aligning two overlapping scans, the MI between the codewords present in the scans is maximized. We apply a James-Stein-type shrinkage estimator to estimate the true MI from the marginal and joint histograms of the codewords extracted from the scans. Experimental results using scans obtained by a vehicle equipped with a 3D laser scanner and an omnidirectional camera are used to validate the robustness of the proposed algorithm over a wide range of initial conditions. We also show that the proposed method works well with 3D data alone.