Robust road segmentation is a key challenge in self-driving research. Though many image-based methods have been studied and high performances in dataset evaluations have been reported, developing robust and reliable road segmentation is still a major challenge. Data fusion across different sensors to improve the performance of road segmentation is widely considered an important and irreplaceable solution. In this paper, we propose a novel structure to fuse image and LiDAR point cloud in an end-to-end semantic segmentation network, in which the fusion is performed at decoder stage instead of at, more commonly, encoder stage. During fusion, we improve the multi-scale LiDAR map generation to increase the precision of the multi-scale LiDAR map by introducing pyramid projection method. Additionally, we adapted the multi-path refinement network with our fusion strategy and improve the road prediction compared with transpose convolution with skip layers. Our approach has been tested on KITTI ROAD dataset and has competitive performance.
Computer vision and pattern recognition research cover a lot of fields, and the a an important topic is feature extraction and analysis @cite_26 @cite_4 @cite_38 @cite_42 @cite_0 @cite_17 @cite_27 . In the past decade, deep learning with CNN becomes a very important feature extractor for classification and segmentation problems @cite_14 @cite_20 @cite_18 . Since road segmentation is a semantic segmentation task, the following subsection is the traditional ideas on road segmentation. Then the subsequent subsection reviews deep neural network based semantic segmentation as our work are inspired by many recent progress in new architecture in CNN classifier and semantic segmentation methods. Finally, data fusion methods in previous works is listed.
@cite_38 @cite_18 @cite_26 @cite_4 @cite_14 @cite_42 @cite_0 @cite_27 @cite_20 @cite_17
2464929676 2524029660 2779475425 1992738091 2552684688 2095176685 2402601457 2791528017 2805903690 2762668439
In this paper, a novel group sparse canonical correlation analysis (GSCCA) method is proposed for simultaneous electroencephalogram (EEG) channel selection and emotion recognition. GSCCA is a group sparse extension of the conventional CCA method to model the linear correlationship between emotional EEG class label vectors and the corresponding EEG feature vectors. In contrast to conventional CCA method or previous GSCCA methods, a major advantage of our GSCCA method is the ability of handling the group feature selection problem from raw EEG features, which makes it very suitable for simultaneously coping with both EEG emotion recognition and automatic channel selection issues where each EEG channel is associated with a group of raw EEG features. To deal with EEG emotion recognition problem, we adopt the popularly used frequency feature to describe the EEG signal by dividing the full EEG frequency band into five parts, i.e., @math , @math , @math , @math , and @math frequency bands, and then extract the frequency band features from each band for GSCCA model learning and emotion recognition. Finally, we conduct extensive experiments on EEG-based emotion recognition based on the SJTU emotion EEG dataset and experimental results demonstrate that the proposed GSCCA method would outperform the state-of-the-art EEG-based emotion recognition approaches. There have been increasing research interests in automatically constructing image dataset by collecting images from the Internet. However, existing methods tend to have a weak domain adaptation ability, known as the "dataset bias problem". To address this issue, in this work, we propose a novel image dataset construction framework which can generalize well to unseen target domains. In specific, the given queries are first expanded by searching in the Google Books Ngrams Corpora (GBNC) to obtain a richer semantic description, from which the noisy query expansions are then filtered out. By treating each expansion as a "bag" and the retrieved images therein as "instances", we formulate image filtering as a multi-instance learning (MIL) problem with constrained positive bags. By this approach, images from different data distributions will be kept while with noisy images filtered out. Comprehensive experiments on two challenging tasks demonstrate the effectiveness of our proposed approach. In the traditional graph embedding framework, the graph is usually built by k-NN or r-ball. Since it is difficult to manually set the parameters k and r in the high-dimensional space, sparse representation-based methods are usually introduced to automatically build the graphs. In recent years, nuclear norm-based matrix regression (NMR) has been proposed for face recognition using the low rank structural information (i.e., the image matrix-based error model). Inspired by NMR, we give a NMR-based projections (NMRP) method for feature extraction and recognition. The experiments on FERET and extended Yale B face databases show that NMR can be used to build the graph while NMRP is an effective feature extraction method. In graph embedding based methods, we usually need to manually choose the nearest neighbors and then compute the edge weights using the nearest neighbors via L2 norm (e.g. LLE). It is difficult and unstable to manually choose the nearest neighbors in high dimensional space. So how to automatically construct a graph is very important. In this paper, first, we give a L2-graph like L1-graph. L2-graph calculates the edge weights using the total samples, avoiding manually choosing the nearest neighbors; second, a L2-graph based feature extraction method is presented, called collaborative representation based projections (CRP). Like SPP, CRP aims to preserve the collaborative representation based reconstruction relationship of data. CRP utilizes a L2 norm graph to characterize the local compactness information. CRP maximizes the ratio between the total separability information and the local compactness information to seek the optimal projection matrix. CRP is much faster than SPP since CRP calculates the objective function with L2 norm while SPP calculate the objective function with L1 norm. Experimental results on FERET, AR, Yale face databases and the PolyU finger-knuckle-print database demonstrate that CRP works well in feature extraction and leads to a good recognition performance. We give a L2 norm graph based on collaborative representation.We propose a collaborative representation based projections (CRP) for feature extraction.CRP is a Rayleigh quotient form and can be calculated via generalized eigenvalue decomposition. The goal of this work is to automatically collect a large number of highly relevant natural images from Internet for given queries. A novel automatic image dataset construction framework is proposed by employing multiple query expansions. In specific, the given queries are first expanded by searching in the Google Books Ngrams Corpora to obtain a richer semantic descriptions, from which the visually non-salient and less relevant expansions are then filtered. After retrieving images from the Internet with filtered expansions, we further filter noisy images by clustering and progressively Convolutional Neural Networks (CNN) based methods. To evaluate the performance of our proposed method for image dataset construction, we build an image dataset with 10 categories. We then run object detections on our image dataset with three other image datasets which were constructed by weak supervised, web supervised and full supervised learning, the experimental results indicated the effectiveness of our method is superior to weak supervised and web supervised state-of-the-art methods. In addition, we do a cross-dataset classification to evaluate the performance of our dataset with two publically available manual labelled dataset STL-10 and CIFAR-10. Sparse representation based classification (SRC) has received much attention in computer vision and pattern recognition. SRC codes a testing sample by sparse linear combination of all the training samples and classifies the testing sample into the class with the minimum representation error. Recently, Zhang analyzes the working mechanism of SRC and points out that it is the collaborative representation but not the L1-norm sparsity that makes SRC powerful. Based on the analysis, they propose a very simple and much more efficient classification scheme, called collaborative representation based classification with regularized least square (CRC_RLS). CRC_RLS is a linear method in nature. Here we propose a kernel collaborative representation based classification with regularized least square (Kernel CRC_RLS, KCRC_RLS) by implicitly mapping the sample into high-dimensional space via kernel tricks. Our approach is highly motivated by the kernel methods which can capture the nonlinear similarity among samples and have been successfully applied in pattern recognition and machine learning. The experimental results on the CENPAMI handwritten digital database, ETH80 database, FERET face database, ORL database, AR face database, demonstrate that Kernel CRC_RLS is effective in classification, leading to promising performance. Recent successes in visual recognition can be primarily attributed to feature representation, learning algorithms, and the ever-increasing size of labeled training data. Extensive research has been devoted to the first two, but much less attention has been paid to the third. Due to the high cost of manual data labeling, the size of recent efforts such as ImageNet is still relatively small in respect to daily applications. In this work, we mainly focus on how to automatically generate identifying image data for a given visual concept on a vast scale. With the generated image data, we can train a robust recognition model for the given concept. We evaluate the proposed webly supervised approach on the benchmark Pascal VOC 2007 dataset and the results demonstrates the superiority of our method over many other state-of-the-art methods in image data collection. Abstract This paper introduces a novel dimensionality reduction algorithm, called collaborative representation based local discriminant projection (CRLDP), for feature extraction. CRLDP utilizes collaborative representation relationships among samples to construct adjacency graphs. Different from most graph-based algorithms which manually construct the adjacency graphs, CRLDP is able to automatically construct the graphs and avoid manually choosing nearest neighbors. In CRLDP, two graphs (the within-class graph and the between-class graph) are constructed. Based on the two constructed graphs, the within-class scatter and the between-class scatter are computed to characterize the compactness and separability of samples, respectively. Then CRLDP seeks to find an optimal projection matrix to maximize the ratio of the between-class scatter to the within-class scatter. Experimental results on ORL, AR and CMU PIE face databases validate the superiority of CRLDP over other state-of-the-art algorithms. Abstract Studies show that refining real-world categories into semantic subcategories contributes to better image modeling and classification. Previous image sub-categorization work relying on labeled images and WordNet’s hierarchy is labor-intensive. To tackle this problem, in this work, we extract textual and visual features to automatically select and subsequently classify web images into semantic rich categories. The following two major challenges are well studied: (1) noise in the labels of subcategories derived from the general corpus; (2) noise in the labels of images retrieved from the web. Specifically, we first obtain the semantic refinement subcategories from the text perspective and remove the noise by using the relevance-based approach. To suppress the search error induced noisy images, we then formulate image selection and classifier learning as a multi-instance learning problem and propose to solve the employed problem by the cutting-plane algorithm. The experiments show significant performance gains by using the generated data of our approach on image categorization tasks. The proposed approach also consistently outperforms existing weakly supervised and web-supervised approaches. Due to the variations of viewpoint, pose, and illumination, a given individual may appear considerably different across different camera views. Tracking individuals across camera networks with no overlapping fields is still a challenging problem. Previous works mainly focus on feature representation and metric learning individually which tend to have a suboptimal solution. To address this issue, in this work, we propose a novel framework to do the feature representation learning and metric learning jointly. Different from previous works, we represent the pairs of pedestrian images as new resized input and use linear Support Vector Machine to replace softmax activation function for similarity learning. Particularly, dropout and data augmentation techniques are also employed in this model to prevent the network from overfitting. Extensive experiments on two publically available datasets VIPeR and CUHK01 demonstrate the effectiveness of our proposed approach.