We present a convolutional network capable of inferring a 3D representation of a previously unseen object given a single image of this object. Concretely, the network can predict an RGB image and a depth map of the object as seen from an arbitrary view. Several of these depth maps fused together give a full point cloud of the object. The point cloud can in turn be transformed into a surface mesh. The network is trained on renderings of synthetic 3D models of cars and chairs. It successfully deals with objects on cluttered background and generates reasonable predictions for real images of cars.
The multi-view perceptron @cite_23 is a network that takes a face image and a random vector as input and generates a random view of this face together with the corresponding viewpoint. In contrast, our model can generate directly the desired view without the need for random sampling. @cite_29 trained a variant of a variational autoencoder with factored hidden representations, where certain dimensions are constrained to correspond to specific factors of variations in the input data, such as viewpoint and lighting. This method is conceptually interesting and it allows to generate previously unseen views of objects, but the quality of predictions made by our network is significantly better, as we show in the experimental section.
@cite_29 @cite_23
2495387757 2131024102 1989529706 2263714001
Large pose variations remain to be a challenge that confronts real-word face detection. We propose a new cascaded Convolutional Neural Network, dubbed the name Supervised Transformer Network, to address this challenge. The first stage is a multi-task Region Proposal Network (RPN), which simultaneously predicts candidate face regions along with associated facial landmarks. The candidate regions are then warped by mapping the detected facial landmarks to their canonical positions to better normalize the face patterns. The second stage, which is a RCNN, then verifies if the warped candidate regions are valid faces or not. We conduct end-to-end learning of the cascaded network, including optimizing the canonical positions of the facial landmarks. This supervised learning of the transformations automatically selects the best scale to differentiate face non-face patterns. By combining feature maps from both stages of the network, we achieve state-of-the-art detection accuracies on several public benchmarks. For real-time performance, we run the cascaded network only on regions of interests produced from a boosting cascade face detector. Our detector runs at 30 FPS on a single CPU core for a VGA-resolution image. Various factors, such as identity, view, and illumination, are coupled in face images. Disentangling the identity and view representations is a major challenge in face recognition. Existing face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. This is different from the behavior of primate brain. Recent studies [5, 19] discovered that primate brain has a face-processing network, where view and identity are processed by different neurons. Taking into account this instinct, this paper proposes a novel deep neural net, named multi-view perceptron (MVP), which can untangle the identity and view features, and in the meanwhile infer a full spectrum of multi-view images, given a single 2D face image. The identity features of MVP achieve superior performance on the MultiPIE dataset. MVP is also capable to interpolate and predict images under viewpoints that are unobserved in the training data. Multiview representations reveal the fundamental attributes of the studied instances from different perspectives. Some common perspectives are reviewed by multiple views simultaneously, while some specific ones are reflected by individual views. That is, there are two kinds of properties embedded in the multiview data: 1) consistency and 2) complementarity. Different from most multiview learning approaches only focusing on either consistency or complementarity, this paper proposes a novel semisupervised multiview learning algorithm, called partially shared latent factor (PSLF) learning, which jointly exploits both consistent and complementary information among multiple views. In PSLF, a nonnegative matrix factorization (NMF)-based formulation is adopted to learn a compact and comprehensive partially shared latent representation, which is composed of common latent factors shared by multiple views and some specific latent factors to each view. With the learned representations of multiview data, we introduce a robust sparse regression model to predict the cluster labels of labeled data. By integrating the NMF-based model and the regression model, we obtain a unified formulation and propose a multiplicative-based alternative algorithm for optimization. In addition, PSLF can learn the weights of different views adaptively according to the reconstruction precisions of data matrices. Our experimental study indicates different multiview data that contains consistent and complementary information in different degrees. In addition, the encouraging results of the proposed algorithm are achieved in comparison with the state-of-the-art algorithms on real-world data sets. We present a convolutional network capable of generating images of a previously unseen object from arbitrary viewpoints given a single image of this object. The input to the network is a single image and the desired new viewpoint; the output is a view of the object from this desired viewpoint. The network is trained on renderings of synthetic 3D models. It learns an implicit 3D representation of the object class, which allows it to transfer shape knowledge from training instances to a new object instance. Beside the color image, the network can also generate the depth map of an object from arbitrary viewpoints. This allows us to predict 3D point clouds from a single image, which can be fused into a surface mesh. We experimented with cars and chairs. Even though the network is trained on artificial data, it generalizes well to objects in natural images without any modifications.