We present a convolutional network capable of inferring a 3D representation of a previously unseen object given a single image of this object. Concretely, the network can predict an RGB image and a depth map of the object as seen from an arbitrary view. Several of these depth maps fused together give a full point cloud of the object. The point cloud can in turn be transformed into a surface mesh. The network is trained on renderings of synthetic 3D models of cars and chairs. It successfully deals with objects on cluttered background and generates reasonable predictions for real images of cars.
@cite_26 trained an 'up-convolutional' network to generate an image of a chair given the chair type and a viewpoint. This method is restricted to generating images of objects from the training set or interpolating between them. Applying the method to a new test image requires re-training the network, which takes several days. While the decoder part of our network is similar to the architecture of , our network also includes an encoder part which infers the high-level representation from a given input image. Hence, at test time we can generate unseen views and depth maps of new objects by simply forward propagating an image of the object through the network. Our approach also yields more accurate predictions.
@cite_26
1893585201
We train a generative convolutional neural network which is able to generate images of objects given object type, viewpoint, and color. We train the network in a supervised manner on a dataset of rendered 3D chair models. Our experiments show that the network does not merely learn all images by heart, but rather finds a meaningful representation of a 3D chair model allowing it to assess the similarity of different chairs, interpolate between given viewpoints to generate the missing ones, or invent new chair styles by interpolating between chairs from the training set. We show that the network can be used to find correspondences between different chairs from the dataset, outperforming existing approaches on this task.