<story><title>An algorithm that recreates 3D objects from tiny 2D images</title><url>https://techcrunch.com/2017/08/23/this-algorithm-cleverly-recreates-3d-objects-from-tiny-2d-images</url></story><parent_chain></parent_chain><comment><author>edanm</author><text>This article is confusingly written. Its explanations sound silly to anyone even a little familiar with 3d, since it spends the first half of the article explaining that this &amp;quot;breakthrough&amp;quot; is &amp;quot;computationally clever and forehead-slappingly simple&amp;quot;, the breakthrough being that you can represent things as surface-models instead of voxels.&lt;p&gt;Well, surface rendering is how almost all 3d work is done already, that&amp;#x27;s definitely not a breakthrough. You can probably spend an entire career never dealing with voxels.&lt;p&gt;What this article never mentions (surprisingly) is that this paper is about neural networks. I&amp;#x27;m not an expert, but as I understand the article, voxel representations have been the standard &lt;i&gt;specifically when building neural networks&lt;/i&gt; to turn 2d images into 3d. The main idea is that you can build a network that only renders high-resolution voxels when close to the surface of the model, and renders very low-resolution voxels everywhere else (say, on the inside of the model). This means you can both represent much larger models memory-wise, but also that you&amp;#x27;re not having to run the NN computations on voxels that are not likely to change, since everything &amp;quot;inside&amp;quot; of the model you probably guessed correctly pretty quickly.&lt;p&gt;Here&amp;#x27;s the paper&amp;#x27;s abstract, much better at explaining itself than the article:&lt;p&gt;&amp;quot;Recently, Convolutional Neural Networks have shown
promising results for 3D geometry prediction. They can
make predictions from very little input data such as for example
a single color image, depth map or a partial 3D volume.
A major limitation of such approaches is that they only
predict a coarse resolution voxel grid, which does not capture
the surface of the objects well. We propose a general
framework, called hierarchical surface prediction (HSP),
which facilitates prediction of high resolution voxel grids.
The main insight is that it is sufficient to predict high resolution
voxels around the predicted surfaces. The exterior and
interior of the objects can be represented with coarse resolution
voxels. This allows us to predict significantly higher
resolution voxel grids around the surface, from which triangle
meshes can be extracted. Our approach is general
and not dependent on a specific input type. In our experiments
we show results for geometry prediction from color
images, depth images and shape completion from partial
voxel grids. Our analysis shows that the network is able to
predict the surface more accurately than a low resolution
prediction.&amp;quot;</text></comment>
<story><title>An algorithm that recreates 3D objects from tiny 2D images</title><url>https://techcrunch.com/2017/08/23/this-algorithm-cleverly-recreates-3d-objects-from-tiny-2d-images</url></story><parent_chain></parent_chain><comment><author>pasta</author><text>Well I&amp;#x27;m sorry but I think more is going on than the article describes.&lt;p&gt;For example the red pickup truck. There is no way the algorithm could create that model from that image only (depth of the trunk).&lt;p&gt;So my guess is that they use the tiny picture to search a database for similar pictures and then create a model with all that data.</text></comment>