Learning to Infer 3D Shape Programs with Differentiable Renderer
Given everyday artifacts, such as tables and chairs, humans recognize
high-level regularities within them, such as the symmetries of a table, the
repetition of its legs, while possessing low-level priors of their geometries,
e.g., surfaces are smooth and edges are sharp. This kind of knowledge
constitutes an important part of human perceptual understanding and reasoning.
Representations of and how to reason in such knowledge, and the acquisition
thereof, are still open questions in artificial intelligence (AI) and cognitive
science. Building on the previous proposal of the \emph{3D shape programs}
representation alone with the accompanying neural generator and executor from
\citet{tian2019learning}, we propose an analytical yet differentiable executor
that is more faithful and controllable in interpreting shape programs
(particularly in extrapolation) and more sample efficient (requires no
training). These facilitate the generator's learning when ground truth programs
are not available, and should be especially useful when new shape-program
components are enrolled either by human designers or -- in the context of
library learning -- algorithms themselves. Preliminary experiments on using it
for adaptation illustrate the aforesaid advantages of the proposed module,
encouraging similar methods being explored in building machines that learn to
reason with the kind of knowledge described above, and even learn this
knowledge itself.