Neural-Symbolic Methods. There has been a strong effort to make a purely neural approach to vision more interpretable [1]}, [2]}, [3]}, [4]}, [5]}, [6]}. An alternative approach to interpretability has been to deeply intertwine symbolic methods into connectionist methods. For computer vision, this is proving fruitful for de-rendering and scene decomposition [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, [13]}. Many of these approaches use shape programs, a decomposition of the scene into a set of rendering instructions. The scene-graphs we construct can be viewed as such shape programs, but in a different representation. This symbolic information is well suited for more complex tasks, such as visual question answering (VQA) [14]}, [15]} and scene manipulation [16]}. We follow many of the presented ideas in our work.
