Over the last years, the field of visual sythesis has witnessed astonishing progress, driven in particular by the emergence of Generative Adversarial Networks [1]}. Tremendous strides have been made in a wide variety of tasks, from image [2]} to video generation [3]}, to super-resolution [4]}, style transfer [5]} and translation [6]}. Meanwhile, a growing body of literature explored object discovery and sequential scene decomposition, albeit in the context of variational inference [7]}, [8]}, [9]}, [10]}. DRAW [11]} uses recurrent attention to encode and decode MNIST digits, while AIR [12]}, MONet [13]} and IODINE [14]} reconstruct simple simulated scenes by iteratively factorizing them into semantically meaningful segments. We draw inspiration from this pioneering line of works and marry the merits of its strong structure with the scalability and fidelity obtained by GANs, yielding a compositional model that successfully creates intricate real-world scenes, while offering a more transparent and controllable synthesis along the way, conceptually meant to better resemble the causal generative process and serve as a more natural emulation of how a human might approach the creative task.
