Shape generation and reconstruction with Reinforcement Learning. RL is successively used in many 2D domains including stroke-based painting [1]}, [2]}, [3]}, shape grammar parsing [4]}, sketch drawing [5]}, [6]}, scene synthesis [7]}, point-cloud registration [8]}. In 3D case, Sharma et al. [9]} proposed a network based on an encoder-decoder architecture and RL to produce a compact program to generate 2D and 3D input shape. The input shape in this work is obtained from primitive shapes by applying Boolean operations formed as a grammar. However, the 3D shapes in the real life could be much more complex to obtain them with such a grammar, which limits the use of this method to simple shapes. While, we train our method on ShapeNet [10]} data set.
The work proposed by Lin et al. [11]}, uses two networks to model 3D shape like human modelers. The first network Prim-Agent represents 3D shape as a combination of cuboids and the second network Mesh-Agent is used to smooth the output of the first network to obtain similar result to the input shape. This method also uses RL with imitation learning from expert demonstrations generated by heuristic approach using modified DAgger [12]} algorithm. Our work is different from their approach in many ways: 1) Our approach is based on more detailed reconstruction of the input shape using slices 2) The input to train the networks also not the same 3) We use continuous action space using DDPG and pre-training, while Lin et al. uses DQN [13]} which uses discrete action space.
