We present a novel approach to train a natural media painting using reinforcement learning. Given a reference image,  our formulation is based on stroke-based rendering that  imitates human drawing and can be learned from scratch without supervision. Our painting agent computes a sequence of actions that represent the primitive painting strokes. In order to ensure that the generated policy is predictable and controllable, we use a constrained learning method and train the painting agent  using the environment model and follows the commands encoded in an observation. We have applied our approach on many benchmarks and our results demonstrate that our constrained agent can handle different painting media and different constraints in the action space to collaborate with humans or other agents.
 Throughout human history, painting has been an essential element of artistic creation. There are many diverse and complex artistic domains with various styles such as watercolor, oil painting, sketching, and so on. As image processing and computer graphics have advanced, there has been a considerable effort to simulate these styles using non-photorealistic rendering (NPR) techniques (Kumar et al. (2019) ). Hertzmann (1998) ; Winkenbach & Salesin (1996) generate compelling results using stroke-based rendering. However, most prior methods in NPR are engineered for a specific application or task, and cannot easily adapt to new styles or medium. Recent developments in machine learning have resulted in significant advancements in computer vision and computer graphics, including computer-based painting systems. Many visual generative methods based on generative adversarial networks (Goodfellow et al. (2014) ) as Zhu et al. (2017) ; Zhou et al. (2018) ; ; Karras et al. (2017) ; Sangkloy et al. (2017) have demonstrated promising results. Many of these machine learning methods have also been applied to stroke-based rendering tasks, including modeling the brush (Xie et al. (2012) ; Zheng et al. (2019) ), generating brushstroke paintings in an artist's style (Xie et al. (2015) ), reconstructing drawings for specific paintings styles (Tang et al. (2018) ), and constructing stroke-based drawings (Ha & Eck (2017a) ; Zhou et al. (2018) ; ; Jia et al. (2019a) ). In this paper, we focus on a more general and challenging problem of training a natural media painting agent for interactive applications. Given a reference image, our goal is to develop a stroke-based rendering approach that can imitate the human drawing or strokes used in generating the image. A key challenge is to develop a method that can learn from scratch without any supervision. In this regard, we present a technique that can handle all inputs and train an agent to manipulate natural painting media such as charcoal, pencil, watercolor, and so on. We build a model-based natural media environment using deep CNN and train a natural media painting agent using model-based reinforcement learning. In order to introduce controls to the agents for interactive applications, we use a constraint representation along with a different framework for training and use the constrained painting agent. These constraints enable the agent to interact with a human or other agents and generate various styles without retraining the model. The novel contributions of our work include: • A method to train an agent that produces a stream of actions subject to constraint for each action. These constraints can include restricting the start location, stroke width, color, and other stroke parameters. • A method to roll out constrained agents so the user can produce new stylistic effects interactively or automatically, as the agent is painting by modulating the action stream. • By incorporate coarse-to-fine strategy, our painting agents can generate high-resolution stylized images using various constraints and paintbrush configurations. We evaluate our algorithm on different paintbrush configurations and datasets to highlights its benefits over prior reinforcement learning based methods. We also employ differing constraint settings to validate our constrained agents and produce new stylistic effects with a single trained model. In this paper, we train natural media painting agents that can generate artistic paintings using various natural media, and collaborate with humans and other agents to get different visual effects. We build a model of natural media environment using deep CNN and train a natural media painting agent using model-based reinforcement learning. To introduce controls to the agents for interactive purposes, we propose constraint representation, a framework for training a constrained painting agent, and various roll-out schemes to apply the agent. We demonstrate our algorithm by applying the trained model using various paintbrushes from MyPaint and constraints set up. The experimental results show that our algorithm can reproduce reference images in multiple artistic styles. For future work, we aim to extend the proposed algorithm by building a unified model for differing paintbrush configuration. In addition, we will train a hierarchical agent that uses a constrained agent as the low-level policy. We would like to apply our approach on other reference images and use for interactive painting systems. A APPENDIX Figure 9 : Roll-out results using Various PaintbrushesWe roll out our natural media painting agents trained with various brushes in MyPaint. To increase the resolutions of the generated images, we incorporate the coarse-to-fine strategy. We use 8 × 8 patches for first row and 4 × 4 for second row. Figure 10: Reproduction of Starry Night using Charcoal We roll out our natural media painting agent trained with charcoal brush in MyPaint to reproduce Van Gogh's starry night.We incorporate the coarse-to-fine strategy by dividing the reference image and canvas into 16 × 16 patches. Figure 11 : Reproduction of Starry Night using Watercolor We roll out our natural media painting agent trained with watercolor brush in MyPaint to reproduce Van Gogh's starry night.We incorporate the coarse-to-fine strategy by dividing the reference image and canvas into 16 × 16 patches.
We train a natural media painting agent using environment model. Based on our painting agent, we present a novel approach to train a constrained painting agent that follows the command encoded in the observation.
{"src": "scitldr-AIC_train", "paper_id": "SyeKGgStDB"}