TECHNICAL FIELD

The present disclosure relates to a field of a non-photorealistic style transfer technology in computer graphics, and in particular to an interactive method for generating strokes with Chinese ink painting style and a device thereof.

BACKGROUND

A Chinese freehand ink painting differs from other painting types (such as Western oil painting and watercolor painting). It uses a few strokes to describe specific scene objects or scenes, expresses an artist's creative intention and artistic personality. Freehand ink painting attracts more and more people, including painting lovers who have no foundation in painting technology, to learn to create ink painting with its unique artistry. Moreover, most graphic commercial software (such as ADOBE ILLUSTRATOR, ADOBE PHOTOSHOP, CORELDRAW, etc.) have Chinese art style generation tools. However, to generate high-quality paintings usually requires professional knowledge and skills, and the conventional painting tools require complex and accurate input in drawing skills. Meanwhile, tools that applies style transfer to generate painting (such asFIG. 1 (b)) only transfer a certain painting style to a given object as a whole, and stroke shape and color of local are rarely considered.

Flower ink painting is one of main categories in Chinese ink painting, which represents an important artistic feature of oriental painting. In other words, typical pen and ink techniques used in flower ink paintings form the unique artistic charm and value of the ink painting. Strokes of the flower ink painting are either curved or straight, stiff or soft, wide or narrow, or pale or thick. Its ink color achieves various changes in dry, wet, and shades through the mixing and diffusion between water and ink. Thus the diversity and complexity of strokes and ink colors, as well as the professional skills of painting make a task of generating paintings with Chinese ink painting style challenging.

In general, non-photorealistic drawing refers to constructing different brush models to output strokes with different artistic style, and depict specific objects like hand-painting. In recent years, there are some excellent works related to non-photorealistic drawing that are worth learning from. There excellent works could be divided in to two aspects, which will be described as follows.

In one aspect, physics-based models are provided. Specifically, in order to provide users with an intuitive and natural feeling, pen-like devices simulate the physical process of painting by modeling physical properties of pens or other media. E.g., Chu and Tai [Chu N. S. H., Tai C.-L., “MoXi: Real-Time Ink Dispersion in Absorbent Paper”, ACM Transactions on Graphics, vol. 24(3), pp. 504-511, 2005] provide a real-time painting system to simulate an effect of ink diffusion on paper with absorption. Meanwhile, Xu et al [Xu S., Tang M., LAU F., et al., “Virtual Hairy Brush for Painterly Rendering”, Graphical Models, vol. 66(5), pp. 263-302, 2004] provides an electronic brush for Chinese calligraphy and painting, which uses only four attributes to define basic characteristics of a real brush. Further, Lu et al [Lu J., Barnes C., DiVerdi S., et al., “RealBrush: Painting with Examples of Physical Media”, ACM Transactions on Graphics, vol. 32(4), page. 117, 2013] provide an interactive data-driven drawing system that synthesizes new strokes by scanned images of real media, which avoids complex physical simnulation calculations. For digital painting software, such as ADOBE BRUSH PACKAGES, ARTRAGE, COREL PAINTER, etc., users, like using a real brush, use a mouse or a digital pen to draw. However, if not trained in the operation of such virtual devices, it is complex and difficult for users without relevant painting experience to operate such devices easily and accurately.

In the other aspect, image-based models are provided. Compared with the physics-based models, the image-based models avoid large-scale calculations and complex manipulations. Specifically, in simulating Western painting, Herzmann et al [Hertzmann A., Jacobs C. E., Oliver N., et al., “Image analogies”, In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, New York, USA, pp. 327-340, 2001] adopt “image analogy” technology to process photo images to generate artistic images. However, in their experimental results, since different image areas are drawn in different styles in Chinese painting, algorithms of Herzmann et al are unable to distinguish these image areas, so the Chinese ink style result is not ideal. In simulating oriental painting, Xie et al. [Xie N., Laga H., Saito S., “IR2s: interactive real photo to Sumi-e”, Ir: Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, Annecy, France, pp. 63-71, 2010] provides a sketching-based interactive system to transform real photos into Japanese ink paintings. Further, Dong et al [Dong L, Lu S. and Jin X., “Real-time Image-based Chinese Ink Painting Rendering”, Multimedia Tools & Applications, vol. 69(3), pp. 605-620, 2014] provide texture mapping and texture synthesis technology to converted images into images with Chinese ink style. However, models in the prior art that simulating the oriental painting style mainly deal with an entire image rather than the strokes that make up the image, which makes them difficult to express rich and varied artistic characteristics of Chinese ink paintings.

SUMMARY

In view of the above technical problems, in order to overcome shortcomings of the prior art, the present disclosure provides an interactive method for generating strokes with Chinese ink painting style and a device for generating strokes with Chinese ink painting style. The present disclosure provides a series of algorithms to convert guide lines drawn by users on the image into strokes in a certain Chinese ink painting style, which achieve an effect of migrating Chinese ink style strokes to the image.

To achieve the above object, the present disclosure provides an interactive method for generating strokes with Chinese ink painting style, includes steps:

obtaining an image including a pattern as an image object;

obtaining a delimiting operation delimiting at least one stroke sample on a pre-stored ink painting sample,

obtaining a basic outline drawn by a user on the image object, the basic outline forming a preliminary basic path of a stroke to be generated;

correcting stroke outlines in the at least one stroke sample to obtain accurate stroke samples as candidate stroke samples; using the candidate stroke samples as references to generate morphological sample groups;

correcting the preliminary basic path to obtain an accurate basic path;

selecting morphological samples best matching the accurate basic path in the morphological sample groups as final stroke samples; and

mapping style features of the final stroke samples onto the accurate basic path to generate an output image with Chinese ink painting style.

In a first optional embodiment, the step of correcting the stroke outlines in the at least one stroke sample to obtain the accurate stroke samples as the candidate stroke samples further comprises:

determining a stroke outline closest to an edge of the stroke sample in the at least one stroke sample to obtain the candidate stroke samples by opposite search algorithm; and

using the candidate stroke samples as the references to generate the morphological sample groups by means of mirror paths and reverse paths. Each of the morphological sample groups comprises at least four morphological samples. The four morphological samples are, respectively, a corresponding candidate stroke sample as one of the references, a mirror stroke sample of the corresponding candidate stroke sample, a reverse path stroke sample of the corresponding candidate stroke sample, and a reverse path stroke sample of the mirror stroke sample of the corresponding candidate stroke sample.

In a second optional embodiment, the step of correcting the preliminary basic path to obtain the accurate basic path comprises:

smoothing the preliminary basic path to obtain the accurate basic path by non-uniform rational B-spline curve algorithm.

In a third optional embodiment, the step of selecting the morphological samples best matching the accurate basic path in the morphological sample groups as the final stroke samples comprises:

dividing the accurate basic path into several path portions, and selecting, using energy formulas, a respective morphological sample in each of the morphological sample groups best matching each path portions as the final stroke samples of the path portions.

In a fourth optional embodiment, the step of dividing the accurate basic path into several path portions, and selecting, using energy formulas, a respective morphological sample in each of the morphological sample groups best matching each path portions as the final stroke samples of the path portions comprises:

defining an energy term inspace by the energy formulas, wherein the energy term is configured to describe an overall difference between a displacement term of a stroke outlines in the final stroke samples and a displacement term of a corresponding accurate basic path (a corresponding path portion), and make the overall difference minimize, so as to automatically select the final stroke samples.

The displacement term is configured to describe a displacement between relative position vector and stroke direction vector at any point on the stroke Outlines and relative position vector and stroke direction vector at any point on the accurate basic path.

In a fifth optional embodiment is further improved on the basis of the fourth optional embodiment, the step of mapping style features of the final stroke samples onto the accurate basic path comprises:

adjusting an aspect ratio of each of the final stroke samples to make the aspect ratios of the final stroke samples consistent with an aspect ratio of corresponding path portions.

In another optional embodiment, the step of the step of mapping style features of the final stroke samples onto the accurate basic path further comprises: rasterizing filling an area of the accurate basic path with scanning lines and filling pixels selected from the final stroke samples into the path portions corresponding to the final stroke samples by pixel-based texture mapping algorithm.

As a further improvement of the fifth optional embodiment, the pixels are limited by following constraints:

a. neighborhood pixels of pShave a color value closest to neighborhood pixels of pF;

b. pSand pFare closest to their respective bone path;

c. a color value of pSis closest to an average color value of pixels surrounding pP.

Where is pSa best pixel selected in a ink stroke area pFis any one of pixels in the area of the accurate basic path. pPis a pixel in the input image including the pattern. And the above constraints are measured by 3 energy terms. Eb(pS) corresponds to the constraint a and depicts a sum of the differences between color values of neighborhood pixels of pSand color values of neighborhood pixels of pF.

Ep(pS) corresponds to the constraint b and depicts how close pSand pFare to their respective bone path.

Ea(pS) corresponds to the constraint c and depicts differences between an average color value of pixels in a circle centered on pSwith radius R and an average color value of pixels in circle centered on pPwith the radius R.

Where the three 3 energy terms satisfy following formulas:

{Eb⁡(pS)=∑i,D∈{S,F}piD-pD≤1⁢λ⁡(piS)-λ⁡(piF)Ep⁡(pS)=pS-CS⁡(t~)rS⁡(t~)-pF-CP⁡(t~)rP⁡(t~)Ea⁡(pS)=1N⁢∑jpjS-pS≤R⁢λ⁡(pjS)-∑jpjP-pP≤R⁢λ⁡(pjP)

λ(p) represents a color value p. piSof is a neighborhood pixel of pS. And piFis a neighborhood pixel of pF. CP({tilde over (t)}) is a point closest to pFon the bone path. At a same value of {tilde over (t)}, CS({tilde over (t)}) is a corresponding point on the bone path in the final stroke samples. rSand rPare, respectively, half of a width of the corresponding final stroke sample. N represents number of pixels in the circle with the radius R. pjSrepresents a pixel near pSin the circle. pjPrepresents a pixel near pPwithin the specified circle of the image.

Furthermore, the step of defining an energy equation composed of the three 3 energy terms in thespace and minimizing the three energy terms to get the best pixel, by pixel-based texture mapping algorithm comprises following formulas:

minpS∈FS⁢[α1⁢Eb⁡(pS)+α2⁢Ep⁡(pS)+α3⁢Ea⁡(pS)].

Where α1, α2, and α3represent balance weight parameters.

In the present disclosure, by correcting the stroke samples, a blank area in the delimiting operation is eliminated and the accurate outline in the stroke samples are obtained.

In the present disclosure, by correcting the preliminary basic path, the unevenness of the edge outline of the preliminary basic path are removed and the preliminary basic path is smooth.

Based on the candidate sample groups, the present disclosure generates their mirror paths, reverse paths, and mirror reverse paths, and classifies the above four kinds of samples as morphological sample groups, such that in a process of best matching with the accurate basic path, the morphology in the candidate samples is expanded, and the morphological sample groups are adapted to line forms of different accurate basic paths that facilitates the path portions to find the best matching morphological samples.

The present disclosure adopts a method of style feature mapping to map width and color features of the final stroke samples to the accurate basic path to realize an effect of replacing the accurate basic path with the final stroke samples.

Furthermore, the present disclosure provides a device for generating strokes with Chinese ink painting style. The device comprises:

an image selecting unit selecting an image including a pattern as an image object;

a stroke sample selecting unit delimiting at least one stroke sample on a pre-stored ink painting sample;

a preliminary basic path selecting unit drawing a basic outline of the pattern on the image object to form a preliminary basic path of a stroke to be generated;

a stroke sample correcting unit correcting the outline of the delimited stroke sample to obtain an accurate stroke sample as a candidate stroke sample;

a morphological sample group generating unit generating morphological sample groups with reference to the candidate stroke samples;

a preliminary basic path correcting unit correcting the preliminary basic path to obtain an accurate basic path;

a final stroke sample selecting unit selecting the morphological samples best matching the accurate basic path in the morphological sample groups as final stroke samples; and

a style feature mapping unit mapping style features of the final stroke samples onto the accurate basic path to generate an output image with Chinese ink painting style.

