TECHNICAL FIELD

The present invention relates to a system and method for rendering an image, and particularly, although not exclusively, to a system and method for incorporating a desired style into an image.

BACKGROUND

Graphical expressions and representations, often in the form of what is now known as images, have regularly been drawn, viewed and appreciated by human beings. Images and other graphical representations, which may be expressed in the form of drawings, artworks, films, photographs or any other suitable expressions often evoke a response from a viewer. This response may include a logical interpretation of the contents of the images or an emotional response as triggered by the manner in which the representations are drawn, expressed or shown.

Such responses may be considered by artists, advertisers, photographers or producers as part of a process to create a suitable image, video or artwork to convey a desired effect. However, the assessment of this desired effect may be difficult to ascertain as it is highly subjective to each respondent. In turn, images that are powerful in triggering a desired response are often created by expert artists, photographers or producers who have vast experiences in working with content, design and art to create a desired effect.

Accordingly, producing an artwork, image or film with the desired effect is a costly and difficult exercise, usually only possible with the employment of skilled producers and artists at high expenses. Thus such productions are often left to large budget projects that would not be feasible for smaller content creators or those that wish to create content as a hobby.

SUMMARY OF THE INVENTION

In accordance with a first aspect of the present invention, there is provided a system for rendering an image comprising:an image gateway arranged to receive content data representative of one or more objects and style data representative of one or more desired rendering styles; andan image manipulation engine arranged to process the content data and the style data to render a target image, wherein the target image includes a representation of the one or more objects and includes the one or more desired rendering styles.

In an embodiment of the first aspect, the image manipulation engine is arranged to process the content data with a pre-trained learning process, wherein the pre-trained learning process is arranged to extract object related information from the content data.

In an embodiment of the first aspect, the image manipulation engine is arranged to process the style data to extract rendering styles related information.

In an embodiment of the first aspect, the style data is processed with the pre-trained learning process arranged to extract rendering styles related information.

In an embodiment of the first aspect, the pre-trained learning process is arranged to extract object related information from the content data by processing the content data with a plurality of iteration, wherein each iteration of processing reveals further object related information.

In an embodiment of the first aspect, the pre-trained learning process is arranged to extract rendering styles related information by processing the style data with the plurality of iterations, wherein each iteration of processing reveals further rendering styles related information.

In an embodiment of the first aspect, the target image is rendered by the pre-trained learning process arranged to combine the object related information and the rendering styles related information.

In an embodiment of the first aspect, the pre-trained learning process combines the object related information and the rendering styles related information with the plurality of iterations, wherein each iteration of processing renders the target image to include additional object related information and rendering styles related information.

In an embodiment of the first aspect, the each iteration of processing to render the target image is optimized.

In an embodiment of the first aspect, the rendering of the target image is optimized by minimizing losses with each iteration between the target image and the additional object related information and rendering styles related information.

In an embodiment of the first aspect, the pre-trained learning process is included in a neural network.

In an embodiment of the first aspect, the each iteration is a propagation of each layer of the neural network.

In an embodiment of the first aspect, he target image is optimized by use of an Adam optimizer.

In an embodiment of the first aspect, the content data includes one or more images showing one or more objects.

In an embodiment of the first aspect, the style data includes one or more images having a desired style.

In an embodiment of the first aspect, the pre-trained learning processes is pre-trained with an object training set to classify a plurality of objects.

In an embodiment of the first aspect, the pre-trained learning process is further pre-trained with a style training set to classify a plurality of styles.

In an embodiment of the first aspect, the object training set and/or the style training set are a plurality of images classified with object identification and/or style identification.

In an embodiment of the first aspect, the neural network is a convolution neural network.

In an embodiment of the first aspect, the one or more images showing one or more objects are one or more frames of an image stream.

In accordance with a second aspect of the present invention, there is provided a method for rendering an image comprising the steps of:receiving content data representative of one or more objects and style data representative of one or more desired rendering styles of the one or more objects; andprocessing the content data and the style data to render a target image to include a representation of the one or more objects, wherein the representation includes the one or more desired rendering styles.

In an embodiment of the second aspect, the step of processing the content data and the style data to render the target image includes processing the content data with a pre-trained learning process, wherein the pre-trained learning process is arranged to extract object related information from the content data.

In an embodiment of the second aspect, the step of processing the content data and the style data to render the target image further includes processing the style data to extract rendering styles related information.

In an embodiment of the second aspect, the style data is processed with the pre-trained learning process arranged to extract rendering styles related information.

In an embodiment of the second aspect, the pre-trained learning process is arranged to extract object related information from the content data by processing the content data with a plurality of iteration, wherein each iteration of processing reveals further object related information.

In an embodiment of the second aspect, the pre-trained learning process is arranged to extract rendering styles related information by processing the style data with the plurality of iterations, wherein each iteration of processing reveals further rendering styles related information.

In an embodiment of the second aspect, wherein the target image is rendered by the pre-trained learning process arranged to combine the object related information and the rendering styles related information.

In an embodiment of the second aspect, the pre-trained learning process combines the object related information and the rendering styles related information with the plurality of iterations, wherein each iteration of processing renders the target image to include additional object related information and rendering styles related information.

In an embodiment of the second aspect, the each iteration of processing to render the target image is optimized.

In an embodiment of the second aspect, the rendering of the target image is optimized by minimizing losses with each iteration between the target image and the additional object related information and rendering styles related information.

In an embodiment of the second aspect, the pre-trained learning process is included in a neural network.

In an embodiment of the second aspect, the each iteration is a propagation of each layer of the neural network.

In an embodiment of the second aspect, the target image is optimized by use of an Adam optimizer.

In an embodiment of the second aspect, the content data includes one or more images showing one or more objects.

In an embodiment of the second aspect, style data includes one or more images having a desired style.

In an embodiment of the second aspect, the pre-trained learning processes is pre-trained with an object training set to classify a plurality of objects.

In an embodiment of the second aspect, the pre-trained learning process is further pre-trained with a style training set to classify a plurality of styles.

In an embodiment of the second aspect, the object training set and/or the style training set are a plurality of images classified with object identification and/or style identification.

In an embodiment of the second aspect, the neural network is a convolution neural network.

In an embodiment of the second aspect, the one or more images showing one or more objects are one or more frames of an image stream.

In accordance with a third aspect of the present invention, there is provided a method for manipulating a neural network comprising the steps of:training the neural network to generate a trained network arranged to process images to extract object information from the images;removing a final output layer of the trained neural network; andreplacing the final output layer with a layer of emotional states nodes arranged to represent one or more emotional states to generate a manipulated neural network.

In an embodiment of the third aspect, the manipulated neural network is further trained to extract emotion information from the images.

In an embodiment of the third aspect, the parameters of the trained network, except the final output layer, is retained within the manipulated neural network.

In an embodiment of the third aspect, the trained network is generated by training the neural network with an image classification database.

In an embodiment the image classification database is an ImageNet (image database organized according to the WordNet hierarchy) database or similar.

In an embodiment of the third aspect, the manipulated neural network is further trained by using an image emotion database.

In an embodiment the image emotion database is an International Affective Picture System (IAPS) dataset or similar.

