Humans comprehend a natural scene at a single glance; painters and other visual artists, through their abstract representations, stressed this capacity to the limit. The performance of computer vision solutions matched that of humans in many problems of visual recognition. In this paper we address the problem of recognizing the genre (subject) in digitized paintings using Convolutional Neural Networks (CNN) as part of the more general dealing with abstract and or artistic representation of scenes. Initially we establish the state of the art performance by training a CNN from scratch. In the next level of evaluation, we identify aspects that hinder the CNNs' recognition, such as artistic abstraction. Further, we test various domain adaptation methods that could enhance the subject recognition capabilities of the CNNs. The evaluation is performed on a database of 80,000 annotated digitized paintings, which is tentatively extended with artistic photographs, either original or stylized, in order to emulate artistic representations. Surprisingly, the most efficient domain adaptation is not the neural style transfer. Finally, the paper provides an experiment-based assessment of the abstraction level that CNNs are able to achieve.
Computer based painting analysis has been in the focus of the computer vision community for a long period. A summary of various directions approached, algorithms and results for not-so-recent solutions can be found in the review of Bentowska and Coddington Bentowska2010 . However, a plurality of works @cite_13 @cite_9 @cite_22 @cite_17 addressed style (art movement) recognition as it is the main label associated with paintings. An intermediate topic is in the work of Monroy2014 which detected and extracted shapes (associated with objects) but as pre-processing for the final task which was that of restoration.
@cite_9 @cite_13 @cite_22 @cite_17
2128907386 2613527656 2615562997 2765362515
We describe a method for automated recognition of painters and schools of art based on their signature styles and studied the computer-based perception of visual art. Paintings of nine artists, representing three different schools of art—impressionism, surrealism and abstract expressionism—were analyzed using a large set of image features and image transforms. The computed image descriptors were assessed using Fisher scores, and the most informative features were used for the classification and similarity measurements of paintings, painters, and schools of art. Experimental results show that the classification accuracy when classifying paintings into nine painter classes is 77p, and the accuracy of associating a given painting with its school of art is 91p. An interesting feature of the proposed method is its ability to automatically associate different artists that share the same school of art in an unsupervised fashion. The source code used for the image classification and image similarity described in this article is available for free download. We address the problem of automatically recognizing artistic movement in digitized paintings. We make the following contributions: Firstly, we introduce a large digitized painting database that contains refined annotations of artistic movement. Secondly, we propose a new system for the automatic categorization that resorts to image descriptions by color structure and novel topographical features as well as to an adapted boosted ensemble of support vector machines. The system manages to isolate initially misclassified images and to correct such errors in further stages of the boosting process. The resulting performance of the system compares favorably with classical solutions in terms of accuracy and even manages to outperform modern deep learning frameworks. Humans are capable of perceiving a natural scene at a glance and painters, through their representations, push this capacity to the limit by abstraction. Yet we still do it... What about machines? In this paper we address the problem of recognizing the theme (scene type) in digitized paintings. The approach is based on the Convolutional Neural Network framework and the chosen architecture is the recent Residual Network. In the first level of evaluation, we determine the recognition rate of a CNN given a database of 80,000 annotated digitized paintings. In the second level we evaluate the impact of extending the training database with photographs directly and via two domain adaptation functions and thus we are able to assess the abstraction level that CNN is capable to achieve. Compared with normal modalities, the representations of paintings are much more complex due to its large intra-class and small inter-class variation. This poses more difficulties in the task of authorship identification. In this paper, we propose a multi-task multi-range (MTMR) representation framework and try to resolve this issue in two ways. First, we investigate how to improve the representation through multi-task learning. Specifically, we attempt to optimize authorship identification with subtly correlated identification tasks such as style, genre and date. Second, in order to make the representation more comprehensive and reduce the information loss from image scaling, we propose a multi-range structure which is composed of local, regional and global representations. Experiments on the two most representative large-scale painting datasets, Rijksmuseum Challenge and Wikiart, have shown that our method significantly outperforms the existing methods. To give better understanding and provide more effective predictions, we utilize random forest as the feature ranking method to analyze the importance of different features and apply external knowledge matching to further examine the predictions. Moreover, the framework's effects of identifying the authorship are visualized on the paintings' artist-characteristic regions and t-SNE is further applied to perform artist-based cluster analysis. Extensive validation has demonstrated that the proposed framework yields superior performance in the chanllenging task of painting authorship identification.