Automatic art analysis has been mostly focused on classifying artworks into different artistic styles. However, understanding an artistic representation involves more complex processes, such as identifying the elements in the scene or recognizing author influences. We present SemArt, a multi-modal dataset for semantic art understanding. SemArt is a collection of fine-art painting images in which each image is associated to a number of attributes and a textual artistic comment, such as those that appear in art catalogues or museum collections. To evaluate semantic art understanding, we envisage the Text2Art challenge, a multi-modal retrieval task where relevant paintings are retrieved according to an artistic text, and vice versa. We also propose several models for encoding visual and textual artistic representations into a common semantic space. Our best approach is able to find the correct image within the top 10 ranked images in the 45.5 of the test samples. Moreover, our models show remarkable levels of art understanding when compared against human evaluation.
With the digitalization of large collections of fine-art paintings and the emergence of publicly available online art catalogs such as WikiArt http: www.wikiart.org or the Web Gallery of Art https: www.wga.hu , computer vision researchers become interested in analyzing fine-art paintings automatically. Early work @cite_19 @cite_22 @cite_0 @cite_23 proposes methods based on handcrafted visual features to identify an author and or a specific style in a piece of art. Datasets used in these kinds of approaches, such as PRINTART @cite_0 and Painting-91 @cite_23 , are rather small, with 988 and 4,266 painting images, respectively. Mensink and Van Gemert introduce in @cite_21 the large-scale Rijksmuseum dataset for multi-class prediction, consisting on 112,039 images from artistic objects, although only 3,593 are from fine-art paintings. With the success of convolutional neural networks (CNN) in large-scale image classification @cite_1 , deep features from CNNs replace handcrafted image representations in many computer vision applications, including painting image classification @cite_17 @cite_5 @cite_25 @cite_13 @cite_7 @cite_15 , and larger datasets are made publicly available @cite_5 @cite_15 . In these methods, paintings are fed into a CNN to predict its artistic style or author by studying its visual aesthetics.
@cite_22 @cite_7 @cite_21 @cite_1 @cite_0 @cite_19 @cite_23 @cite_5 @cite_15 @cite_13 @cite_25 @cite_17
2128907386 2765362515 2161753062 2166242527 1574901103 2766086266 296507979
We describe a method for automated recognition of painters and schools of art based on their signature styles and studied the computer-based perception of visual art. Paintings of nine artists, representing three different schools of art—impressionism, surrealism and abstract expressionism—were analyzed using a large set of image features and image transforms. The computed image descriptors were assessed using Fisher scores, and the most informative features were used for the classification and similarity measurements of paintings, painters, and schools of art. Experimental results show that the classification accuracy when classifying paintings into nine painter classes is 77p, and the accuracy of associating a given painting with its school of art is 91p. An interesting feature of the proposed method is its ability to automatically associate different artists that share the same school of art in an unsupervised fashion. The source code used for the image classification and image similarity described in this article is available for free download. Compared with normal modalities, the representations of paintings are much more complex due to its large intra-class and small inter-class variation. This poses more difficulties in the task of authorship identification. In this paper, we propose a multi-task multi-range (MTMR) representation framework and try to resolve this issue in two ways. First, we investigate how to improve the representation through multi-task learning. Specifically, we attempt to optimize authorship identification with subtly correlated identification tasks such as style, genre and date. Second, in order to make the representation more comprehensive and reduce the information loss from image scaling, we propose a multi-range structure which is composed of local, regional and global representations. Experiments on the two most representative large-scale painting datasets, Rijksmuseum Challenge and Wikiart, have shown that our method significantly outperforms the existing methods. To give better understanding and provide more effective predictions, we utilize random forest as the feature ranking method to analyze the importance of different features and apply external knowledge matching to further examine the predictions. Moreover, the framework's effects of identifying the authorship are visualized on the paintings' artist-characteristic regions and t-SNE is further applied to perform artist-based cluster analysis. Extensive validation has demonstrated that the proposed framework yields superior performance in the chanllenging task of painting authorship identification. This paper offers a challenge for visual classification and content-based retrieval of artistic content. The challenge is posed from a museum-centric point of view offering a wide range of object types including paintings, photographs, ceramics, furniture, etc. The freely available dataset consists of 112,039 photographic reproductions of the artworks exhibited in the Rijksmuseum in Amsterdam, the Netherlands. We offer four automatic visual recognition challenges consisting of predicting the artist, type, material and creation year. We include a set of baseline results, and make available state-of-the-art image features encoded with the Fisher vector. Progress on this challenge improves the tools of a museum curator while improving content-based exploration by online visitors of the museum collection. The style of an image plays a significant role in how it is viewed, but style has received little attention in computer vision research. We describe an approach to predicting style of images, and perform a thorough evaluation of different image features for these tasks. We find that features learned in a multi-layer network generally perform best -- even when trained with object class (not style) labels. Our large-scale learning methods results in the best published performance on an existing dataset of aesthetic ratings and photographic style annotations. We present two novel datasets: 80K Flickr photographs annotated with 20 curated style labels, and 85K paintings annotated with 25 style genre labels. Our approach shows excellent classification performance on both datasets. We use the learned classifiers to extend traditional tag-based image search to consider stylistic constraints, and demonstrate cross-dataset understanding of style. Statistical approaches to processing natural language text have become dominant in recent years. This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear. The book contains all the theory and algorithms needed for building NLP tools. It provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations. The book covers collocation finding, word sense disambiguation, probabilistic parsing, information retrieval, and other applications. This paper aims to generate a better representation of visual arts, which plays a key role in visual arts analysis works. Museums and galleries have a large number of artworks in the database, hiring art experts to do analysis works (e.g., classification, annotation) is time consuming and expensive and the analytic results are not stable because the results highly depend on the experiences of art experts. The problem of generating better representation of visual arts is of great interests to us because of its application potentials and interesting research challenges---both content information and each unique style information within one artwork should be summarized and learned when generating the representation. For example, by studying a vast number of artworks, art experts summary and enhance the knowledge of unique characteristics of each visual arts to do visual arts analytic works, it is non-trivial for computer. In this paper, we present a unified framework, called DeepArt, to learn joint representations that can simultaneously capture contents and style of visual arts. This framework learns unique characteristics of visual arts directly from a large-scale visual arts dataset, it is more flexible and accurate than traditional handcraft approaches. We also introduce Art500k, a large-scale visual arts dataset containing over 500,000 artworks, which are annotated with detailed labels of artist, art movement, genre, etc. Extensive empirical studies and evaluations are reported based on our framework and Art500k and all those reports demonstrate the superiority of our framework and usefulness of Art500k. A practical system for visual arts retrieval and annotation is implemented based on our framework and dataset. Code, data and system are publicly available at http: deepart.ece.ust.hk. With the vast expansion of digital contemporary painting collections, automatic theme stylization has grown in demand in both academic and commercial fields. The recent interest in deep neural networks has provided powerful visual features that achieve state-of-the-art results in various visual classification tasks. In this work, we examine the perceptiveness of these features in identifying artistic styles in paintings, and suggest a compact binary representation of the paintings. Combined with the PiCoDes descriptors, these features show excellent classification results on a large scale collection of paintings.