Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the scientific advances in the human visual perception and neuroaesthetics, we design Brain-Inspired Deep Networks (BDN) for this task. BDN first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend BDN to predicting the distribution of human ratings, since aesthetics ratings are often subjective. Another highlight is our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that our biological inspired and task-specific BDN model gains significantly performance improvement, compared to other state-of-the-art models with the same or higher parameter capacity.
Most current work treat aesthetics assessment as a conventional classification problem: the user ratings of each photo are transformed into a ordinal scalar rating (by averaging, etc.), which is taken as the label of the photo. For example, RAPID @cite_35 simply divided all samples as aesthetic or unaesthetic, and trained a binary classification model. However, it is common for different users to rate visual subjects inconsistently or even oppositely due to the subjective problem nature @cite_34 . Since human aesthetic assessment depends on multiple dimensions such as composition, colorfulness, or even emotion @cite_33 , it is difficult for individuals to reliably convert their experiences to a single rating, resulting in noisy estimates of real aesthetic responses. In @cite_16 , Wu et.al. first proposed to represent each photo's rating as a distribution vector over basic ratings, constituting a structural regression problem. Gao et.al. @cite_15 formulated the aesthetic assessment as a multi-label task, where multiple aesthetic attributes were predicted jointly via bayesian networks.
@cite_35 @cite_33 @cite_16 @cite_15 @cite_34
2023169975 2806817979 2749958701 244014878
Current research of aesthetic assessment for images assumes one aesthetic score or one aesthetic label for an image, ignoring the relations of multiple aesthetic-related attributes. However, most images can be described by multiple aesthetic attributes simultaneously. Therefore, in this paper, we propose multiple aesthetic attribute prediction and classification by modeling relations among aesthetic attributes through Bayesian Networks (BN). In order to realize continuous aesthetic attribute prediction, each aesthetic attribute is represented by a three-node BN, including the discretized aesthetic attribute label, the predicted aesthetic attribute score, and the measurement of the aesthetic attribute score. In addition, the relations among multiple aesthetic attributes are modeled by another discrete BN, whose structure and conditional probabilities are learned from the training data. The attribute measurements are obtained by an existing image-driven regression method. With the learned BN, we infer the true discrete label and continuous score for each attribute by combining the relations among attributes with the previously obtained measurements. Experiments on the Memorability dataset show the superiority of our proposed approach to current image-driven methods for both multiple continuous aesthetic attribute score prediction and multiple discrete aesthetic attribute label classification, indicating the effectiveness of the captured relations for aesthetic quality assessment. The ability to rank the images based on their appearance finds many real-world applications, such as image retrieval or image album creation. Despite the recent dominance of deep learning methods in computer vision which often result in superior performance, they are not always the methods of choice because they lack interpretability. In this paper, we investigate the possibility of improving the image aesthetic inference of the convolutional neural networks with hand-designed features that rely on domain expertise in various fields. We perform a comparison of hand-crafted feature sets in their ability to predict fine-grained aesthetics scores on two image aesthetics data sets. We observe that even feature sets published earlier are able to compete with more recently published algorithms and, by combining the algorithms, a significant improvement in predicting image aesthetics is possible. By using a tree-based learner, we perform the feature elimination to understand the best performing features overall and across different image categories. Only roughly 15 and 8 of the features are needed to achieve full performance in predicting a fine-grained aesthetic score and binary classification, respectively. By combining the hand-crafted features with metafeatures that predict the quality of an image based on convolutional neural network features, the model performs better than a baseline VGG16 model. One can, however, achieve more significant improvement in both aesthetics score prediction and binary classification by fusing the hand-crafted features and the penultimate layer activations. Our experiments indicate an improvement up to 2.2 achieving current state-of-the-art binary classification accuracy on the aesthetic visual analysis data set when the hand-designed features are fused with activations from VGG16 and ResNet50 networks. Aesthetic quality prediction is a challenging task in the computer vision community because of the complex interplay with semantic contents and photographic technologies. Recent studies on the powerful deep learning based aesthetic quality assessment usually use a binary high-low label or a numerical score to represent the aesthetic quality. However the scalar representation cannot describe well the underlying varieties of the human perception of aesthetics. In this work, we propose to predict the aesthetic score distribution (i.e., a score distribution vector of the ordinal basic human ratings) using Deep Convolutional Neural Network (DCNN). Conventional DCNNs which aim to minimize the difference between the predicted scalar numbers or vectors and the ground truth cannot be directly used for the ordinal basic rating distribution. Thus, a novel CNN based on the Cumulative distribution with Jensen-Shannon divergence (CJS-CNN) is presented to predict the aesthetic score distribution of human ratings, with a new reliability-sensitive learning method based on the kurtosis of the score distribution, which eliminates the requirement of the original full data of human ratings (without normalization). Experimental results on large scale aesthetic dataset demonstrate the effectiveness of our introduced CJS-CNN in this task. Photo quality assessment from the view of human aesthetics, which tries to classify images into the categories of good and bad, has drawn a lot of attention in computer vision field. Up to now, experts have proposed many methods to deal with this problem. Most of those methods are based on the design of hand-crafted features. However, due to the complexity and subjectivity of human’s aesthetic activities, it is difficult to describe and model all the factors that affect the photo aesthetic quality. Therefore those methods just obtain limited success. On the other hand, deep convolutional neural network has been proved to be effective in many computer vision problems and it does not need human efforts in the design of features. In this paper, we try to adopt a deep convolutional neural network that “understands” images well to conduct the photo aesthetic quality assessment. Firstly, we implement a deep convolutional neural network which has eight layers and millions of parameters. Then to “teach” this network enough knowledge about images, we train it on the ImageNet which is one of the largest available image database. Next, for each given image, we take the activations of the last layer of the neural network as its aesthetic feature. The experimental results on two large and reliable image aesthetic quality assessment datasets prove the effectiveness of our method.