In this paper, we present a novel method of no-reference image quality assessment (NR-IQA), which is to predict the perceptual quality score of a given image without using any reference image. The proposed method harnesses three functions (i) the visual attention mechanism, which affects many aspects of visual perception including image quality assessment, however, is overlooked in the NR-IQA literature. The method assumes that the fixation areas on an image contain key information to the process of IQA. (ii) the robust averaging strategy, which is a means --- supported by psychology studies --- to integrating multiple step-wise evidence to make a final perceptual judgment. (iii) the multi-task learning, which is believed to be an effectual means to shape representation learning and could result in a more generalized model. To exploit the synergy of the three, we consider the NR-IQA as a dynamic perception process, in which the model samples a sequence of "informative" areas and aggregates the information to learn a representation for the tasks of jointly predicting the image quality score and the distortion type. The model learning is implemented by a reinforcement strategy, in which the rewards of both tasks guide the learning of the optimal sampling policy to acquire the "task-informative" image regions so that the predictions can be made accurately and efficiently (in terms of the sampling steps). The reinforcement learning is realized by a deep network with the policy gradient method and trained through back-propagation. In experiments, the model is tested on the TID2008 dataset and it outperforms several state-of-the-art methods. Furthermore, the model is very efficient in the sense that a small number of fixations are used in NR-IQA.
In addition to the patch-based methods, some methods combine hand-crafted low-level features with deep networks as an alternative approach. For example, Tang al @cite_15 first extract the LBIQ features @cite_13 and feed the features into a Restricted Boilzman Machine to predict image quality scores. Hou al @cite_1 pose IQA problem as a classification problem. They slot images into different categories according to the image quality and propose a quality pooling method under the Bayesian framework to predict quality scores.
@cite_1 @cite_15 @cite_13
2162692770 1537865522 2509123681 2806817979
We develop an efficient general-purpose blind no-reference image quality assessment (IQA) algorithm using a natural scene statistics (NSS) model of discrete cosine transform (DCT) coefficients. The algorithm is computationally appealing, given the availability of platforms optimized for DCT computation. The approach relies on a simple Bayesian inference model to predict image quality scores given certain extracted features. The features are based on an NSS model of the image DCT coefficients. The estimated parameters of the model are utilized to form features that are indicative of perceptual quality. These features are used in a simple Bayesian inference approach to predict quality scores. The resulting algorithm, which we name BLIINDS-II, requires minimal training and adopts a simple probabilistic model for score prediction. Given the extracted features from a test image, the quality score that maximizes the probability of the empirically determined inference model is chosen as the predicted quality score of that image. When tested on the LIVE IQA database, BLIINDS-II is shown to correlate highly with human judgments of quality, at a level that is competitive with the popular SSIM index. We develop an efficient general-purpose blind no-reference image quality assessment (IQA) algorithm that utilizes curvelet domain features of phase congruency values and local spectral entropy values in distorted images. A 2-stage framework of distortion classification followed by quality assessment is used for mapping feature vectors to prediction scores. We utilize a support vector machine (SVM) to train an image distortion and quality prediction model. The resulting algorithm which we name Phase Congruency and Spectral Entropy based Quality (PCSEQ) index is capable of assessing the quality of distorted images across multiple distortion categories. We explain the advantages of phase congruency features and spectral entropy features. We also thoroughly test the algorithm on the LIVE IQA databse and find that PCSEQ correlates well with human judgments of quality. It is superior to the full-reference (FR) IQA algorithm SSIM and several top-performance no-reference (NR) IQA methods such as DIIVINE and SSEQ. We also tested PCSEQ on the TID2008 database to ascertain whether it has performance that is database independent. This paper presents a no reference image (NR) quality assessment (IQA) method based on a deep convolutional neural network (CNN). The CNN takes unpreprocessed image patches as an input and estimates the quality without employing any domain knowledge. By that, features and natural scene statistics are learnt purely data driven and combined with pooling and regression in one framework. We evaluate the network on the LIVE database and achieve a linear Pearson correlation superior to state-of-the-art NR IQA methods. We also apply the network to the image forensics task of decoder-sided quantization parameter estimation and also here achieve correlations of r = 0.989. The ability to rank the images based on their appearance finds many real-world applications, such as image retrieval or image album creation. Despite the recent dominance of deep learning methods in computer vision which often result in superior performance, they are not always the methods of choice because they lack interpretability. In this paper, we investigate the possibility of improving the image aesthetic inference of the convolutional neural networks with hand-designed features that rely on domain expertise in various fields. We perform a comparison of hand-crafted feature sets in their ability to predict fine-grained aesthetics scores on two image aesthetics data sets. We observe that even feature sets published earlier are able to compete with more recently published algorithms and, by combining the algorithms, a significant improvement in predicting image aesthetics is possible. By using a tree-based learner, we perform the feature elimination to understand the best performing features overall and across different image categories. Only roughly 15 and 8 of the features are needed to achieve full performance in predicting a fine-grained aesthetic score and binary classification, respectively. By combining the hand-crafted features with metafeatures that predict the quality of an image based on convolutional neural network features, the model performs better than a baseline VGG16 model. One can, however, achieve more significant improvement in both aesthetics score prediction and binary classification by fusing the hand-crafted features and the penultimate layer activations. Our experiments indicate an improvement up to 2.2 achieving current state-of-the-art binary classification accuracy on the aesthetic visual analysis data set when the hand-designed features are fused with activations from VGG16 and ResNet50 networks.