In this project we analysed how much semantic information images carry, and how much value image data can add to sentiment analysis of the text associated with the images. To better understand the contribution from images, we compared models which only made use of image data, models which only made use of text data, and models which combined both data types. We also analysed if this approach could help sentiment classifiers generalize to unknown sentiments.
@cite_2 performed a sentiment classification of Flickr and Twitter images with a Convolutional Neural Networks (CNN) model. To the best of our knowledge @cite_2 were the first to apply deep learning methods to image sentiment classification, and achieved state of the art results by doing so.
@cite_2
1784731433 2784099047 2606006859 2950714608
This paper introduces a visual sentiment concept classification method based on deep convolutional neural networks (CNNs). The visual sentiment concepts are adjective noun pairs (ANPs) automatically discovered from the tags of web photos, and can be utilized as effective statistical cues for detecting emotions depicted in the images. Nearly one million Flickr images tagged with these ANPs are downloaded to train the classifiers of the concepts. We adopt the popular model of deep convolutional neural networks which recently shows great performance improvement on classifying large-scale web-based image dataset such as ImageNet. Our deep CNNs model is trained based on Caffe, a newly developed deep learning framework. To deal with the biased training data which only contains images with strong sentiment and to prevent overfitting, we initialize the model with the model weights trained from ImageNet. Performance evaluation shows the newly trained deep CNNs model SentiBank 2.0 (or called DeepSentiBank) is significantly improved in both annotation accuracy and retrieval performance, compared to its predecessors which mainly use binary SVM classification models. Twitter sentiment analysis according to five points scales has attracted research interest due to its potential use in commercial and public social media application. A multi-point scale classification is a popular way used by many companies to evaluate the sentiment of product reviews (e.g. Alibaba, Amazon and eBay). Most of the classification approaches addressed this problem using traditional classification algorithm that requires expert knowledge to select the best features. Even though deep learning has been utilized, most of them employed a simple structure that not enough to capture the important features. In this paper, a complex structure of convolutional neural network (CNN) is proposed to classify the tweet into five-point scale and obtain a more several tweet representation. After a series of experiments with CNN including different hyperparameters and pooling strategies (Max and Average), we found that the best structure for our model is three convolutional layers, each one followed by average pooling layer. The proposed multi-layers convolutional neural network (MLCNN) model achieve the lowest Macro average mean absolute error (MAEM) and outperforms the state-of-the-art approach on tweet 2016 dataset for Ordinal classification. Experimental results show the ability of average pooling to preserve significant features that provide more expressiveness to ordinal scale. The convolutional neural network (CNN), which is one of the deep learning models, has seen much success in a variety of computer vision tasks. However, designing CNN architectures still requires expert knowledge and a lot of trial and error. In this paper, we attempt to automatically construct CNN architectures for an image classification task based on Cartesian genetic programming (CGP). In our method, we adopt highly functional modules, such as convolutional blocks and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity represented by the CGP encoding method are optimized to maximize the validation accuracy. To evaluate the proposed method, we constructed a CNN architecture for the image classification task with the CIFAR-10 dataset. The experimental result shows that the proposed method can be used to automatically find the competitive CNN architecture compared with state-of-the-art models. The convolutional neural network (CNN), which is one of the deep learning models, has seen much success in a variety of computer vision tasks. However, designing CNN architectures still requires expert knowledge and a lot of trial and error. In this paper, we attempt to automatically construct CNN architectures for an image classification task based on Cartesian genetic programming (CGP). In our method, we adopt highly functional modules, such as convolutional blocks and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity represented by the CGP encoding method are optimized to maximize the validation accuracy. To evaluate the proposed method, we constructed a CNN architecture for the image classification task with the CIFAR-10 dataset. The experimental result shows that the proposed method can be used to automatically find the competitive CNN architecture compared with state-of-the-art models.