This paper proposes a deep convolutional neural network model for ordinal regression by considering a family of probabilistic ordinal link functions in the output layer. The link functions are those used for cumulative link models, which are traditional statistical linear models based on projecting each pattern into a 1-dimensional space. A set of ordered thresholds splits this space into the different classes of the problem. In our case, the projections are estimated by a non-linear deep neural network. To further improve the results, we combine these ordinal models with a loss function that takes into account the distance between the categories, based on the weighted Kappa index. Three different link functions are studied in the experimental study, and the results are contrasted with statistical analysis. The experiments run over two different ordinal classification problems, and the statistical tests confirm that these models improve the results of a nominal model and outperform other proposals considered in the literature.
@cite_52 proposed a complex CNN architecture for solving Twitter Sentiment Classification as an ordinal problem. They checked that using average pooling preserves significant features that provide more expressiveness to ordinal scale. They didn't propose any method to include the ordinal information into the classifier, but they tried to find the best CNN model architecture based on an ordinal metric.
@cite_52
2784099047
Twitter sentiment analysis according to five points scales has attracted research interest due to its potential use in commercial and public social media application. A multi-point scale classification is a popular way used by many companies to evaluate the sentiment of product reviews (e.g. Alibaba, Amazon and eBay). Most of the classification approaches addressed this problem using traditional classification algorithm that requires expert knowledge to select the best features. Even though deep learning has been utilized, most of them employed a simple structure that not enough to capture the important features. In this paper, a complex structure of convolutional neural network (CNN) is proposed to classify the tweet into five-point scale and obtain a more several tweet representation. After a series of experiments with CNN including different hyperparameters and pooling strategies (Max and Average), we found that the best structure for our model is three convolutional layers, each one followed by average pooling layer. The proposed multi-layers convolutional neural network (MLCNN) model achieve the lowest Macro average mean absolute error (MAEM) and outperforms the state-of-the-art approach on tweet 2016 dataset for Ordinal classification. Experimental results show the ability of average pooling to preserve significant features that provide more expressiveness to ordinal scale.