Sharing multimodal information (typically images, videos or text) in Social Network Sites (SNS) occupies a relevant part of our time. The particular way how users expose themselves in SNS can provide useful information to infer human behaviors. This paper proposes to use multimodal data gathered from Instagram accounts to predict the perceived prototypical needs described in Glasser's choice theory. The contribution is two-fold: (i) we provide a large multimodal database from Instagram public profiles (more than 30,000 images and text captions) annotated by expert Psychologists on each perceived behavior according to Glasser's theory, and (ii) we propose to automate the recognition of the (unconsciously) perceived needs by the users. Particularly, we propose a baseline using three different feature sets: visual descriptors based on pixel images (SURF and Visual Bag of Words), a high-level descriptor based on the automated scene description using Convolutional Neural Networks, and a text-based descriptor (Word2vec) obtained from processing the captions provided by the users. Finally, we propose a multimodal fusion of these descriptors obtaining promising results in the multi-label classification problem.
@cite_13 surveyed methods published from 2005 to 2017 about automatic depression assessment based on visual cues. They addressed several research questions, including the number of modalities employed, facial signs, experimental protocols for dataset acquisition, feature descriptors, decision methods, and scores. They concluded that results are consistent with the social withdrawal, emotion-context insensitivity, reduced reactivity hypotheses of depression, and the importance of dynamic features multimodal approaches through the quantitative analysis. They also mentioned that the multitude of reported approaches on automatic depression assessment is not mature enough because clinical research questions such as the capacity to distinguish between different depression sub-types or the influence of ethnicity and culture on the progress of mental health were not addressed systematically. Finally, they argued that visual cues need to be supplemented by information from other modalities to achieve clinically useful results.
@cite_13
2760537051
Automatic depression assessment based on visual cues is a rapidly growing research domain. The present exhaustive review of existing approaches as reported in over sixty publications during the last ten years focuses on image processing and machine learning algorithms. Visual manifestations of depression, various procedures used for data collection, and existing datasets are summarized. The review outlines methods and algorithms for visual feature extraction, dimensionality reduction, decision methods for classification and regression approaches, as well as different fusion strategies. A quantitative meta-analysis of reported results, relying on performance metrics robust to chance, is included, identifying general trends and key unresolved issues to be considered in future studies of automatic depression assessment utilizing visual cues alone or in combination with vocal or verbal cues.