Social media, as a major platform for communication and information exchange, is a rich repository of the opinions and sentiments of 2.3 billion users about a vast spectrum of topics. To sense the whys of certain social user's demands and cultural-driven interests, however, the knowledge embedded in the 1.8 billion pictures which are uploaded daily in public profiles has just started to be exploited since this process has been typically been text-based. Following this trend on visual-based social analysis, we present a novel methodology based on Deep Learning to build a combined image-and-text based personality trait model, trained with images posted together with words found highly correlated to specific personality traits. So the key contribution here is to explore whether OCEAN personality trait modeling can be addressed based on images, here called ics , appearing with certain tags with psychological insights. We found that there is a correlation between those posted images and their accompanying texts, which can be successfully modeled using deep neural networks for personality estimation. The experimental results are consistent with previous cyber-psychology results based on texts or images. In addition, classification results on some traits show that some patterns emerge in the set of images corresponding to a specific text, in essence to those representing an abstract concept. These results open new avenues of research for further refining the proposed personality model under the supervision of psychology experts.
An early attempt to model personality from images was presented in Steele @cite_15 . This work explored the characteristics of profile pictures and the relationship with the impression that other users had of the owners of these pictures. Their findings show that users better agreed with the targets’ self-reported personalities when the profile picture was a photo of a human, taken outdoors instead of indoors, and the face appears smiling.
@cite_15
9991798 2118265251 2141418530 2133009022
With the increasing proliferation of data production technologies (like cameras) and consumption avenues (like social media) multimedia has become an interaction channel among users today. Images and videos are being used by the users to convey innate preferences and tastes. This has led to the possibility of using multimedia as a source for user-modeling, thereby contributing to the field of personalization, recommender systems, content generation systems and so on. This work investigates approaches for modeling personality traits (based on the Five Factor Modeling approach) of users based on a collection of images they tag as ‘favorite’ on Flickr. It presents several insights for improving the personality estimation performance by proposing better features and modeling approaches. The efficacy of the improved personality modeling approach is demonstrated by its use in an image recommendation system with promising results. Despite the crucial role of physical appearance in forming first impressions, little research has examined the accuracy of personality impressions based on appearance alone. This study examined the accuracy of observers’ impressions on 10 personality traits based on full-body photographs using criterion measures based on self and peer reports. When targets’ posture and expression were constrained (standardized condition), observers’ judgments were accurate for extraversion, self-esteem, and religiosity. When targets were photographed with a spontaneous pose and facial expression (spontaneous condition), observers’ judgments were accurate for almost all of the traits examined. Lens model analyses demonstrated that both static cues (e.g., clothing style) and dynamic cues (e.g., facial expression, posture) offered valuable personality-relevant information. These results suggest that personality is manifested through both static and expressive channels of appearance, and observers use this information to form... Manipulated versions of three-dimensional faces that have different profiles, but almost the same appearance in frontal views, provide a novel way to investigate if and how humans use class-specific knowledge to infer depth from images of faces. After seeing a frontal view, participants have to select the profile that matches that view. The profiles are original (ground truth), average, random other, and two solutions computed with a linear face model (3D Morphable Model). One solution is based on 2D vertex positions, the other on pixel colors in the frontal view. The human responses demonstrate that humans neither guess nor just choose the average profile. The results also indicate that humans actually use the information from the front view, and not just rely on the plausibility of the profiles per se. All our findings are perfectly consistent with a correlation-based inference in a linear face model. The results also verify that the 3D reconstructions from our computational algorithms (stimuli 4 and 5) are similar to what humans expect, because they are chosen to be the true profile equally often as the ground-truth profiles. Our experiments shed new light on the mechanisms of human face perception and present a new quality measure for 3D reconstruction algorithms. First impressions of social traits, such as trustworthiness or dominance, are reliably perceived in faces, and despite their questionable validity they can have considerable real-world consequences. We sought to uncover the information driving such judgments, using an attribute-based approach. Attributes (physical facial features) were objectively measured from feature positions and colors in a database of highly variable “ambient” face photographs, and then used as input for a neural network to model factor dimensions (approachability, youthful-attractiveness, and dominance) thought to underlie social attributions. A linear model based on this approach was able to account for 58 of the variance in raters’ impressions of previously unseen faces, and factor-attribute correlations could be used to rank attributes by their importance to each factor. Reversing this process, neural networks were then used to predict facial attributes and corresponding image properties from specific combinations of factor scores. In this way, the factors driving social trait impressions could be visualized as a series of computer-generated cartoon face-like images, depicting how attributes change along each dimension. This study shows that despite enormous variation in ambient images of faces, a substantial proportion of the variance in first impressions can be accounted for through linear changes in objectively defined features.