Social media, as a major platform for communication and information exchange, is a rich repository of the opinions and sentiments of 2.3 billion users about a vast spectrum of topics. To sense the whys of certain social user's demands and cultural-driven interests, however, the knowledge embedded in the 1.8 billion pictures which are uploaded daily in public profiles has just started to be exploited since this process has been typically been text-based. Following this trend on visual-based social analysis, we present a novel methodology based on Deep Learning to build a combined image-and-text based personality trait model, trained with images posted together with words found highly correlated to specific personality traits. So the key contribution here is to explore whether OCEAN personality trait modeling can be addressed based on images, here called ics , appearing with certain tags with psychological insights. We found that there is a correlation between those posted images and their accompanying texts, which can be successfully modeled using deep neural networks for personality estimation. The experimental results are consistent with previous cyber-psychology results based on texts or images. In addition, classification results on some traits show that some patterns emerge in the set of images corresponding to a specific text, in essence to those representing an abstract concept. These results open new avenues of research for further refining the proposed personality model under the supervision of psychology experts.
The relationship between language and personality has been studied extensively. As commented before, Yarkoni @cite_26 performed a large-scale analysis of personality and word use in a large sample of blogs whose authors answered questionnaires to assess their personality. This way, by analyzing the text written by users whose personality is known, the author could investigate the relation between word use and personality. The results of the analysis concluded that the usage of some specific words are correlated with the personality of the blogs' authors.
@cite_26
1973871257 2119595472 1972820248 2114030807
Abstract Previous studies have found systematic associations between personality and individual differences in word use. Such studies have typically focused on broad associations between major personality domains and aggregate word categories, potentially masking more specific associations. Here I report the results of a large-scale analysis of personality and word use in a large sample of blogs ( N = 694). The size of the dataset enabled pervasive correlations with personality to be identified for a broad range of lexical variables, including both aggregate word categories and individual English words. The results replicated category-level findings from previous off-line studies, identified numerous novel associations at both a categorical and single-word level, and underscored the value of complementary approaches to the study of personality and word use. We analyzed 700 million words, phrases, and topic instances collected from the Facebook messages of 75,000 volunteers, who also took standard personality tests, and found striking variations in language with personality, gender, and age. In our open-vocabulary technique, the data itself drives a comprehensive exploration of language that distinguishes people, finding connections that are not captured with traditional closed-vocabulary word-category analyses. Our analyses shed new light on psychosocial processes yielding results that are face valid (e.g., subjects living in high elevations talk about the mountains), tie in with other research (e.g., neurotic people disproportionately use the phrase ‘sick of’ and the word ‘depressed’), suggest new hypotheses (e.g., an active life implies emotional stability), and give detailed insights (males use the possessive ‘my’ when mentioning their ‘wife’ or ‘girlfriend’ more often than females use ‘my’ with ‘husband’ or 'boyfriend’). To date, this represents the largest study, by an order of magnitude, of language and personality. Can language use reflect personality style? Studies examined the reliability, factor structure, and validity of written language using a word-based, computerized text analysis program. Daily diaries from 15 substance abuse inpatients, daily writing assignments from 35 students, and journal abstracts from 40 social psychologists demonstrated good internal consistency for over 36 language dimensions. Analyses of the best 15 language dimensions from essays by 838 students yielded 4 factors that replicated across written samples from another 381 students. Finally, linguistic profiles from writing samples were compared with Thematic Apperception Test coding, self-reports, and behavioral measures from 79 students and with self-reports of a 5-factor measure and health markers from more than 1,200 students. Despite modest effect sizes, the data suggest that linguistic style is an independent and meaningful way of exploring personality. Three experiments were conducted to determine the psychometric properties of language in dyadic interactions. Using text-analysis, it was possible to assess the degree to which people coordinate their word use in natural conversations. In Experiments 1 (n = 130) and 2( n = 32), college students interacted in dyadic conversations in laboratory-based private Internet chat rooms. Experiment 3 analyzed the official transcripts of the Watergate tapes involving the dyadic interactions between President Richard Nixon and his aids H. R. Haldeman, John Erlichman, and John Dean. The results of the three studies offer sub- stantial evidence that individuals in dyadic interactions exhibit linguistic style matching (LSM) on both the conversation level as well as on a turn-by-turn level. Furthermore, LSM is unrelated to ratings of the quality of the interaction by both participants and judges. We propose that a coordination-engagement hypothesis is a better description of linguistic behaviors than the coordination-rapport hypothesis that has been proposed in the nonver- bal literature. For years social psychologists have exalted the power of the situa- tion. We comfortably acknowledge that across different situations, with different people, we may act in a range of ways, or even talk using a variety of styles. Aware of this tendency, Gergen (1972) began his explorations of our shifting masks of identity. In writing letters to close friends, he realized that he came across as a "completely different per- son" in each letter. "In one, I was morose, pouring out a philosophy of existential sorrow; in another I was a lusty realist: in a third I was a lighthearted jokester" (p. 32). Based merely on his word choices, Gergen inadvertently varied his style to adapt to the recipients of his letters. This is a prime demonstration of our inherent knowledge of the mutability of our language with respect to varying social contexts.