Automatically extracting social meaning and intention from spoken dialogue is an important task for dialogue systems and social computing . We describe a system for detecting elements of interactional style : whether a speaker is awkward , friendly , or flirtatious . We create and use a new spoken corpus of 991 4-minute speed-dates . Participants rated their interlocutors for these elements of style . Using rich dialogue , lexical , and prosodic features , we are able to detect flirtatious , awkward , and friendly styles in noisy natural conversational data with up to 75 % accuracy , compared to a 50 % baseline . We describe simple ways to extract relatively rich dialogue features , and analyze which features performed similarly for men and women and which were gender-specific . How can we extract social meaning from speech , deciding if a speaker is particularly engaged in the conversation , is uncomfortable or awkward , or is particularly friendly and flirtatious ? Understanding these meanings and how they are signaled in language is an important sociolinguistic task in itself . Extracting them automatically from dialogue speech and text is crucial for developing socially aware computing systems for tasks such as detection of interactional problems or matching conversational style , and will play an important role in creating more natural dialogue agents ( Pentland , 2005 ; Nass and Brave , 2005 ; Brave et al . , 2005 ) . Cues for social meaning permeate speech at every level of linguistic structure . Acoustic cues such as low and high F0 or energy and spectral tilt are important in detecting emotions such as annoyance , anger , sadness , or boredom ( Ang et al . , 2002 ; Lee and Narayanan , 2002 ; Liscombe et al . , 2003 ) , speaker characteristics such as charisma ( Rosenberg and Hirschberg , 2005 ) , or personality features like extroversion ( Mairesse et al . , 2007 ; Mairesse and Walker , 2008 ) . Lexical cues to social meaning abound . Speakers with links to depression or speakers who are under stress use more first person singular pronouns ( Rude et al . , 2004 ; Pennebaker and Lay , 2002 ; Cohn et al . , 2004 ) , positive emotion words are cues to agreeableness ( Mairesse et al . , 2007 ) , and negative emotion words are useful cues to deceptive speech ( Newman et al . , 2003 ) . The number of words in a sentence can be a useful feature for extroverted personality ( Mairesse et al . , 2007 ) . Finally , dialog features such as the presence of disfluencies can inform listeners about speakers ' problems in utterance planning or about confidence ( Brennan and Williams , 1995 ; Brennan and Schober , 2001 ) . Our goal is to see whether cues of this sort are useful in detecting particular elements of conversational style and social intention ; whether a speaker in a speed-dating conversation is judged by the interlocutor as friendly , awkward , or flirtatious . The results presented here should be regarded with some caution . The sample is not a random sample of English speakers or American adults , and speed dating is not a natural context for expressing every conversational style . Therefore , a wider array of studies across populations and genres would be required before a more general theory of conversational styles is established . On the other hand , the presented results may under-reflect the relations being captured . The quality of recordings and coarse granularity ( 1 second ) of the time-stamps likely cloud the relations , and as the data is cleaned and improved , we expect the associations to only grow stronger . Caveats aside , we believe the evidence indicates that the perception of several types of conversational style have relatively clear signals across genders , but with some additional gender contextualization . Both genders convey flirtation by laughing more , speaking faster , and using higher and more variable pitch . Both genders convey friendliness by laughing more , and using collaborative completions . However , we do find gender differences ; men asl more questions when ( labeled as ) flirting , women ask fewer . Men labeled as flirting are softer , but women labeled as flirting are louder . Women flirt-ing swear more , while men are more likely to use sexual vocabulary . Gender differences exist as well for the other variables . Men labeled as friendly use you while women labeled as friendly use I. Friendly women are very disfluent ; friendly men are not . While the features for friendly and flirtatious speech overlap , there are clear differences . Men speaker faster and with higher f0 ( min ) in flirtatious speech , but not faster and with lower f0 ( min ) in friendly speech . For men , flirtatious speech involves more questions and repair questions , while friendly speech does not . For women , friendly speech is more disfluent than flirtatious speech , and has more collaborative style ( completions , repair questions , appreciations ) . We also seem to see a model of collaborative conversational style ( probably related to the collaborative floor of Edelsky ( 1981 ) and Coates ( 1996 ) ) , cued by the use of more collaborative completions , repair questions and other questions , you , and laughter . These collaborative techniques were used by both women and men who were labeled as friendly , and occurred less with men labeled as awkward . Women themselves displayed more of this collaborative conversational style when they labeled the men as friendly . For women only , collaborative style included appreciations ; while for men only , collaborative style included overlaps . In addition to these implications for social science , our work has implications for the extraction of meaning in general . A key focus of our work was on ways to extract useful dialog act and disfluency features ( repair questions , backchannels , appreciations , restarts , dispreferreds ) with very shallow methods . These features were indeed extractable and proved to be useful features in classification . We are currently extending these results to predict date outcomes including ' liking ' , extending work such as Madan and Pentland ( 2006 ) .
Methods to extract social meanings such as engagement from speech remain unknown while it is important in sociolinguistics and to develop socially aware computing systems.
They create a spoken corpus from conversations in speed-dating and perform analysis using extracted dialogue features with a focus on genders.
They found several gender dependent and independent phenomena in conversations related to the speed of speaking, laughing or asking questions.