video 2 net Extracting dynamic interaction networks from multi person discussion videos
Networks in multi person discussions, such as meetings and games, elicit interpersonal and group dynamics that reveal liking, trust, and deception relations. Extracting interaction networks from videos of group discussions is challenging because discussions are highly dynamic and all participants' behaviors are inter dependent. Here we present an algorithm, called video2net, a multi layered collective classification model to extract dynamic interaction networks from multi person discussion videos. Every participant is modeled using a separate classifier that predicts the participant's focus of attention every 1/3rd second. video2net incorporates two crucial dependencies in the model: temporal dependency and interparticipant dependency. First, video2net ensures temporal smoothness of a participant's prediction by using the participant's output from the previous time step as an input in the current time step. Second, video2net incorporates inter participant dependencies by creating a multi layer collective classification model such that the predictions of all participants' classifiers are used as inputs to every participant's classifier. We evaluate the performance of video2net on a novel dataset of 5 videos (total of 35 participants, 109 minutes, and 7604 total ground truth labels) on the popular Resistance game in supervised prediction. We compare video2net with three state of the art algorithms. We show that video2net outperforms the best algorithm by 11.5% accuracy in predicting a participant's focus of attention in the next 1/3rd second. More interestingly, we show that video2net consistently performs the best in longer term predictions up to 3 seconds into the future. Finally, we conduct an ablation study to demonstrate that the collective classification component of video2net contributes the most the performance. Altogether, this work presents a computational approach to extract dynamic interaction networks from discussion videos and paves the way towards network analysis of discussions. ACM Reference Format: Chongyang Bai, Srijan Kumar, Jure Leskovec, Miriam Metzger, Jay Nunamaker, and V.S. Subrahmanian. 2018. video2net: Extracting dynamic interaction networks from multi person discussion videos. In Proceedings of ACM Conference (Conference'17) ACM, New York, NY, USA, 7 pages. https:/doi.org/10.1145/nnnnnnn.nnnnnnn Conference'17, July 2017, Washington, DC, USA 2018. ACM ISBN 978 x xxxx xxxx x/YY/MM. $15.00 https:/doi.org/10.1145/nnnnnnn.nnnnnnn