Conversation Group Detection With Spatio-Temporal Context
In this work, we propose an approach for detecting conversation groups in
social scenarios like cocktail parties and networking events, from overhead
camera recordings. We posit the detection of conversation groups as a learning
problem that could benefit from leveraging the spatial context of the
surroundings, and the inherent temporal context in interpersonal dynamics which
is reflected in the temporal dynamics in human behavior signals, an aspect that
has not been addressed in recent prior works. This motivates our approach which
consists of a dynamic LSTM-based deep learning model that predicts continuous
pairwise affinity values indicating how likely two people are in the same
conversation group. These affinity values are also continuous in time, since
relationships and group membership do not occur instantaneously, even though
the ground truths of group membership are binary. Using the predicted affinity
values, we apply a graph clustering method based on Dominant Set extraction to
identify the conversation groups. We benchmark the proposed method against
established methods on multiple social interaction datasets. Our results showed
that the proposed method improves group detection performance in data that has
more temporal granularity in conversation group labels. Additionally, we
provide an analysis in the predicted affinity values in relation to the
conversation group detection. Finally, we demonstrate the usability of the
predicted affinity values in a forecasting framework to predict group
membership for a given forecast horizon.