We initially construct a fully-connected graph by computing the
pairwise similarity between all multimodal shot vectors
\(h_1, h_2, \dots , h_M\) . In order to avoid dense connections in the
graph, which lead to worse contextualization and computational
overhead, we select a small but variable-length neighborhood per shot
and we sparsify the graph accordingly. During sparsification, we also
constrain the adjacency matrix of the graph to be upper triangular
(i.e., allowing only future connections between shots). For more
details about the graph sparsification, see
[1]} and the Appendix. We also contextualize
all shots with respect to the whole movie using a transformer
encoder [2]} and additionally encode each shot
via a one-layer Graph Convolution Network (GCN;
[3]}, [4]}, [5]}). Finally,
we combine the global (transformer-based) and local (graph-based)
representations for each shot in order to compute the probability
\(p(y_{it} |h_i, \mathcal {F}, \theta _1)\) . The network is trained by
minimizing the binary cross-entropy loss. After training, we use the
learned sparse graph \(\mathcal {G}\)  as input to
Algorithm REF .
<FIGURE>