We propose using a second
regularization loss between the two networks in order to also enforce
consistency between the two graph-based representations (i.e., over
video shots and screenplay scenes). The purpose of this loss is
twofold: to improve TP predictions for the two networks, as shown in
previous work on contrastive representation learning
[1]}, [2]}, [3]}, and
also to help learn more accurate connections between shots (recall
that the shot-based graph serves as input to our trailer generation
algorithm; Section REF ). In comparison with
screenplay scenes, which describe self-contained events in a movie,
video shots are only a few seconds long and rely on surrounding
context for their meaning. We hypothesize that by enforcing the graph
neighborhood for a shot to preserve semantics similar to the
corresponding screenplay scene, we will encourage the selection of
appropriate neighbors in the shot-based graph.
