Structure of the reasoning network. For the EVVE dataset, we set a softmax layer as the decision network. The video imprint is generated based on the ResNet-50 [1]} model and its imprint descriptors are first reduced to 256 dimension with PCA-whitening before being fed to the reasoning network (RNet). For the CCV and MED14 datasets, we add a fully connected layer in front of the softmax layer as the decision network for better performance. Besides the ResNet-50 model, we also evaluate the framework with VGG (16 layers) [2]} model on these two datasets. The dimension of the imprint descriptors is set to 1024 and 512 for ResNet-50 and VGG, respectively. For all the datasets, the internal vectors \({b}_{\bf i}\)  and \({m}_{\bf i}\)  have the same dimensions with the input imprint descriptors.
<TABLE><TABLE>