Our full model also utilizes skip connections. The idea behind skip connections is to feed the output of a layer directly to a deeper layer while skipping a few layers in between. Skip connections provide some known advantages: 1) creation of an alternative gradient path, 2) prevention of the gradient vanishing problem, and 3) assistance with model convergence. Previous work has shown the effectiveness of skip connections in deep computer vision networks [1]}, [2]}. In our model, one skip connection is taken from the MLP encoder and is added to the output of the transformer before feeding to Head Win. The second skip connection is taken from the output of the CNN encoder and added to the output of the transformer to be fed to Head Build.
