Another approach to condition one modality with the other is the conditional normalization (CN). Instead of focusing attention on a particular region of space or a particular time window, the CN highlights some feature maps based on a given input. Various forms of CN have proven to be highly effective across a number of domains and modalities: image stylization [1]}, speech recognition [2]}, visual question answering [3]} and audio question answering [4]}. As presented in our previous work [5]}, we propose to condition the audio path with the visual information by using the Feature-wise Linear Modulation (FiLM) method [6]}. The FiLM layer highlights audio feature maps based on visual information.
