Our feature extractor network is a larger version of the adapted VGG proposed in [1]}. This cnn comprises four convolution blocks, each of which contains two concatenated convolutional layers followed by a max pooling with a \(2\times 2\)  stride. Hence given a spectrogram of \(N\)  frames, the VGG performs a down-sampling reducing its output into a sequence of \(N/16\)  representations. The output of the VGG \(h\in \mathbb {R}^{M\times N/16\times D^{\prime }}\)  is a set of \(M\)  feature maps with \(N/16\times D^{\prime }\)  dimension. These feature maps are concatenated into a unique vector sequence. This reshaped sequence of hidden states can now be defined as \(h \in \mathbb {R}^{N/16\times D}\) , where \(D=MD^{\prime }\)  corresponds to the hidden state dimension.
