Convolutional Neural Networks [1]} are a popular choice of encoder for encoding information from the audio input. Convolutional Neural Networks has done well for tasks such as sound event detection, audio tagging, by being able to extract local features using convolutional filters. This has also led to previous work in Automated Audio Captioning also similarly using a single convolutional neural network as the encoder. However, we argue that this is insufficient for Automated Audio Captioning, as this task requires the model to capture not just local events (e.g a loud honk) that occur in narrower timeframes, but also overarching events (e.g. car engine running in the background) that can span the whole duration of the audio. To combat this, we propose to append the transformer encoder [2]} to the convolutional encoder. The transformer encoder consists of several layers of multi-head self attention and feedforward layers, and many variants of the transformer have broken the state of the art on Natural Language Processing benchmarks [3]}. We hypothesise that the combination of both the convolutional encoder and transformer encoder, along with transfer learning, will allow the model to perform much better.
