In contrast to previous work <cite>(Cífka and Bojar, 2018)</cite> , we demonstrate that there is a correlation between translation performance and trainable downstream tasks when adjusting the size of the intermediate layer.