We find that this setting outperforms the originally published BERT BASE results and that removing the NSP loss matches or slightly improves downstream task performance, in contrast to<cite> Devlin et al. (2019)</cite> .