Most relevant for us is the work of <cite>Belinkov and Bisk (2018)</cite> , who evaluated on natural noise obtained from Wikipedia edit histories (e.g., Max and Wisniewski, 2010) . <cite>They</cite> find that robustness to natural noise can be obtained by training on the same noise model, but that (a) training on synthetic noise does not yield robustness to natural noise at test time, and (b) training on natural noise significantly impairs performance on clean text. In contrast, we show that training on the right kind and the right amount of synthetic noise can yield substantial improvements on natural noise at test time, without significantly impairing performance on clean data. Our ablation results suggest that deletion and insertion noise -which were not included by <cite>Belinkov and Bisk</cite> -are essential to achieving robustness to natural noise.