In this paper, we introduced a simple yet effective training method to help remedy the reversal curse in LLMs. Our reverse training works by first segmenting the input sequence into chunks and then reversing the ordering of chunks, but leaves the word-ordering in each chunk intact. A chunk can be a token, a word, an entity name, or a random number of tokens. The model is then trained on both the original sequences, and this reversed data. We evaluated on a symbolic reverse task and a reversing biography task that both demonstrated the necessity of preserving word-ordering within chunks. Next, we applied our reverse training to the realistic setting of LLM pre-training, which minimized the reversal curse on real-world knowledge. Evaluations on common benchmark tasks reveal that reverse training (particularly at the word level) during pre-training does not interfere with the forward prediction ability of LLMs, and actually improves metrics in the data-bound (rather than compute-bound) setting compared to standard training. When our method is applied to finetuning on fictitious facts, prediction accuracy rose from 0% to 70-100%. The reversal curse is a serious flaw in how LLMs acquire knowledge and reverse training opens a new promising direction in resolving it.