Does the unit of reversal matter?To understand the effect of segment granularity when reversing sequences, we evaluate the performance of the following training methods on the fictitious celebrities task: standard finetuning, token and word reversal finetuning, entity-preserving reversal finetuning, and random segment reversal finetuning with varying \(k\) as described in Section 2. The results are summarized in Table 9. In general, we find that reversing at a fine-grained level such as token or word level does not significantly help to resolve the reversal curse, and only improves performance on the reverse tasks by 2-3%. Preserving entities during reversal makes it possible to predict names, but not descriptions. This indicates a close relation between the unit of reversal training and the target "concepts" (e.g. names, descriptions) of the reversal task. Similarly, the random segment reversal performs poorly at predicting descriptions when the segment length limit is set lower than the typical length of a description. The results from Section 3.1 also support this hypothesis.