Identical models are trained via supervised learning (SL) and reinforcement learning (RL)We show only the greedy rollout baseline for clarity. Critic baseline results are available in Appendix .
Figure REF  illustrates that, when using greedy decoding during inference, RL models perform better on the training size as well as on larger graphs.
Conversely, SL models improve over their RL counterparts when performing beam search or sampling.
In Appendix , we find that the rollout baseline, which encourages better greedy behaviour, leads to the model making very confident predictions about selecting the next node at each decoding step, even out of training size range.
In contrast, SL models are trained with teacher forcing, i.e. imitating the optimal solver at each step instead of using their own prediction.
This results in less confident predictions and poor greedy decoding, but makes the probability distribution more amenable to beam search and sampling techniques, as shown in Figures REF .
Our results advocate for tighter coupling between the training and inference phase of learning-driven TSP solvers, mirroring recent findings in generative models for text [1]}, [2]}.
