@inproceedings{
charton2023learning,
title={Learning the greatest divisor - Explainable predictions in transformers},
author={Francois Charton},
booktitle={The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23},
year={2023},
url={https://openreview.net/forum?id=8eeIlrtluJ}
}
We train small transformers to calculate the greatest common divisor (GCD) of two positive integers, and show that their predictions are fully explainable. 
During training, models learn a list $\mathcal D$ of divisors, and predict the largest element of $\mathcal D$ that divides both inputs. 
We also show that training distributions have a large impact on performance. Models trained from uniform operands only learn a handful of GCD (up to $38$ out of $100$). 
Training from log-uniform operands boosts performance to $73$ correct GCD, and training from a log-uniform distribution of GCD to $91$.
Learning the greatest divisor - Explainable predictions in transformers
Francois Charton