\begin{table}
\begin{tabular}{c|c c c|c c} \hline \hline Base & Accuracy & Correct GCD & Correct predictions \\ \hline
10 & 85 & 13 & \{1,2,4,8,16\} & \{1,5,25\} \\
420 & 97 & 38 & \{1,2,4,8,16\} & \{1,3,9\} & \{1,5,25\} \\
1000 & 94 & 22 & \{1,2,4,8,16\} & \{1,5,25\} & \{1,3\} \\
2017 & 85 & 4 & \{1,2\} & \{1,3\} & \\
2023 & 91 & 16 & \{1,2,4\} & \{1,3\} & \{1,7\} & \{1,17\} \\
4913 & 93 & 17 & \{1,2,4\} & \{1,3\} & \{1,5\} & \{1,17\} \\ \hline \hline \end{tabular}
\end{table}
Table 1: **Initial performances.**\(4\)-layer transformers trained to predict GCD, on 300 million examples. Our _test_ set only contains GCD up to \(100\), and accuracy is computed on a reweighted test with equal occurance of each GCD. Thus, the Correct GCD lists all those that can be formed from the correct predictions by forming products across the sets (within the first 100 GCD). We freeze the 0th generation model at this stage and use its prediction to generate synthetic data. For each GCD outside the set of its correct predictions, the model will predict the largest GCD it has learned that divides the ground truth.