For our reverse (denoising) model \(p_\xi (x_t, t)\) , we adopt a vanilla bidirectional transformer architecture [1]} with relative positional embeddings as described in [2]}. Our six-dimensional input is linearly upscaled to the model's embedding dimension (\(d=384\) ). To incorporate the timestep \(t\)  into this model, we generate random Fourier feature embeddings [3]} as done in [4]} and add these embeddings to each upscaled input. To convert the transformer's final per-position representations to our six outputs, we apply a regression head consisting of a densely connected layer, followed by GELU activation [5]}, layer normalization, and finally a fully connected layer outputting our six values. We train this network with the AdamW optimizer [6]} over 10,000 epochs, with a learning rate that linearly scales from 0 to \(5 \times 10^{-5}\)  over 1,000 epochs, and back to 0 over the final 9,000 epochs. Validation loss appears to plateau after \(\approx \)  1,400 epochs; additional training does not improve validation loss, but appears to lead to a poorer diversity of generated structures. We thus take a model checkpoint at 1,484 epochs for all subsequent analyses.
