Inspired by the in vivo protein folding process, we introduce a generative model that acts on the inter-residue angles in protein backbones instead of on Cartesian atom coordinates (Figure REF ). This treats each residue as an independent reference frame, thus shifting the equivariance requirements from the neural network to the coordinate system itself. For generation, we use a denoising diffusion probabilistic model (diffusion model, for brevity) [1]}, [2]} with a vanilla transformer parameterization without any equivariance constraints. Diffusion models train a neural network to start from noise and iteratively “denoise” it to generate data samples. Such models have been highly successful in a wide range of data modalities from images [3]}, [4]} to audio [5]}, [6]}, and are easier to train with better modal coverage than methods like generative adversarial networks (GANs) [7]}, [8]}. We present a suite of validations to quantitatively demonstrate that unconditional sampling from our model directly generates realistic protein backbones – from recapitulating the natural distribution of protein inter-residue angles, to producing overall structures with appropriate arrangements of multiple structural building block motifs. We show that our generated backbones are diverse and designable, and are thus biologically plausible protein structures. Our work demonstrates the power of biologically-inspired problem formulations and represents an important step towards accelerating the development of new proteins and protein-based therapies.
