Diffusion models were also applied to natural language processing tasks. [1]} proposed a multinomial diffusion process for categorical data and applied it to language modeling. [2]} generalize the multinomial diffusion process with Discrete Denoising Diffusion Probabilistic Models (D3PMs) and improve the generated results for the text8 and One Billion Word (LM1B) datasets.
<FIGURE>