We parameterize the inversion model using the method described in Section 4 and select T5-base (Raffel et al., 2020) as our encoder-decoder backbone, which has \(222M\) parameters. We set the maximum sequence length to \(64\) for all experiments. We train models for \(100\) epochs with Adam optimizer with a learning rate of \(2e-4\). We use a constant learning rate with linear warmup over the first \(25,000\) training steps. We train in bfloat16 precision.