**Modular** is a newly proposed parameter-expansion method specifically for encoder-decoder models which freezes the original, pretrained encoder while adding a new, randomly initialized encoder that is updated during continued pretraining. For the newly added encoder, we vary the size to _T5-small_ while keeping the size of the original encoder and decoder to be _T5-large_.