Parameter-expansionProgressive Neural Networks (PNN) (Rusu et al., 2016) is one of the earliest parameter-expansion/sharing approaches that introduce new sets of parameters for each new task where previous parameters are frozen and can be connected via lateral connections allowing for _positive forward transfer_. PNN not only prevents _negative backward transfer_ but also surpassed the previous pretrain-finetune approach in terms of _positive forward transfer_ in some tasks.