Multi-Task Representation Learning with Common Embedding:  Consider a collection of tasks \(\mathcal {T}_1, \mathcal {T}_2, \cdots , \mathcal {T}_n\) , and consider the problem of learning an permutation-invariant representation feature space among all the tasks. Partition a neural network into two pieces, an embedding network and a task-specific network. The layers in the embedding network learn an invariant representation feature space that is common to all the tasks, while the task-specific layers learn features specific to that task. During training, the goal is to find a good set of representation features so that new tasks can be learned quickly. Such a network has been shown to generalize well for unseen tasks, as everything but the last layer is learned across all the tasks. This idea has been used in both few-shot learning, where the task-specific network is just the last layer [1]}, and in continual learning [2]}.
