The simplest MTL approach is hard parameter sharing, which shares hidden representations across different tasks, and only the last prediction layers are special for different tasks [1]}.
However, hard parameter sharing suffers from conflicts among tasks, due to the simple sharing of representations.
To deal with this problem, some approaches propose to learn weights of linear combinations to fuse hidden representations in different tasks, such as Cross-Stitch Network [2]} and Sluice Network [3]}.
However, in different samples, the weights of different tasks stay the same, which limits the performances of MTL.
This inspires the research on applying gating structures in MTL [4]}[5]}[6]}[7]}.
Mixture-Of-Experts (MOE) first proposes to share and combine several experts through a gating network [7]}.
Based on MOE, to make the weights of different tasks varying across different samples and to improve the performances of MTL, Multi-gate MOE (MMOE) [4]} proposes to use different gates for different tasks.
Progressive Layered Extraction (PLE) further extends MMOE, and incorporates multi-level experts and gating networks [5]}.
Besides, attention networks are also utilized for assigning weights of tasks according to different feature representations [11]}[12]}.
<TABLE>