Our experimental results show that our novel multi-task learning framework leads to a new state-of-the-art accuracy score of 91.39% for TAG supertagging on the Penn Treebank dataset (Marcus et al., 1993; Chen et al., 2006) which is a significant improvement over the previous multi-task result for supertagging that combines supertagging with graph-based parsing <cite>(Kasai et al., 2018)</cite> .