Existing works on zero-shot semantic segmentation [1]}, [2]}, [3]}, [2]} are usually based on fully convolutional networks (FCN) [5]}, an architecture widely used for fully supervised semantic segmentation. In FCN, semantic segmentation is modeled as a pixel classification problem, where a linear classifier is applied on each pixel feature to produce the classification results, with each column of the linear classifier weight matrix representing each category. In zero-shot semantic segmentation, that classifier column of a novel category cannot be learnt from training data since there is no single pixel annotation of this category. Thus usually a generalizable metric between the category semantic and pixel features is learnt to seize such zero-shot capability [2]}, [7]}. However, this pixel-level vision-category correspondence/metric is difficult to be well learnt given only tens or hundreds of categories involved by the semantic segmentation tasks.
