Jigsaw Content Prediction.  Jigsaw's inclusion is inspired by a recent work [1]} that explores the potential of self-supervised tasks in architecture search. We follow [2]} to design the self-supervised task Jigsaw. The input image is divided into 9 patches and shuffled according to one of 1000 preset permutations. The goal of this task is to classify which permutation is used. We use a Siamese network to extract the feature map of each of the 9 image tiles and concatenate them. We apply random flip, color jitter, and random grayscale with a probability of 0.3 for data augmentation. Referring to the settings of Taskonomy, each network is trained for 10 epochs since Jigsaw tasks converge very quickly. The decoder, optimizer, and learning rate scheduler settings are the same as above.
