Learning to Learn Single Domain Generalization Supplementary Materials
Task models: We design specific task models and employ different training strategies for the three datasets according to their characteristics. In Digits dataset, the model architecture is conv poolconv pool fc fc softmax. There are two 5x 5 convolutional layers with 64 and 128 channels respectively. Each convolutional layer is followed by a max pooling layer with the size of 2 x 2. The size of the two Fully Connected (FC) layers is 1024 and the size of the softmax layer is 10. In CIFAR 10 C [2] we use Wide Residual Network (WRN) [11] with 16 layers and the width is 4. The first layer is a 3x3 convolutional layer. It converts the original image with 3 channels to feature maps of 16 channels. Then the features go through three groups of 3x3 convolutional layers. Each group consists of two blocks and each block is composed of two convolutional layers with the same number of channels. And their channels are {64, 128, 256} respectively. Each convolutional layer is followed by batch normalization (BN) [3] An average pooling layer with the size of 8 x 8 is appended to the output of the third group. Finally, a softmax layer with the size of 10 predicts the distribution over classes. In SYTHIA [7] we use FCN 32s [5] with the backbone of ResNet 50 [1] The model begins with ResNet 50. 1x1 convolutional layer with 14 channels is appended to predict scores for each class at each of the coarse output locations. A deconvolution layer is followed to up sample the coarse outputs to the original size through bilinear interpolation. Wasserstein Auto Encodes: We follow [8] to implement WAEs but slightly modifying architectures for the three datasets according to their characteristics. In Digits dataset, the encoder and decoder are built with FC layers. The encoder consists of two FC layers with the size of 400 and 20 respectively. Accordingly, the decoder consists of two FC layers with the size of 400 and 3072 respectively. The discriminator consists of two FC layers with the size of 128 and 1 respectively. The architecture of is shown in Fig. 1 (a) In CIFAR 10 C [2] the encoder begins with four convolutional layers with the channels of {16, 32, 32, 32} And two FC layers with the size of 1024 and 512 are followed. Accordingly, the decoder begins with two FC layers with the size of 512 and 1024 respectively. And four deconvolution layers with the channels of {32, 32, 16, 3} are followed. Each layer is followed by BN [3] except for the final layer of the decoder. The discriminator consists of two FC layers with the size of 128 and 1 respectively. The architecture is shown in Fig. 1 (b) In SYTHIA [7] the encoder begins with three convolutional layers with the channels of {32, 64, 128} And two FC layers with the size of {3840, 512} are followed. Accordingly, the decoder begins with two FC layers with the size of {512, 3840} And three deconvolution layers with the channels of {64, 32, 3} are followed. Each layer is followed by BN [3] except for the final layer of the decoder. The discriminator consists of three FC layers with the size of {512, 512, 1} The architecture is shown in Fig. 1 (c) We apply the Adam optimizer in training WAEs. The learning rate is 0.001 for Digits and 0.0001 for both CIFAR10 C and SYTHIA. The training epoches is 20 for Digits, 100 for CIFAR 10 C [2] and 200 for SYTHIA [7]