Data augmentation is one of the most effective approaches for improving the accuracy of modern machine learning models, and it is also indispensable to train a deep model for meta-learning. However, most current data augmentation implementations applied in meta-learning are the same as those used in the conventional image classification. In this paper, we introduce a new data augmentation method for meta-learning, which is named as ``Task Level Data Augmentation'' (referred to Task Aug). The basic idea of Task Aug is to increase the number of image classes rather than the number of images in each class. In contrast, with a larger amount of classes, we can sample more diverse task instances during training. This allows us to train a deep network by meta-learning methods with little over-fitting. Experimental show that our approach achieves state-of-the-art performance on miniImageNet, CIFAR-FS, and FC100 few-shot learning benchmarks. Once paper is accepted, we will provide the link to code. Although the machine learning systems have achieved a human-level ability in many fields with a large amount of data, learning from a few examples is still a challenge for modern machine learning techniques. Recently, the machine learning community has paid significant attention to this problem, where few-shot learning is the common task for meta-learning (e.g., ; ; ;). The purpose of few-shot learning is to learn to maximize generalization accuracy across different tasks with few training examples. In a classification application of the few-shot learning, tasks are generated by sampling from a conventional classification dataset; then, training samples are randomly selected from several classes in the classification dataset. In addition, a part of the examples is used as training examples and testing examples. Thus, a tiny learning task is formed by these examples. The meta-learning methods are applied to control the learning process of a base learner, so as to correctly classify on testing examples. Data augmentation is widely used to improve the training of deep learning models. Usually, the data augmentation is regarded as an explicit form of regularization;;. Thus, the data augmentation aims at artificially generating the training data by using various translations on existing data, such as: adding noises, cropping, flipping, rotation, translation, etc. The general idea of data augmentations is increasing the number of data by change data slightly to be different from original data, but the data still can be recognized by human. The new data involved in the classes are identical to the original data. However, the minimum units of meta-learning are the tasks rather than data. Increasing the data of original class cannot increase the types of task instances. Therefore, "Task Aug" increases the data that can be clearly recognized as the different classes as the original data. With novel classes, the more diverse task instances can be generated. This is important for the meta-learning, since metalearning models must predict unseen classes during the testing phase. Therefore, a larger number of classes is helpful for models to generate task instances with different classes. In this work, the natural images are augmented by being rotated 90, 180, 270 degrees (we show examples in Figure 1). We compare two cases, 1) the new images are converted to the classes of original images and 2) the new images are separated to the new classes. The proposed method is evaluated by experiments with the state of art meta-learning . The experimental analysis shows that Task Aug can reduce over-fitting and improve the performance, while the conventional data augmentation (referred to Data Aug) of rotation, which converts the novel data into the classes of original data, does not improve the performance and even causes the worse . In the comparative experiments, Task Aug achieves the best accuracy of the meta-learning methods applied. Besides, the best of our experiments exceed the current state-of-art over a large margin. Meta-learning involves two hierarchies learning processes: low-level and high-level. The low-level learning process learns to deal with general tasks, often termed as the "inner loop"; and the highlevel learning process learns to improve the performance of a low-level task, often termed as the "outer loop". Since models are required to handle sensory data like images, deep learning methods are often applied for the "outer loop". However, the machine learning methods applied for the "inner loop" are very diverse. Based on different methods in the "inner loop", meta-learning can be applied in image recognition;;;; , image generation;; , reinforce learning; , and etc. This work focuses on few-shot learning image recognition based on meta-learning. Therefore, in the experiment, the methods applied in the "inner loop" are able to classify data, and they are K-nearest neighbor (KNN), Support Vector Machine (SVM) and ridge regression, respectively;;. Previous studies have introduced many popular regularization techniques to few-shot learning from deep learning, such as weight decay, dropout, label smooth, and data augmentation. Common data augmentation techniques for image recognition are usually designed manually and the best augmentation strategies depend on dataset. However, in natural color image datasets, random cropping and random horizontal flipping are the most common. Since the few-shot learning tasks consist of natural color images, the random horizontal flipping and random cropping are applied in few-shot learning. In addition, color (brightness, contrast, and saturation) jitter is often applied in the works of few-shot learning;. Other data augmentation technologies related to few-shot learning include generating samples by few-shot learning and generating samples for few-shot learning. The former tried to synthesize additional examples via transferring, extracting, and encoding to create the data of the new class, that are intra-class relationships between pairs of reference classes' data instances;. The later tried to apply meta-learning in a few-shot generation to generate samples from other models. In addition to these two types of studies, the data augmentation technology most closed to the new proposed approach is applied to Omniglot dataset, which consists of handwritten words. They created the novel classes by rotating the original images 90, 180 and 270 degrees. However, this approach cannot be applied for the natural color image directly, and we will explain the reasons and the solutions in Section 3. We adopt the formulation purposed by to describe the N -way K-shot task. A few-shot task contains many task instances (denoted by T i), each instance is a classification problem consisting of the data sampled from N classes. The classes are randomly selected from a classes set. The classes set are split into M tr, M val and M test for a training class set C tr, a validation classes set C val, and a test classes set C test. In particular, each class cannot overlap others (i.e., the classes used during testing are unseen classes during training). Data is randomly sampled from C tr, C val and C test, so as to create task instances for training meta-set S tr, validation meta-set S val, and test meta-set S test, respectively. The validation and testing meta-sets are used for model selection and final evaluation, respectively. The data in each task instance, T i, are divided into training examples D tr and validation examples D val. Both of them only contains the data from N classes which sampled from the appropriate classes set randomly (for a task instance applied during training, the classes form a subset of the training classes set C tr). In most settings, the training set.. K} consists of K data instances from each class, this processing usually called as a "shot". The validation set, D val, consists of several other data instances from the same classes, this processing is usually called as a "query". An evaluation is provided for generalization performance on the N classification task instance D tr. Note that: the validation set of a task instance D val (for optimizing model during "outer loop") is different from the held-out validation classes set C val and meta-set S val (for model selection). This work is to increase the size of the training classes set, M tr, by rotating all images within the training classes set with 90, 180, 270 degrees. The size, M tr, is increased for three times. In the Omniglot dataset consisting of handwritten words , this approach works well, since it can rotate a handwritten word multiple of 90 degrees and treat the new one as another word; in addition, it is really possible that the novel word is similar to some words, which are not included in the training classes but existed. However, for natural images, it is not the same cause. For examples, the images in the third line of Figure 1 are difficult to identify which images are rotated. Moreover, the images are rarely rotated in the photos taken by humans. Despite the two problems, the fundamental features of the novel images can provide useful information. We assign novel classes that contain smaller weights than the original classes, so as to make models prioritize learning the features of the original classes, and make the features of the novel classes as a supplement to prevent the augmented data from taking up large capacity in the model. The smaller weights are implemented in two ways, 1) lower probability and 2) delay selecting the novel classes. For a class in a task instance, the probability of the class coming from the novel classes is p, and the probability coming from the original classes is 1 − p. Besides, The initial p is set to 0, then linearly rises from 0 to p max after generating T task instances. The max probability p max is set lower than the proportion of the novel classes in all classes to make each novel class have a lower probability than each original class. The whole process of Task Aug on a classes set is summarized in Algorithm 1 and Figure 2. In this work, we also compare the methods with the training protocol with ensemble method in addition to the standard training protocol, which choosing a model by the validation set. The training protocol with an ensemble method use the models with different training epoch to Algorithm 1 Task Level Data Augmentation. Require: Classes set C = {c 1, c 2, . . ., c M}; Max possibility for Task Aug p max; The delay to Task Aug T; The current count t; The number of ways, shots and queries N, K, H 1: Rotate all x ∈ {x|(x, y) ∈ D} 90r degrees 17: an ensemble model, in order to better use the models obtained in a single training process, and this approach has been proved to be valid for meta-learning by experiments. We adopt this ensemble method. However, unlike and that we did not use cyclic annealing for learning rate and any methods to select models. We directly took the average of the prediction of all models, which are saved according to an interval of 1 epoch. In Section 4, the methods with this ensemble approach are marked by "+ens". We evaluate the proposed method on few-shot learning tasks. In order to ensure fair, both the of baseline and Task Aug were run in our own environment. The comparative experiment is designed to answer the following questions: Is Task Aug able to improve the performance of meta-learning? How much should the probably for the novel classes be set? Will converting the novel data into the classes of the original data cause worse , which are generated by being rotated 90, 180, 270 degrees?, we used ResNet-12 network in our experiments. The ResNet-12 network had four residual blocks which contains three 3 × 3 convolution, batch normalization and Leaky ReLU with 0.1 negative slope. One 2 × 2 max-pooling layer is used for reducing the size of the feature map. The numbers of the network channels were 64, 160, 320 and 640, respectively. DropBlock regularization is used in the last two residual blocks, the conventional dropout is used in the first two residual blocks. The block sizes of DropBlock were set to 2 and 5 for CIFAR derivatives and ImageNet derivatives, respectively. In all experiments, the dropout possibility was set to 0.1. The global average pooling was not used for the final output of the last residual block. For ProtoNets, we did not use a higher way for training than testing like. Instead, the equal number of shot and way were used in both training and evaluation, and its output multiplied by a learnable scale before the softmax following;, For M-SVM, we set training shot to 5 for CIFAR-FS; 15 for FC100; and 15 for miniImageNet; regularization parameter of SVM was set to 0.1; and a learnable scale was used following. We did not use label smoothing like, because we did not find that label smoothing can improve the performance in our environment. This was also affirmed from the author's message on GitHub, that Program language packages and environment might affect of the meta-learning method. For R2-D2, we set the same training shot as for M-SVM, and used a learnable scale and bias following. It was different from we used a fixed regularization parameter of ridge regression which was set to 50 because has confirmed that making it learnable might not be helpful. Last, for all methods, each class in a task instance contained 6 test (query) examples during training and 15 test (query) examples during testing. Stochastic gradient descent (SGD) was used. , we set weight decay and Nesterov momentum to 0.0005 and 0.9, respectively. Each mini-batch contained 8 task instances. The meta-learning model was trained for 60 epochs, and 1000 mini-batchs for each epoch. We set the initial learning rate to 0.1, then multiplied it by 0.06, 0.012, and 0.0024 at epochs 20, 40 and 50, respectively, as in. The , which are marked by "ens" were used the 60 models saved after each epoch to become an ensemble model. For the final epoch, the training classes set was augmented by the validation classes set. We chose the model at the epoch where we got the best model during training on the training classes set only. The of the final run are marked by "+val" in this subsection. For data augmentation, we adopted random random crop, horizontal flip, and color (brightness, saturation, and contrast) jitter data augmentation following the work of;. We set p max to 0.5 for CIFAR-FS and FC100; 0.25 for miniImageNet; and T was set to 80000 for all experiments. The FC100 are also derived from CIFAR-100 , and the 100 classes are grouped into 20 superclasses. The training, validation, and testing classes contain 60 classes from 12 superclasses, 20 classes from 4 superclasses, and 20 classes from 4 superclasses, respectively. The target is to minimize the information overlap between classes to make it more challenging than current few-shot classification tasks. Same as CIFAR-FS, there are 600 nature color images of size 32 × 32 in each class. Results. In Table 1, we compare our with the previous studies, and the table shows that the highest accuracies of our experiments exceeded the current state-of-art accuracies from 3% to 5%. Besides, Table 2 and Table 3 summarize the on the CIFAR-FS and FC100 5-way tasks, and in most cases our method rises accuracy by 0.5%-3%. Figure 3: The accuracies (%) on meta-test sets with varying probability p max for the novel classes. The 95% confidence interval is denoted by the shaded region. In general, the performance of Task Aug on most of the regimes is better than Data Aug and baseline. To identify whether the rotation multi 90 degrees for Task Aug is better than that for Data Aug, we analyzed the experiment on CIFAR-FS and miniImageNet. The linear rising of p was also used for Data Aug, and T = 80000 for both Task Aug and Data Aug. In the analysis, the training classes set was not augmented by the validation classes set. As shown in Figure 3, we observed that: with p max, the accuracy rises at first, reaches the peaks between 0.25 and 0.5, then declines and reaches baseline when p max = 0.75 at the end, which is the proportion of the novel classes in all classes. On the other hand, the rotation multi 90 degrees for Data Aug can not improve or even cause worse performance.
We propose a data augmentation approach for meta-learning and prove that it is valid.