論文 / 著書情報
Article / Book Information
TitleInitialization Using Perlin Noise for Training Networks with a LimitedAmount of Data
AuthorNakamasa Inoue, Eisuke Yamagata, Hirokatsu Kataoka
J ournal/Book nameProceedings of ICPR 2020 25th International Conference on PatternRecognition,   ,   ,  pp. 1023-1028
Pub. date2021,  5
DOI https://doi.org/10.1109/ICPR48806.2021.9412955
Copyright(c)2021 IEEE. Personal use of this material is permitted.  Permissionfrom IEEE must be obtained for all other uses, in any current or futuremedia, including reprinting/republishing this material for advertising orpromotional purposes, creating new collective works, for resale orredistribution to servers or lists, or reuse of any copyrighted componentof this work in other works.
Note This file is author (final) version.
Powered by T2R2 (Tokyo Institute Research Repository)
Initialization Using Perlin Noise forTraining Networks with a Limited Amount of Data
Nakamasa Inoue1∗, Eisuke Yamagata1∗, Hirokatsu Kataoka2
1Tokyo Institute of Technology, Tokyo, Japan.2National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan.E-mail: inoue@c.titech.ac.jp, yamagata.e.ab@m.titech.ac.jp, hirokatsu.kataoka@aist.go.jp
Abstract—We propose a novel network initialization methodusing Perlin noise for training image classiﬁcation networks witha limited amount of data. Our main idea is to initialize thenetwork parameters by solving an artiﬁcial noise classiﬁcationproblem, where the aim is to classify Perlin noise samplesinto their noise categories. Speciﬁcally, the proposed methodconsists of two steps. First, it generates Perlin noise sampleswith category labels deﬁned based on noise complexity. Second, itsolves a classiﬁcation problem, in which network parameters areoptimized to classify the generated noise samples. This methodproduces a reasonable set of initial weights (ﬁlters) for imageclassiﬁcation. To the best of our knowledge, this is the ﬁrst workto initialize networks by solving an artiﬁcial optimization problemwithout using any real-world images. Our experiments showthat the proposed method outperforms conventional initializationmethods on four image classiﬁcation datasets.
I. INTRODUCTION
Image classiﬁcation is one of the most important topicsin the ﬁeld of pattern recognition and computer vision, withwide applications such as internet search engines, robotics,and security. Over the last 10 years, many studies have shownthe effectiveness of deep neural networks for various imageclassiﬁcation tasks, including object recognition and actionrecognition. Most neural networks utilize a large-scale datasetfor training because network performance increases withdataset size. For example, deep convolutional networks trainedon the ImageNet dataset [1], which consists of 1.2 millionimages, have been shown to achieve human-level performancein terms of 1,000-class object classiﬁcation accuracy.Training networks with a limited amount of data is a chal-lenging problem because network optimization methods oftenfall into a local solution if the network has many parameters. Arecent trend in attempts to solve this problem is to apply ﬁne-tuning, i.e., to resume training from a pre-trained network. Forexample, for object recognition, networks are often ﬁrst trainedon the ImageNet dataset, and then ﬁne-tuned on another smalldataset. This approach works well because ImageNet is largeenough to obtain reasonable image representations with somevisual ﬁlters in hidden layers. However, its application isoften limited to non-commercial use such as research andeducational purposes. In practice, it is not always easy tocollect such a large number of images.
* indicates equal contribution.
Fig. 1.Network initialization using Perlin noise. The proposed methodconsists of two components, namely a noise sample generator Gη forgenerating Perlin noise samples and a noise label generator Lη for generatingnoise category labels. A network is initialized by solving an artiﬁcial noiseclassiﬁcation problem, where the aim is to classify the generated noise samplesinto their categories.
This paper proposes a network initialization method thatobtains visual ﬁlters without using any real-world images.The proposed method utilizes Perlin noise for initializingnetworks. Perlin noise [2] is a type of noise used in the ﬁeldof computer graphics to render natural objects such as clouds,ﬁre, and stones. Speciﬁcally, the proposed method consistsof two steps. First, it generates Perlin noise samples withcategory labels deﬁned based on noise complexity. Second,it solves a classiﬁcation problem, in which network param-eters are optimized to classify the generated noise samples.Compared with conventional initialization methods such asHe initialization [3] based on Gaussian noise, our methodprovides more complex texture-like visual ﬁlters as initialvalues. We experimentally show that the proposed methodoutperforms conventional initialization methods utilized inmost state-of-the-art image classiﬁcation systems. Further, wedemonstrate that the proposed method contributes to data-efﬁcient pre-training in terms of accuracy improvement usingfewer samples for pre-training.In summary, this paper makes the following contributions:(1) A novel method is proposed for initializing networks,which solves an artiﬁcial classiﬁcation problem withoutusing real-world images. It consists of two components,namely a noise sample generator and a noise label
generator, as shown in Figure 1.
(2) Category deﬁnitions are given for Perlin noise samplesbased on noise complexity. This provides a speciﬁcdeﬁnition of an artiﬁcial classiﬁcation problem to besolved in the proposed method.
(3) Extensive experiments are conducted on four datasets.In addition to a performance comparison with conven-tional initialization methods, we show that the proposedmethod contributes to data-efﬁcient pre-training.
II. RELATED WORK
This section summarizes the network architectures, initial-ization methods, pre-training methods, and datasets discussedin related studies.
A. Network Architectures
Many previous studies have shown the effectiveness ofconvolutional neural networks for image classiﬁcation, whichresults from their ability to learn visual patterns from a largenumber of images. The basic idea of convolution and poolingmechanisms was proposed in the 1990s [4]. Since then, varioustypes of network architectures have been proposed. AlexNet[5], which has seven hidden layers, was the ﬁrst successful ap-plication of large-scale training using natural images. VGGNet[6] and InceptionNet [7] stack more layers to explore deeperarchitectures. ResNet [8] introduces skip connections to avoidthe gradient vanishing problem. ResNet also has extensionssuch as ResNeXt [9], DenseNet [10], and SE-ResNet [11], andis the most widely used architecture for image classiﬁcationtasks such as object recognition [1], action recognition [12],and scene understanding [13].To train these networks, a large number of images areneeded because the networks typically have more than amillion parameters to be optimized. Therefore, training witha limited amount of data is a challenging problem in the ﬁeldof pattern recognition and computer vision.
B. Initialization Methods
In most state-of-the-art optimization methods, the parame-ters of neural networks are initialized independently by utiliz-ing probabilistic distributions such as the Gaussian distributionand the uniform distribution.He initialization [3] is the most popular method, and utilizesa Gaussian distribution with a mean of zero and a variancethat is scaled depending on the number of hidden units toinitialize the weight parameters at each layer. This method iseffective for recent deep convolutional networks with ReLUactivations [14]. Xavier initialization [15] utilizes a uniformdistribution to initialize weights. This is particularly effectivefor networks with a smooth activation function such as thesigmoid or tangent functions. Sparse initialization [16] limitsthe number of non-zero initial weights. It was proposed with aHessian-free optimization method. Other classic initializationmethods include normal initialization, which uses a standardnormal distribution, and zero initialization, which assigns zero
to all weights. Zero initialization is a good choice for biasparameters or parameters that should be sparse.Most of these methods can be viewed as initialization usingGaussian noise or uniform noise. In contrast, our idea is toutilize Perlin noise, a type of noise closer to natural visualpatterns, for image classiﬁcation.
C. Pre-Training Methods
Pre-training is a framework for initializing neural networksusing a set of collected images. The idea is to ﬁrst train anetwork on a large-scale dataset, and then adopt the resultingnetwork as the starting point for training on another dataset.A recent trend for pre-training is to utilize a large-scaledataset such as ImageNet [1], Places 365 [13], and Kinet-ics 400-700 [12], [17]. Because these datasets consist ofmore than a million images with high-quality human-annotatedlabels, networks pre-trained on them have a reasonable set ofvisual ﬁlters at some hidden layers for recognizing naturalobjects from images. This is one of the reasons why trainingfrom pre-trained parameters outperforms training from scratchin terms of image classiﬁcation accuracy. However, the useof these datasets is often limited to research and academicpurposes. Because it is not always easy to collect such alarge number of images, data-efﬁcient pre-training methodsare desirable.Our method is presented here as an initialization methodbecause it does not use any natural images or any manuallyannotated labels. This also distinguishes our method fromsemi-supervised [18] [19] [20] [21] and unsupervised [22]learning. Additionally, it can also be viewed as a new typeof pre-training method because it has a step that solves aclassiﬁcation problem. Notably, we show experimentally thatthe proposed method contributes to data-efﬁcient pre-training.
D. Datasets
Various datasets have been created for evaluating imageclassiﬁcation methods. Each dataset consists of a set of imagesand labels for a speciﬁc task, such as hand-written characterrecognition and object recognition.Examples of hand-written character recognition datasetsinclude MNIST [4] and Omniglot [23]. The MNIST datasetconsists of 70,000 binary images of hand-written digits from 0to 9. The Omniglot dataset consists of about 40,000 grayscaleimages of 1,623 different hand-written characters from 50different alphabets. It was originally proposed as a dataset forone-shot learning to explore new learning methods.Small- and large-scale object recognition datasets have beencreated. Small-scale datasets are often used for evaluatingimage classiﬁcation methods. They include Cifar-10/100 [24],Caltech-101/256 [25], and Pascal VOC [26]. Each of thesedatasets consists of 10-100k natural images in 10-100 objectcategories. The Describable Textures Dataset (DTD) [27]focuses on texture categories related to objects. Large-scaledatasets such as ImageNet [1] and COCO [28] are often usedfor pre-training. They consist of more than a million naturalimages.
Fig. 2. Four steps used to generate Perlin noise. Noise samples are generatedon a 2n × 2m grid.
III. PROPOSED METHOD
In this section, we present the proposed network initializa-tion method that uses Perlin noise. Let Nθ be a neural networkfor image classiﬁcation with a set of parameters θ. The goalis to ﬁnd reasonable initial values for θ.Our main idea is to initialize the parameters by solvingan artiﬁcial noise classiﬁcation problem. Speciﬁcally, the pro-posed method consists of two steps. First, it generates noisedata D = {(ϵi, yi)}Ti=1, where ϵi is a Perlin noise sampleand yi is a category label. Second, it solves a classiﬁcationproblem on D, in which network parameters are optimizedto maximize the noise classiﬁcation accuracy. Note that eventhough this step can also be viewed as pre-training, the entireprocess is proposed as an initialization method because it doesnot use any natural images or any human-annotated labels. Tothe best of our knowledge, this is the ﬁrst work to initializenetworks by solving an artiﬁcial classiﬁcation problem usingnoise. The rest of this section describes the details of eachstep.
A. Generation of Noise Data
This subsection describes the generation of noise dataD = {(ϵi, yi)}Ti=1 in the ﬁrst step of the proposed method.As shown in Figure 1, the proposed method has two compo-nents, namely a noise sample generator Gη and a noise labelgenerator Lη with shared parameter η. Noise samples and theirlabels are generated as ϵi ∼ Gη and yi ∼ Lη, respectively.In this work, we utilize Perlin noise [2] to deﬁne thesetwo generators. Perlin noise was selected for the followingreasons: 1) It can render the textures of some natural objects.For example, in the ﬁeld of computer graphics, it is used torender clouds, ﬁre, and stones. 2) It has moderate complexityfor deﬁning categories (see the visualization of some noise
Fig. 3.Examples of noise categories. Labels attached to noise samplescorrespond to the complexity of Perlin noise.
categories in Figure 3). The deﬁnitions of the two generatorsusing Perlin noise are given below.
A-1. Noise Sample Generator
Let W, H, and C be the width, height, and number ofchannels of inputs of the network Nθ for image classiﬁca-tion, respectively. The noise sample generator Gη generatesϵi ∈ RW ×H×C by following the Perlin noise generationalgorithm in [2], summarized below. Note that it has parameterη = (n, m) in Step 1 and random values in Step 2.Step 1: Deﬁnition of a grid. This step deﬁnes a two-dimensional grid on a blank image whose size is W × H. Wedeﬁne a 2n × 2m grid (n, m ≥ 1) as shown in Figure 2 (a).Step 2: Placement of random gradient vectors. This stepplaces random gradient vectors vp,q at each grid point, i.e.,for p = 0, 1, 2, · · · , 2n and q = 0, 1, 2, · · · , 2m, as shownin Figure 2 (b). The magnitude and angle of each vp,q areuniformly sampled from [0, R) and [0, 2π), respectively. Here,we set R = 0.01 · max(W, H).Step 3: Generation of gradients. At each pixel, this stepcomputes the dot product of the gradient vectors at the fourcorners of the cell the pixel belongs to and the distance vectorsbetween the pixel and the corresponding corners. The four dotproduct values corresponding to the four corners are assignedto each pixel. This step is illustrated in Figure 2 (c). Note thatthe ﬁgure shows only one of the four values at each pixel.Step 4: Interpolation. Finally, through linear interpolationbetween the four values computed in the previous step, theﬁnal value of each pixel on ϵi is determined. An example isshown in Figure 2 (d).
A-2. Noise Label Generator
In the above algorithm, the way that the interval gradientvectors are deﬁned affects the complexity of the generated Per-
Fig. 4.Intra-category variation of noise samples. Image representations ofnoise samples for three categories (y = 1, 13, 43, 67) are shown.
lin noise. For example, compared to noise samples computedfrom a dense grid, those computed from a sparser grid willalso be sparser in terms of noise complexity.We use this difference in complexity to deﬁne categories ofnoise samples.Speciﬁcally, to samples ϵi ∼ Gη generated with parameterη = (n, m), the noise label generator Lη attaches labels
yi = (n − 1)M + m.(1)
Here, we assume that the range of η is {(n, m) : 1 ≤ n ≤N, 1 ≤ m ≤ M}. This means that the range of yi is{1, 2, · · · , NM} and the number of noise categories is NM.Figure 3 and Figure 4 show examples of noise categoriesand intra-category variation of noise samples, respectively.As can be seen, from category to category, noise complexityvaries from coarse to ﬁne with changes in gradient direction.This artiﬁcial category deﬁnition facilitates the acquisition ofa reasonable set of visual ﬁlters in a portion of the hiddenlayers of the neural network. For example, with a convolutionalneural network, we obtain some texture-like ﬁlters at thebottom layer of the network (results are shown in Figure 5in Sec. 4).Note that the proposed method has three hyperparameters,namely N, M, and K. N and M are for determining thenumber of noise categories and K is the number of instancesper category. Finally, the generated noise data D consist ofT = NMK noise samples with labels.
B. Optimization
The network parameters are optimized by solving a classiﬁ-cation problem on the generated noise data D = {(ϵi, yi)}Ti=1.In this step, any type of objective function and optimizer canbe introduced. Examples include cross-entropy loss with the
SGD or ADAM optimizer, which starts from He initialization.In contrast to standard pre-training, which solves an optimiza-tion problem on real-world data, this step solves an artiﬁcialoptimization problem, and as such this is a kind of closed-form problem. Introducing other types of optimizers to thisstep to solve this closed-form problem more efﬁciently, basedon the characteristics of Perlin noise, will be considered infuture work.
IV. EXPERIMENTS
In this section, we show the effectiveness of the proposedinitialization method on four image classiﬁcation datasets.
A. Datasets and Evaluation Measures
Cifar-10. This dataset consists of 60,000 color images, eachof which has a label from 10 object classes. We follow thestandard evaluation procedure, in which 50,000 images areused for training and 10,000 images are used for testing.Classiﬁcation accuracy over 10 classes is reported.Cifar-100. This dataset consists of 60,000 color images, eachof which has a label from 100 object classes. We followthe standard evaluation procedure, in which 500 and 100images per class are used for training and testing, respectively.Classiﬁcation accuracy over 100 classes is reported.Omniglot. This dataset consists of 38,300 grayscale imagesof 1,623 different hand-written characters. It was originallyproposed as a one-shot learning dataset. We use the datasplit proposed in [23]. 1,623 and 30,837 images are used fortraining and testing, respectively. Classiﬁcation accuracy over1,623 classes is reported.Describable Textures Dataset (DTD). This dataset consistsof 5640 images of 47 texture categories, such as checkered,striped, meshed, and marbled. We follow the standard evalu-ation procedure with three equal parts for training, validation,and testing.ImageNet. This dataset consists of 1.2 million color imagesof 1,000 object classes. We use this dataset to show thatthe proposed initialization method helps improve the dataefﬁciency of pre-training.
B. Implementation Details
Our implementation uses ResNet [8] as a backbone network.The results obtained using ResNet50 and ResNet152 arereported. For optimization, we use cross-entropy loss and theSGD optimizer in all experiments. For comparison, we reportresults obtained using He initialization [3], Xavier initializa-tion [15], sparse initialization [16], and normal initialization.Notably, He initialization is utilized in most state-of-the-artmethods for image classiﬁcation tasks.
C. Results
C-1. Main ResultsTable I shows a performance comparison of initializationmethods on the four datasets. As shown, the proposed methodalways outperforms the conventional methods. This shows theeffectiveness of initialization using Perlin noise.
TABLE IPERFORMANCE COMPARISON ON FOUR DATASETS. CLASSIFICATION ACCURACIES (%) FOR EACH DATASET WITH TWO TYPES OF NETWORK ARE SHOWN.
MethodCifar-10Cifar-100OmniglotDTD
ResNet50 ResNet152 ResNet50 ResNet152 ResNet50 ResNet152 ResNet50 ResNet152
Normal initialization92.6293.4775.1675.592.662.3713.685.24
Xavier initialization [15]92.3093.5873.8575.145.885.5727.5124.75
He initialization [3]93.5093.4374.1775.734.613.0624.3120.14
Proposed method93.7694.2777.4278.2117.5418.7155.0354.18
Fig. 5. Visualization of ﬁlters of the ﬁrst convolutional (conv1) layer. Filters before and after training on Cifar-10 are shown. (a) He initialization and (b)proposed method.
To analyze the reason for the superior performance of theproposed method, Figure 5 shows the ﬁlters at the bottom layer(conv1 layer) before and after training. As can be seen, theproposed method gives a set of various ﬁlters, including sometexture-like and edge-like ﬁlters. Therefore, after training, itobtains a useful set of colored ﬁlters for image classiﬁcationeven if the number of training samples is not very large.Figure 6 shows validation accuracy curves for the Cifar-100 dataset. Our method not only achieves higher accuracy atthe end of training, but its training starts at higher accuracy.This shows that, as an initial value for training, texture andedge ﬁlters are a better choice than randomized ﬁlters. Inthis experiment, we used the ﬁxed learning rate schedule inthe ofﬁcial PyTorch implementation tuned for training fromscratch for a fair comparison. Optimizing the learning rateschedule for our method may lead to faster convergence.In the present paper, we did not use color Perlin noisebecause some applications, including hand-written characterrecognition, use grayscale images. In future work, a noisesample generator that uses the RGB color space can be appliedfor cases where the dataset consists of color images.
C-2. HyperparametersTo explore the effect of changes in hyperparameters, Table IIshows the results obtained using various values for N, M,and K for noise data generation. Note that, as describedin Sec. 3, (N, M) determines the number of Perlin noisecategories based on noise complexity, and K controls thenumber of instances per category for generating T = NMK
Fig. 6. Validation accuracy curves for the Cifar-100 dataset. He initializationand our method are compared.
noise samples.A comparison of two strategies, namely increasing thenumber of categories and increasing the number of instances,shows that the former is more effective than the latter forimproving image classiﬁcation accuracy. This supports ourassumption that noise complexity is an important factor fordetermining categories.Performance improvements were observed across bothnetwork structures, with ResNet152 always outperformingResNet50. In general, it is difﬁcult to train a large network witha limited amount of data. However, our method is adaptable
TABLE IIEXPERIMENTAL RESULTS OBTAINED USING VARIOUS VALUES FORHYPERPARAMETERS IN NOISE DATA GENERATION. ACCURACIES FOR THECIFAR-100 DATASET ARE SHOWN. NETWORK: BACKBONE NETWORK(RESNET50 OR RESNET152). # CATEGORIES: NUMBER OF PERLIN NOISECATEGOREIS; N × M CATEGOREIS ARE GENERATED. # INSTANCES:NUMBER OF INSTANCES PER CATEGORY.
Network#categories#instances
1005001, 000
10 × 1076.2875.9976.44
ResNet5018 × 1876.9576.9275.53
36 × 3675.7777.4278.03
10 × 1077.0977.0977.77
ResNet15218 × 1877.9277.6977.23
36 × 3678.3078.2177.51
to networks of different sizes because noise complexity can beadjusted based on the network size. This advantage improvesthe performance of training large networks with a limitedamount of data. To further improve performance, future noisesample generators can be designed using other types of noise.
C-3. Data-Efﬁcient Pre-TrainingFinally, we combine our method with pre-training to explorehow this combination improves image classiﬁcation perfor-mance. Figure 7 shows the results obtained using variousnumbers of images from the ImageNet (ILSVRC 2012) datasetfor pre-training. According to the results, our method isparticularly effective when the number of pre-training imagesis small. Although this paper focuses on network initializationwithout using any real-world images, the results imply thatcombining our method with pre-training methods would leadto more data-efﬁcient pre-training.
V. CONCLUSION
This paper proposed a novel network initialization methodthat uses Perlin noise. Experiments on four image classiﬁcationdatasets demonstrated that the proposed initialization is effec-tive for training networks for image classiﬁcation. In particular,our method is effective for training with a limited amount ofdata because it provides texture-like initial ﬁlters that improveimage classiﬁcation accuracy. Our future work will focus onextending this initialization method to other types of noise,including colored noise, and data such as audio and videodata, as well as introducing new noise category deﬁnitions.Acknowledgment This work was partially supported by grants JSPSKAKEN 19H01134 and JST ACT-I (JPMJPR16U5). Computationalresource of AI Bridging Cloud Infrastructure (ABCI) provided byNational Institute of Advanced Industrial Science and Technology(AIST) was used.
REFERENCES
[1] O. Russakovsky, et al., ImageNet Large Scale Visual RecognitionChallenge. In Springer IJCV, vol. 115, no. 3, pp. 211–252, 2015.
[2] K. Perlin, An Image Synthesizer. In SIGGRAPH, pp. 287–296, 1985.[3] K. He, et al., Delving Deep into Rectiﬁers: Surpassing Human-Level Per-formance on ImageNet Classiﬁcation. Proc. ICCV, pp. 1026–1034, 2015.
Fig. 7. Performance comparison using various numbers of samples for pre-training. Accuracy for the Cifar-100 dataset is shown. ResNet152 was pre-trained using images from the ImageNet dataset.
[4] Y. LeCun, et al., Gradient-based Learning Applied to Document Recog-nition. Proc. of the IEEE, pp. 2278–2324, 1998.
[5] A. Krizhevsky, et al., ImageNet Classiﬁcation with Deep ConvolutionalNeural Networks. Proc. NeurIPS, pp.1–9, 2012.
[6] K. Simonyan and A. Zisserman. Very Deep Convolutional Networks forLarge-Scale Image Recognition. Proc. ICLR, 2015.
[7] C. Szegedy, et al., Going Deeper with Convolutions. Proc. CVPR, 2015.[8] K. He, et al., Deep Residual Learning for Image Recognition.Proc. CVPR, 2016.
[9] S. Xie, et al., Aggregated Residual Transformations for Deep NeuralNetworks. Proc. CVPR, 2017.
[10] G.Huang,etal.,DenselyConnectedConvolutionalNetworks.Proc. CVPR, 2017.
[11] J. Hu, et al., Squeeze-and-Excitation Networks. Proc. CVPR, 2018.[12] J. Carreira and A. Zisserman. Quo Vadis, Action Recognition? A NewModel and the Kinetics Dataset. Proc. CVPR, 2017.
[13] B. Zhou, et al., Places: A 10 million Image Database for SceneRecognition. In IEEE Trans. on PAMI, vol. 40, no. 6, pp. 1452–1464,2017.
[14] F. Manessi and A. Rozza. Learning Combinations of Activation Func-tions. Proc. ICPR, pp. 61–66, 2018.
[15] X. Glorot and Y. Bengio. Understanding the Difﬁculty of Training deepFeedforward Neural Networks. Proc. AISTAS, pp. 249–256, 2010.
[16] J. Martens. Deep Learning via Hessian-free Optimization. Proc. ICML,2010.
[17] W. Kay, et al., The Kinetics Human Action Video Dataset. CoRR1705.06950, 2017.
[18] Z. Hailat, et al., Deep Semi-Supervised Learning. Proc. ICPR, pp. 2154–2159, 2018.
[19] L. Chen, et al., Semi-Supervised Convolutional Neural Networks withLabel Propagation for Image Classiﬁcation. Proc. ICPR, pp. 1319–1324,2018.
[20] Z. Ling, et al., Semi-Supervised Learning via Convolutional NeuralNetwork for Hyperspectral Image Classiﬁcation. Proc. ICPR, pp. 1–6, 2018.
[21] A. Robles-Kelly and Ran Wei. Semi-Supervised Image Labelling UsingBarycentric Graph Embeddings. Proc. ICPR, pp. 1518–1523, 2016.
[22] A. Ghaderi and V. Athitsos. Selective Unsupervised Feature Learningwith Convolutional Neural Network. Proc. ICPR, pp. 2486–2490, 2016.
[23] B. M. Lake, et al., Human-level Concept Learning Through ProbabilisticProgram Induction. Science, vol. 350(6266), pp. 1332–1338, 2015.
[24] A. Krizhevsky. Learning Multiple Layers of Features from Tiny Images.Technical Report TR-2009, University of Toronto, 2019.
[25] L. Fei-Fei, et al., One-Shot learning of object categories. IEEE Trans.Pattern Recognition and Machine Intelligence, vol. 28 , no. 4, 2006.
[26] M. Everingham, et al., The Pascal Visual Object Classes (VOC) Chal-lenge. In Springer IJCV, vol. 88, no. 2, pp. 303–338, 2010.
[27] M. Cimpoi, et al., Describing Textures in the Wild. Proc. CVPR, 2014.[28] T.-Y. Lin, et al., Microsoft COCO: Common Objects in Context.Proc. ECCV, 2014.