While deep neural networks have shown outstanding results in a wide range of applications,
 learning from a very limited number of examples is still a challenging
 task. Despite the difficulties of the few-shot learning, metric-learning techniques
 showed the potential of the neural networks for this task. While these methods
 perform well, they don’t provide satisfactory results. In this work, the idea of
 metric-learning is extended with Support Vector Machines (SVM) working mechanism,
 which is well known for generalization capabilities on a small dataset.
 Furthermore, this paper presents an end-to-end learning framework for training
 adaptive kernel SVMs, which eliminates the problem of choosing a correct kernel
 and good features for SVMs. Next, the one-shot learning problem is redefined
 for audio signals. Then the model was tested on vision task (using Omniglot
 dataset) and speech task (using TIMIT dataset) as well. Actually, the algorithm
 using Omniglot dataset improved accuracy from 98.1% to 98.5% on the one-shot
 classification task and from 98.9% to 99.3% on the few-shot classification task. Deep learning has shown the ability to achieve outstanding results for real-world problems in various areas such as image, audio and natural language processing BID18 . However these networks require large datasets, so the model fitting demands significant computational resources. On the other hand, there are techniques for learning on small datasets, such as data augmentation and special regularization methods, but these methods' accuracy is far from desirable on a very limited dataset. As well as slowness of the training process is caused by the many weight update iterations, which is required due to the parametric aspect of the model.Humans are capable of learning the concept from only a few or even from one example. This learning characteristic differs much from the deep neural networks' learning curve. This discovery leads us to one-shot learning task BID6 , which consists of learning each class from only one example. Nevertheless, one single example is not always enough for humans to understand new concepts. In view of the previous fact, the generalization of one-shot learning task exists as well, it is called few-shot learning or k-shot learning, where the algorithm learns from exactly k samples per class.Deep learning approaches data-poor problems by doing transfer learning BID2 : the parameters are optimized on a closely related data-rich problem and then the model is fine-tuned on the given data. In contrast, one-shot learning problem is extremely data-poor, but it requires similar approach as transfer learning: in order to learn good representation, the model is trained on similar data, where the classes are distinct from the one-shot dataset. In the next step, standard machine learning tools are used on the learned features to classify the one-shot samples. As a matter of fact, BID26 claimed that parameterless models perform the best, but they concentrated on only k-nearest neighbors algorithm. Considering this observation this work applies Support Vector Machine BID0 , which can be regarded as a parameterless model. This paper presents the k-shot related former work in the following section. Then the proposed model, which is called Siamese kernel SVM, is introduced with a brief summary of the used wellknown methods. In Section 4 the experimental setup is described for both a vision and an auditory task, where minor refinement of the problem is required. In this work, Siamese kernel SVM was introduced, which is capable of state-of-the-art performance on multiple domains on few-shot learning subject to accuracy. The key point of this model is combining Support Vector Machines' generalizing capabilities with Siamese networks one-shot learning abilities, which can improve the combined model's results on the k-shot learning task. The main observation of this work is that learning representation for another model is much easier when the feature extractor is taught as an end-to-end version of the other model. In addition, parameterless models achieve the best results on the previously defined problem, which makes SVMs an adequate choice for the task. This paper also introduced the concept of k-sec learning, which can be used for audio and video recognition tasks, and it gave a baseline for this task on the TIMIT dataset. The author hopes defining k-sec learning task encourage others to measure one-shot learning models' accuracy on various domains.
The proposed method is an end-to-end neural SVM, which is optimized for few-shot learning.
{"src": "scitldr-AIC_train", "paper_id": "B1EVwkqTW"}