CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2019-006321, filed on Jan. 17, 2019, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to a learning method.

BACKGROUND

In the case of learning a deep learning model, the volume of training data (learning data) is a factor that significantly contributes to the performance, and thus it is desirable to have as large volume of the training data as possible. If the training data becomes insufficient and if applied data of a type not available in the training data is applied to the deep learning model that was learnt using the training data, then it is highly likely that an appropriate output result is not obtained and the learning ends up in failure.

In a business situation in which customer data is used, in view of the risk of divulging of the contract agreement or the information, it is difficult to indefinitely hold on particular customer data or to share it in other customer tasks. Hence, there are times when the training data becomes insufficient.

When the training data becomes insufficient, it is common practice to perform data augmentation. The data augmentation includes processing such as noise addition, parallel translation, and data missing; and the range of the training data can be expanded to the range of the applied data.

A conventional technology is known in which intermediate feature quantities that are obtained as a result of inputting the original training data to a deep learning model are held, so that the available volume of data at the time of newly learning the deep learning model is increased without having to hold on to the original training data.

FIG. 18is a diagram for explaining the conventional technology. With reference toFIG. 18, a deep learning model10includes a first neural network (NN)10aand a second neural network (NN)10b. When training data is input thereto, the first NN10acalculates intermediate feature quantities. When the intermediate feature quantities are input thereto, the second NN10bcalculates output labels. It is assumed that the parameter of the first NN10aand the second NN10bis already learnt using a plurality of sets of training data of a customer P as stored in a database10P. When the learning is over, the information in the database10P is returned to the customer P (or is destroyed).

For example, when training data xP1is input to the first NN10a, an intermediate feature quantity zP1gets calculated. When the intermediate feature quantity zP1is input to the second NN10b, an output label yP1′ gets calculated. In the conventional technology, before the information in the database10P is returned, the intermediate feature quantity zP1that is calculated from the training data xP1is stored in a database13. Moreover in the conventional technology, the intermediate feature quantities that are calculated from the other sets of training data stored in the database10P are also stored in the database13.

Subsequently, in the conventional technology, a new deep learning model11is learnt (sequentially learnt) using a database11Q, in which a plurality of sets of training data of a customer Q is stored, and using the database13. The deep learning model11includes a first NN11aand a second NN11b. In the conventional technology, the already-learnt parameter of the first NN10ais set as the parameter of the first NN11a. Similarly, the already-learnt parameter of the second NN10bis set as the parameter of the second NN11b.

For example, when training data xQ1in the database11Q is input to the first NN11a, an intermediate feature quantity zQ1gets calculated. When the intermediate feature quantity zQ1is input to the second NN11b, an output label yQ1′ gets calculated. In the conventional technology, the parameter of the second NN11bis learnt in order to ensure that the output label yQ1′ approaches the correct label (not illustrated).

Moreover, when the intermediate feature quantity zP1is input to the second NN11bfrom the database13, an output label yP1′ gets calculated. In the conventional technology, the parameter of the second NN11bis learnt in order to ensure that the output level yP1′ approaches the correct label (not illustrated).

As described above, in the conventional technology, in the case of learning the parameter of the second NN11b, the learning is performed using the intermediate feature quantities calculated from the training data stored in the database11Q as well as using the intermediate feature quantities stored in the database13. For that reason, even if the database10P is returned to the customer P (or is destroyed), it becomes possible to increase the available volume of data at the time of learning a new deep learning model.

[Non-patent Literature 1] Utako Yamamoto et al. “Deformation estimation of an elastic object by partial observation using a neural network”

In the case in which a deep learning model is learnt using the conventional technology explained with reference toFIG. 18, if all intermediate feature quantities that are calculated from a plurality of sets of data-augmented training data are stored, then the volume of stored data becomes large.

FIGS. 19 and 20are diagrams for explaining the issues faced in the conventional technology. The following explanation is given with reference toFIG. 19. In the conventional technology, as a result, of performing data augmentation with respect to the original training data xP1stored in the database10P, sets of training data xP1.1, xP1.2, and xP1.3are generated. In the conventional technology, when the sets of training data xP1.1, xP1.2, and xP1.3are separately input to the first NN10a, intermediate feature quantities zP1.1, zP1.2, and zP1.3get respectively calculated. Then, in the conventional technology, when the intermediate feature quantities zP1.1, zP1.2, and zP1.3are separately input to the second NN10b, output labels yP1.1′, yP1.2′, and yP1.3′ get respectively calculated.

In the conventional technology, the intermediate feature quantities zP1.1, zP1.2, and zP1.3are stored in the database13. Herein, if data augmentation is performed with respect to each set of original training data stored in the database10P, then all intermediate feature quantities of each set of data-augmented training data get stored in the database13, thereby resulting in an increase in the volume of data.

Generally, in the case of learning a deep learning model, it is preferable to have as large volume of training data as possible. However, if the distribution of the intermediate feature quantities of the training data obtained by data augmentation of particular original training data is not separated from the distribution of the intermediate feature quantities of the training data obtained by data augmentation of some other original training data; then, even if those intermediate feature quantities are used, the output labels obtained by inputting the intermediate feature quantities to the second NN10bare not appropriately separated thereby possibly leading to a decline in the learning accuracy. Hence, it is not always the case that data augmentation is performed to store as large volume of training data as possible in the database13.

The following explanation is given with reference toFIG. 20. In order to reduce the volume of data stored in the database13, it is possible to think of simply reducing the intermediate feature quantities. In the example illustrated inFIG. 20, from among the intermediate feature quantities zP1.1, zP1.2, and zP1.3, the intermediate feature quantities zP1.1and zP1.2are omitted. However, if the intermediate feature quantities are reduced according to this method, the effects of data augmentation is achieved and the learning accuracy of the deep learning model undergoes a decline. That is, even if the training data xQ1is subjected to data augmentation to generate data-augmented training data xQ1.1, xQ1.2, and xQ1.3; it is not possible to continually make use of the effects achieved as a result of performing data augmentation with respect to the training data xP1using the deep learning model10. Meanwhile, when the sets of post-augmentation training data xQ1.1, xQ1.2, and xQ1.3are input to the first NN11a, intermediate feature quantities zQ1.1, zQ1.2, and zQ1.3get respectively calculated. When the intermediate feature quantities zQ1.1, zQ1.2, and zQ1.3are input to the second NN11b, output labels yQ1.1′, yQ1.2′, and yQ1.3′ get respectively calculated.

SUMMARY

According to an aspect of an embodiment, a learning method, executed by a computer, includes: first learning at last one parameter of a learning model such that each intermediate feature quantity becomes similar to a reference feature quantity, the each intermediate feature quantity being calculated as a result of inputting a plurality of sets of augmentation training data to a first neural network in the learning model, the plurality of augmentation training data being generated by performing data augmentation based on same first original training data; and second learning at last one parameter of a second network, in the learning model, using second original training data, which is different than the first original training data, and using the reference feature quantity.

The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, as claimed.

