TECHNICAL FIELD

One or more embodiments of the present application relate to the field of computer technology, and in particular relate to a feature selection method, device and apparatus for constructing a machine learning model.

BACKGROUND

In order to construct a machine learning model having optimal performance, data analysts and data mining engineers tend to violently derive numerous dimensions of features (also known as variables) according to service experience and understanding of data. However, this process often generates a great amount of redundant and subtle information, which has little value for the desired to-be-constructed machine learning model, and even generates side effects. Therefore, in the process of constructing the machine learning model, we need to continuously carry out tests and carefully perform feature filtering, and finally construct an optimal machine learning model.

For the above-mentioned feature selection process, when it is carried out manually, it is usually very labor-intensive and slows down the speed of model construction, so it is usually carried out automatically. In the conventional technology, there are several automatic feature selection methods, including filter-based feature selection, embedded feature selection, and wrapper-based feature selection. When these feature selection methods are used to filter features, they usually only consider the performance of features on some divided data sets.

Therefore, it is necessary to provide a feature selection method that can filter out more accurate features.

SUMMARY

One or more embodiments of the present application describe a feature selection method, device and apparatus for constructing a machine learning model, which can filter out more accurate features.

In the first aspect, an embodiment provides a method and system for feature selection. During operation, the system obtains a training data set and features associated with the training data set; divides the training data set into a first number of training data subsets; and forms a plurality of feature-selecting data sets. A respective feature-selecting data set comprises a second number of training data subsets, the second number being smaller than the first number. The system processes, in parallel, each feature-selecting data set, which comprises: computing a first evaluation index for the features based on the feature-selecting data set; obtaining a group of index ranks corresponding to the features based on the first evaluation index; and obtaining a group of importance ranks corresponding to the features based on the feature-selecting data set and a machine-learning model. The system further obtains a group of total ranks corresponding to the features by fusing groups of index ranks and groups importance ranks obtained from processing the plurality of feature-selecting data sets; and selecting target features from the features based on the group of total ranks.

In a variation on this embodiment, obtaining the group of total ranks of the features comprises: fusing the groups of index ranks obtained from processing the plurality of feature-selecting data sets to obtain a group of total index ranks; fusing the groups of importance ranks obtained from processing the plurality of feature-selecting data sets to obtain a group of total importance ranks; and fusing the group of total index ranks and the group of total importance ranks to obtain the group of total ranks of the plurality of features.

In a further variation, processing each feature-selecting data set further comprises: computing additional evaluation indices for the features based on the feature-selecting data set; and obtaining additional groups of index ranks corresponding to the features based on the additional evaluation indices.

In a further variation, fusing the groups of index ranks to obtain the group of total index ranks of the plurality of features comprises: for each evaluation index selected from a set of evaluation indices comprising the additional evaluation indices and the first evaluation index: extracting from groups of index ranks association with the set of evaluation indices and the plurality of feature-selecting data sets, a plurality of groups of index ranks associate with the selected evolution index; and performing a first rank fusion operation to respectively fuse corresponding ranks of the features in the extracted plurality of groups of index ranks to obtain a group of comprehensive index ranks corresponding to the selected evaluation index. Fusing the groups of index ranks further comprises performing a second rank fusion operation to respectively fuse groups of index ranks obtained for the set of evaluation indices to obtain the total index ranks of the features.

In a further variation, the first rank fusion operation or the second rank fusion operation comprises one of: a mean operation, a maximum operation, a minimum operation, a weighted average operation and a robust rank aggregation (RRA) operation.

In a variation on this embodiment, dividing the training data set comprises one of: dividing the training data set based on time; and dividing the training data set randomly.

In a variation on this embodiment, the first evaluation index comprises one of: an information value (IV), a Gini coefficient (GINI), an information gain (IG), mutual information (MI), a Relief score, and a sample stability index (PSI).

In a variation on this embodiment, the training data set is divided into k training data subsets, wherein k different feature-selecting data sets are formed, and wherein each feature-selecting data set comprises k-1 training data subsets.

In the second aspect, a feature selection method for constructing a machine learning model is provided, including:

obtaining a training data set;

dividing the training data set in a preset division manner to obtain k groups of training data subsets;

executing, in parallel, the following processes on the k groups of training data subsets for k times:

selecting k-1 groups of training data subsets from the k groups of training data subsets and regarding the same as a current training data set;

computing, according to the current training data set, m evaluation indices for a plurality of features;

ranking the plurality of features according to each evaluation index to obtain m groups of index ranks of the plurality of features;

training a machine learning model on the basis of the current training data set to predict a group of importance ranks of the plurality of features;

fusing k*m groups of index ranks and k groups of importance ranks obtained after k times of execution to obtain total ranks of the plurality of features;

selecting target features from the plurality of features according to the total ranks.

In the third aspect, a feature selection device for constructing a machine learning model is provided, including:

an obtaining unit configured to obtain a training data set;

a division unit configured to divide, in a preset division manner, the training data set obtained by the obtaining unit to obtain k groups of training data subsets;

an execution unit configured to execute, in parallel, the following processes on the k groups of training data subsets obtained by the division unit for k times:

selecting k-1 groups of training data subsets from the k groups of training data subsets and regarding the same as a current training data set;

computing, according to the current training data set, m evaluation indices for a plurality of to-be-filtered features;

ranking the plurality of features according to each evaluation index to obtain m groups of index ranks of the plurality of features;

training a machine learning model on the basis of the current training data set to predict a group of importance ranks of the plurality of features;

a fusion unit configured to fuse k*m groups of index ranks and k groups of importance ranks obtained by the execution unit after k times of execution to obtain total ranks of the plurality of features;

a selection unit configured to select target features from the plurality of features according to the total ranks obtained by the fusion unit.

In the fourth aspect, a feature selection apparatus for constructing a machine learning model is provided, including:

a memory;

one or more processors; and

one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the program is executed by the processor to implement the following steps:

obtaining a training data set;

dividing the training data set in a preset division manner to obtain k groups of training data subsets;

executing, in parallel, the following processes on the k groups of training data subsets for k times:

selecting k-1 groups of training data subsets from the k groups of training data subsets and regarding the same as a current training data set;

computing, according to the current training data set, m evaluation indices for a plurality of to-be-filtered features;

ranking the plurality of features according to each evaluation index to obtain m groups of index ranks of the plurality of features;

training a machine learning model on the basis of the current training data set to predict a group of importance ranks of the plurality of features;

fusing k*m groups of index ranks and k groups of importance ranks obtained after k times of execution to obtain total ranks of the plurality of features;

selecting target features from the plurality of features according to the total ranks.

One or more embodiments of the present application provide a feature selection method, device and apparatus for constructing a machine learning model. A training data set is obtained. The training data set is divided in a preset division manner to obtain k groups of training data subsets. The following processes are executed, in parallel, on the k groups of training data subsets for k times: selecting k-1 groups of training data subsets from the k groups of training data subsets and regarding the same as a current training data set; computing, according to the current training data set, m evaluation indices for a plurality of to-be-filtered features; ranking the plurality of features according to each evaluation index to obtain m groups of index ranks of the plurality of features; training a machine learning model on the basis of the current training data set to predict a group of importance ranks of the plurality of features; fusing k*m groups of index ranks and k groups of importance ranks obtained after k times of execution to obtain total ranks of the plurality of features; and selecting target features from the plurality of features according to the total ranks. Accordingly, it can be seen that after the k group of training data subsets are obtained by division, the selection of the current training data set, the index ranks of the plurality of features, and the importance ranks are executed in parallel for k times. Therefore, the comprehensive performance of the plurality of features in each group of training data subsets can be considered, and thus more accurate features can be obtained by filtering. In addition, the feature selection method provided by the present application also comprehensively considers a plurality of evaluation indices of each feature, and thus filtering can be performed to obtain more stable and more effective features.

