INTRODUCTION

The present technology relates generally to machine learning systems. More particularly, embodiments of the present invention relate to: (1) A Continuously Learning, Stable and Robust Online Machine Learning System (OMLS); (2) An Online Machine Learning System (OMLS) that is Human Understandable; and (3) An Online Machine Learning System (OMLS) that can Learn in a User-Friendly Way from both Human and Data at the Same Time.

1. A Continuously Learning, Stable and Robust Online Machine Learning System (OMLS)

Machine learning models that can learn from data have been used in practice in many different industries such as finance, telecommunications, insurance and health. In most of the current practice, the machine learning models are trained on the available data set in batch mode. In this mode, the available dataset is divided into non-overlapping training and validation (and sometimes additional test) sets. For different hyperparameter settings, the machine learning model is trained. The hyperparameters could be, for example, for a neural network: the number of layers, the numbers of hidden units, choice of activation functions, optimization methods; for a decision tree: the maximum number of layers, the minimum number of instances for a node to become a parent, the minimum impurity before which a node can become a parent; for a clustering algorithm: the number of clusters, distance functions used. The machine learning algorithms' parameters are updated to optimize a performance criterion as accuracy, AUC (Area Under the ROC Curve), precision, recall, f-measure on a training dataset [Alpaydin, E. (2014). Introduction to machine learning. MIT press]. The performance of the trained model is evaluated on the validation set and the model hyperparameters are decided based on the validation set performance. One of the underlying reasons for this training-validation practice is the fact that most machine learning models that are complex enough can fit (and hence overfit) any training set if they are given hyperparameters of enough complexity, and it is desired to have stable (not overfitted) models while the world that created the data keeps changing. Although many businesses have terabytes of data in their systems, the emerging patterns in the data start with small number of observations, therefore stable, not overfitting models are desired in order to be able to take reliable actions on them. Another desirable property of any machine learning system is that it is robust to changes in the world which triggers changes in the data that can manifest themselves, for example, as additions/deletions/modifications of the features or patterns. To sum up, businesses need stable and robust machine learning systems with high performance.

In the batch learning scenario, the validated model is deployed and it is used to make predictions. Models are usually periodically trained and validated to make sure that their performance does not degrade over time, however, especially when data keep continuously changing, the models may perform poorly in between these periodical trainings and also it is difficult to know when to train the models again especially in situations (such as churn, fraud, etc.) where the labels for the current instances are not available immediately. Furthermore, the model trained in the previous iteration may be very different from the new one, and understanding and operating with the new model can be difficult for the business people.

In order to overcome the shortcomings of batch learning models, online learning models that learn continuously from data have been used [Pedro Domingos and Geoff Hulten. Mining High-Speed Data Streams. In Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '00, pages 71-80, New York, N.Y., USA, 2000. ACM; Bifet, A., Holmes, G., Kirkby, R., & Pfahringer, B. (2010). Moa: Massive online analysis. The Journal of Machine Learning Research, 11, 1601-1604; Aggarwal, C. C. (2014). A Survey of Stream Classification Algorithms. Data Classification: Algorithms and Applications, 245; Cataltepe, Z., Ekmekci, U., Cataltepe, T., & Kelebek, I. (2016, April). Online feature selected semi-supervised decision trees for network intrusion detection. In Network Operations and Management Symposium (NOMS), 2016 IEEE/IFIP (pp. 1085-1088). IEEE; Jaworski, M., Duda, P., & Rutkowski, L. (2018)]. New splitting criteria for decision trees in stationary data streams. IEEE transactions on neural networks and learning systems, 29(6), 2516-2529]. While these models usually are able to adapt to changing datasets, they have not yet been adapted in domains such as banking, insurance or health, due to performance, robustness, stability and also understandability issues. In these domains, and in general, the decision makers need to be accountable for their actions. However, since the models keep continuously changing, it is difficult to keep the performances of the models and also track of how models decided at different times and hence be accountable for their decisions. It is also difficult to understand if the decisions made at different times are comparable to each other. Since data behavior and also data sources keep changing, businesses also need machine learning systems that can include new features or can function when some old features disappear. Based on the machine learning outputs, some actions, such as CRM, sales, marketing, pricing decisions are planned, an online machine learning system also needs to be as stable as possible, so that the business continuity can be maintained.

2. An Online Machine Learning System (OMLS) that is Human Understandable

One of the problems of complex machine learning models in general and deep learning models in particular is that they produce black box models that are not easily understandable by the domain experts. Even though there are attempts to explain these models [Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (pp. 4765-4774)], the explanations are oriented towards data scientists and may still not be understandable and hence trustable by the business people. They are even more difficult to understand by the people who are affected by the machine learning models' outcomes, for example a credit card applicant whose application has been rejected, or an insurance customer whose claim was marked as fraudulent. Even if the machine learning model is not a deep neural network but a combination of a set of various machine learning algorithms, the outputs of the algorithms may not be easy to understand by humans. Attempts to make machine learning models understandable, such as [Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (pp. 4765-4774)], require different models to be trained for each instance and hence are not feasible for learning on streaming data. For continuously learning machine learning models that keep changing in time, understandable models become even more of a problem since the decision makers need to understand not a single model, but basically a set of models whose internal parameters and hence their explanations keep changing all the time.

3. An Online Machine Learning System (OMLS) that can Learn in a User-Friendly Way from both Human and Data at the Same Time

In almost every domain, the patterns in the collected data keep changing rapidly due to external factors (such as political, economical, social, environmental or regulatory changes) or business decisions such as different campaigns or products offered to the customers by the company or its competitors. When batch machine learning systems are used, the only way the machine learning system can adapt to these changes is through complete retraining of the model when enough data has been accumulated, which not only takes time but also may cause loss of business opportunity and also discontinuity in the business processes. When continuous machine learning systems are used, the model adapts to the change without an interruption, however, it will still take time to learn the change. In many cases, when there is or is expected to be a change known to the domain expert, there is a lost opportunity if the expert's knowledge cannot be integrated into the machine learning system.

It is important that the domain expert can update the machine learning models to help the algorithms perform better in the presence of change. In order to be able to update the machine learning models, not only a user friendly mechanism built within the business processes, but also understandable machine learning models are required.

In addition to model updates due to change, the domain expert time may be required to get input for certain instances, such as fraud or anomaly. However, getting the label for an instance is expensive because the domain expert time is usually expensive or not available. The human expert can provide feedback on the selected alerts or instances where the online machine learning system is not certain and accelerate the learning through these feedbacks. Active machine learning [Settles, B. (2012). Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6(1), 1-114] and co-training [Blum, A., & Mitchell, T. (1998, July). Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory (pp. 92-100). ACM] are some of the techniques that have been used to get feedback from the humans for batch machine learning systems.

