TECHNICAL FIELD

The present application generally relates to systems for assessing the health of an analytical model, and, in particular, to an ensemble approach to assessing analytical model health and alerting to model degradation.

BACKGROUND

Analytical models for event detection are important to a range of fields and industries. For example, various analytical models are used to detect banking fraud, aid in regulatory compliance, and many other complex, data-driven problems. Many fields require the most up-to-date models for accurate and timely event detection. In some fields, for example, many types of fraud, a third party agent is actively working to escape detection by current analytical models. One aspect of detection modeling is determining when a model has degraded and should be replaced or updated.

SUMMARY

According to some embodiments, the present disclosure describes a computer-implemented method for alerting to analytical model degradation in a data processing system. The method includes performing steps by a processing device, including performing, by the processing device, a distribution analysis on the selected detection model to determine a first health rating for the selected detection model, performing, by the processing device, a survival analysis on the selected detection model to determine a second health rating for the selected detection model, generating, by the processing device, an indicative score for the detection model based on the first health rating and the second health rating, and comparing, by the processing device, the indicative score to a threshold value and alerting to model degradation based on the comparison.

According to other embodiments, the present disclosure describes a detection modeling system. The detection modeling system may include a processing device and a memory coupled to the processing device, the processing device configured to execute programmed instructions stored in the memory to perform a distribution analysis on the selected detection model to determine a first health rating for the selected detection model, perform a survival analysis on the selected detection model to determine a second health rating for the selected detection model, generate an indicative score for the detection model based on the first health rating and the second health rating, and compare the indicative score to a threshold value and alert to model degradation based on the comparison.

According to additional embodiments, the present disclosure describes a non-transitory computer readable medium having stored thereon instructions for alerting to analytical model degradation comprising executable code, which when executed by at least one processing device performs disclosed methods consistent with disclosed embodiments.

Additional features and advantages of this disclosure will be made apparent from the following detailed description of illustrative embodiments that proceeds with reference to the accompanying drawings.

