FIELD

Embodiments described herein relate to automatic reduction of software regression and, in particular, to systems and methods for detecting unhealthy software payloads and preventing the rollout of unhealthy software payloads to computing platforms in cloud computing environments.

SUMMARY

Cloud computing and other shared computing environments include multiple computing platforms to provide software applications and services to users via the Internet and other networks. Applications that provide content creation, communication, and data storage and manipulation interact with one another and one or more hosting platforms. The software code for the applications and the platforms is regularly updated to add features, correct errors, respond to user requests, and the like. In some cases, software payloads that may include many thousands of code changes across some or all of the applications are rolled out to the computing platforms. Application of the software payloads to software systems sometimes results in software regression and software performance regression.

To help prevent software regression caused by software payload rollouts, embodiments described herein, among other things, automatically detect potential regressions as a software payload begins to rollout, and prevents the software regression from propagating to all platforms in a computing environment. Using certain embodiments presented herein, software regression is detected and mitigated early in a payload rollout, resulting in a better user experience. Some embodiments described herein therefore result in more efficient use of computing system resources, and improved operation of computing systems for users.

In particular, one embodiment provides a system for automatically reducing software regression for a software payload applied to a plurality of computing platforms by a software updater. The system includes an anomaly detector configured to analyze telemetry data for the computing platforms to produce failure data for the computing platforms, an escalation engine configured to receive payload rollout data from the software updater and track state information for the computing platforms, and a decision engine communicatively coupled to the plurality of computing platforms, the software updater, the anomaly detector, and the escalation engine. The decision engine includes an electronic processor. The electronic processor is configured to receive, from the software updater, a health request associated with the software payload. The electronic processor is configured to, in response to receiving the health request, retrieve, from the escalation engine, a plurality of identifiers identifying a subset of the plurality of computing platforms that have completed deployment of the software payload. The electronic processor is configured to determine a plurality of unified logging service (ULS) tags associated with the software payload. The electronic processor is configured to query the anomaly detector for failure data, for the subset of the plurality of computing platforms, corresponding to the ULS tags, the failure data including pre-deployment failure data and post-deployment failure data. The electronic processor is configured to detect a potential software regression associated with the software payload by comparing the pre-deployment failure data and the post-deployment failure data. The electronic processor is configured to, in response to detecting a potential software regression, transmit, to the escalation engine, an event based on the potential software regression.

Another embodiment provides a method for automatically reducing software regression for a software payload applied to a plurality of computing platforms by a software updater. The method includes receiving, from a software updater, a health request associated with the software payload. The method includes, in response to receiving the health request, retrieving, from an escalation engine, a plurality of identifiers identifying a subset of the plurality of computing platforms that have completed deployment of the software payload. The method includes determining a plurality of unified logging service (ULS) tags associated with the software payload. The method includes querying an anomaly detector for failure data, for the subset of the plurality of computing platforms, corresponding to the ULS tags, the failure data including pre-deployment failure data and post-deployment failure data. The method includes detecting a potential software regression associated with the software payload by comparing the pre-deployment failure data and the post-deployment failure data. The method includes, in response to detecting a potential software regression, querying a root cause analyzer based on the potential software regression. The method includes receiving, from the root cause analyzer, at least one change list identifier identifying at least one potential root cause for the potential software regression. The method includes transmitting, to an escalation engine, an event based on the potential software regression and the at least one potential root cause.

Yet another embodiment provides a non-transitory computer-readable medium including instructions executable by an electronic processor to perform a set of functions. The set of functions includes receiving, from a software updater, a health request associated with the software payload. The set of functions includes, in response to receiving the health request, retrieving, from an escalation engine, a plurality of identifiers identifying a subset of the plurality of computing platforms that have completed deployment of the software payload. The set of functions includes determining a plurality of unified logging service (ULS) tags associated with the software payload. The set of functions includes querying an anomaly detector for failure data, for the subset of the plurality of computing platforms, corresponding to the ULS tags, the failure data including pre-deployment failure data and post-deployment failure data. The set of functions includes detecting a potential software regression associated with the software payload by comparing the pre-deployment failure data and the post-deployment failure data. The set of functions includes, in response to detecting a potential software regression, querying a root cause analyzer based on the potential software regression. The set of functions includes receiving, from the root cause analyzer, at least one change list identifier identifying at least one potential root cause for the potential software regression. The set of functions includes transmitting, to an escalation engine, an event based on the potential software regression and the at least one potential root cause.

