TECHNICAL FIELD

The present disclosure generally relates to computerized systems and methods for an experiment platform engine. In particular, embodiments of the present disclosure relate to inventive and unconventional systems used for the configuration and computation of metrics required to interpret the results of A/B tests.

BACKGROUND

A/B testing, also known as “randomized controlled experiment”, is a methodology for improving processes, often relating with user experiences of computer systems. Resulting data obtained from this methodology may be used to improve various technical aspects of the computer systems to allow for quicker operation, easier use, reduced errors rate, optimized user interfaces, and other enhancements to the user satisfaction of the computer systems.

One aspect of A/B testing is the processing of the data to obtain useful results. For a given system, or among several systems, numerous A/B tests may be designed for each operation or sub-system. Each test may in turn generate different types of data, which may in some cases require different processing. An experiment platform engine may be utilized to process and analyze these data to obtain the useful results.

Existing experiment platform engines may be limited in configurability, processing speed and throughput, thus reducing the overall capability and utility. In some instances, one A/B test may differ significantly from another A/B test, thus one experiment platform engine may be ill suited to analyze data of both A/B tests without extensive customization. Even for related A/B tests, minor variations may require modification to the experiment platform engine, which may cause delay to the overall system operation. Delays may reduce the throughput of the experiment platform engines such that the overall number of tests and data being conducted and processed are less than optimal.

Therefore, there is a need for improved methods and systems to implement experiment platform engine.

SUMMARY

One aspect of the present disclosure is directed to a computerized method for controlled testing, comprising: providing a design specification for event data, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and generating a user interface to display the output data.

Another aspect of the present disclosure is directed to a method for controlled testing, comprising: an user interface; a memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising: providing a design specification for event data via the user interface, the design specification including at least one of an event data source, an event data filter, and an event format; retrieving testing metrics; configuring testing events based on the design specification, by: retrieving testing events from the event data source specified in the design specification; filtering the retrieved testing events based in the design specification; and formatting the filtered testing events based on the design specification; generating analysis data by applying testing metrics to the configured event data; generating output data based on testing rules; and displaying the output data on the user interface.

Yet another aspect of the present disclosure is directed to a system platform for controlled testing, comprising; a user interface display; one or more database; one or more memory storing computer readable instruction; and at least one processor configured to execute the stored instructions to perform the step of: providing a design specification in form of a standardized design document specifying: at least one of event data source, an event data filter, and an event format; one or more testing metric sources; and one or more testing rule sources; retrieving testing metrics from the designated testing source stored in the one or more database; configuring testing events based on the design specification by: retrieving testing events from the event data source stored in the one or more database; filtering the retrieved testing events based on at least one of a testing period, testing region, or an event data type; and formatting the filtered testing events based on one or more event data format rules based on the design specification; generating analysis data by applying testing metric to the configured event data; generating output data based on testing rules; and generating the user interface to display the output data.

Other systems, methods, and computer-readable media are also discussed herein.

