FIELD OF TECHNOLOGY

The embodiments described herein pertain in general to conducting controlled, randomized experiments and more particularly to determining which version of content undergoing an experiment to display to users.

BACKGROUND

Content variation testing is an experiment that tests changes to content against a current version of the content. One example of content variation testing is web page variation testing where an experiment is conducted that tests changes to a web page against another design of the web page. During variation testing of a web page, changes to a web page that produce positive results are determined. The positive results associated with a particular change to a web page validates that the change to the web page should be used in a production version of the web page.

In conventional variation testing, when a user requests a web page from a website, the website may determine rules associated with variation test for the web page. For example, the rules may specify that one thousand users view a first version of a web page whereas another thousand users view a second version of the web page. When a user request a web page, the user may be randomly assigned to either the first version of the web page or the second version of the web page according to the rules. However, conventional variation testing techniques result in a user seeing different versions of a web page when the user uses different devices or web browsers to view the web page. This degrades the user's experience as the user experiences inconsistency when visiting the web page using different devices or web browsers.

BRIEF SUMMARY

The embodiments herein describe a variation testing system environment for performing variation testing of a web page. Users requesting a web page from a content provider are not randomly assigned to one of a plurality of variations of the web page. In one embodiment, each user's identifier is used as a basis for determining which version of the web page is provided to a client device of the user. Each user identifier is input into a hash function to generate an assignment identifier based on the user identifier. According to the value of each assignment identifier, a variation of the web page is selected and the selected variation of the web page is provided to each user based on the assignment identifier. As long as a given user maintains the same user identifier, the user will receive the same variation of the web page during visits to the web page using different client devices or web browsers.

Features and advantages described in this summary and the following detailed description are not all-inclusive. Many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims hereof.

