<commit_before>---
layout: seo.hbs
seo_title: Comparing a Multivariate Test to an A/B Test
visible_title: Comparing a Multivariate Test to an A/B Test
body_class:
- mvt
body: |
 #### The Difference Between a Multivariate Test and an A/B Test

 What is the difference between A/B testing and multivariate testing? Let's take a look at the methodology, common uses, advantages, and limitations of these testing methods.

 #### A/B Testing

 A/B Testing, which you may also have heard referred to as split testing, is a method of website optimization in which the conversion rates of two versions of a page — version A and version B — are compared to one another using live traffic. Site visitors are bucketed into one version or the other. By tracking the way visitors interact with the page they are shown — the videos they watch, the buttons they click, or whether or not they sign up for a newsletter — you can determine which version of the page is most effective.

 ![Tracks clicks on CTA](https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-assets.cloudvent.net/raw/seo-pages/ab-flow-cta.png)

 ##### Common Uses

 A/B Testing is the least complex method of evaluating a page design, and is useful in a variety of situations.

 One of the most common ways A/B testing is utilized is to test two very different design directions against one another. For example, the current version of a company's home page might have in-text calls to action, while the new version might eliminate most text, but include a new top bar advertising the latest product. After enough visitors have been funneled to both pages, the number of clicks on each page's version of the call to action can be compared. It's important to note that even though many design elements are changed in this kind of A/B test, only the impact of the design as a whole on each page's business goal is tracked, not individual elements.

 A/B testing is also useful as an optimization option for pages where only one element is up for debate. For example, a pet store running an A/B test on their site might find that 85% more users are willing to sign up for a newsletter held up by a cartoon mouse than they are for one emerging from the coils of a boa constrictor. When A/B testing is used in this way, a third or even fourth version of the page is often included in the test, which is sometimes called an A/B/C/D test. This, of course, means that traffic to the site must be split into thirds or fourths, with a lesser percentage of visitors visiting each site.

 ##### Advantages

 Simple in concept and design, A/B testing is a powerful and widely used testing method.

 Keeping the number of tracked variables small means these tests can deliver reliable data very quickly, as they do not require a large amount of traffic to run. This is especially helpful if your site has a small number of daily visitors.  Splitting traffic into more than three or four segments would make it hard to finish a test. In fact, A/B testing is so speedy and easy to interpret that some large sites use it as their primary testing method, running cycles of tests one after another rather than more complex multivariate tests.

 A/B testing is also a good way to introduce the concept of optimization through testing to a skeptical team, as it can quickly demonstrate the quantifiable impact of a simple design change.

 ##### Limitations

 A/B testing is a versatile tool, and when paired with smart experiment design and a commitment to iterative cycles of testing and redesign, it can help you make huge improvements to your site. However, it is important to remember that the limitations of this kind of test are summed up in the name. A/B testing is best used to measure the impact of two to four variables on interactions with the page. Tests with more variables take longer to run, and in and of itself, A/B testing will not reveal any information about interaction between variables on a single page.

 If you need information about how many different elements interact with one another, multivariate testing is the optimal approach!

 #### Multivariate Testing

 Multivariate testing uses the same core mechanism as A/B testing, but compares a higher number of variables, and reveals more information about how these variables interact with one another. As in an A/B test, traffic to a page is split between different versions of the design. The purpose of a multivariate test, then, is to measure the effectiveness each design combination has on the ultimate goal.

 Once a site has received enough traffic to run the test, the data from each variation is compared to find not only the most successful design, but also to potentially reveal which elements have the greatest positive or negative impact on a visitor's interaction.

 ![Browser](https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-assets.cloudvent.net/raw/seo-pages/mvt_browser-table.png)

 ##### Common Uses
<commit_msg>fix inconsitent capitalization for A/B testing https://github.com/optimizely/marketing-website/issues/251<commit_after>---
layout: seo.hbs
seo_title: Comparing a Multivariate Test to an A/B Test
visible_title: Comparing a Multivariate Test to an A/B Test
body_class:
- mvt
body: |
 #### The Difference Between a Multivariate Test and an A/B Test

 What is the difference between A/B testing and multivariate testing? Let's take a look at the methodology, common uses, advantages, and limitations of these testing methods.

 #### A/B Testing

 A/B testing, which you may also have heard referred to as split testing, is a method of website optimization in which the conversion rates of two versions of a page — version A and version B — are compared to one another using live traffic. Site visitors are bucketed into one version or the other. By tracking the way visitors interact with the page they are shown — the videos they watch, the buttons they click, or whether or not they sign up for a newsletter — you can determine which version of the page is most effective.

 ![Tracks clicks on CTA](https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-assets.cloudvent.net/raw/seo-pages/ab-flow-cta.png)

 ##### Common Uses

 A/B testing is the least complex method of evaluating a page design, and is useful in a variety of situations.

 One of the most common ways A/B testing is utilized is to test two very different design directions against one another. For example, the current version of a company's home page might have in-text calls to action, while the new version might eliminate most text, but include a new top bar advertising the latest product. After enough visitors have been funneled to both pages, the number of clicks on each page's version of the call to action can be compared. It's important to note that even though many design elements are changed in this kind of A/B test, only the impact of the design as a whole on each page's business goal is tracked, not individual elements.

 A/B testing is also useful as an optimization option for pages where only one element is up for debate. For example, a pet store running an A/B test on their site might find that 85% more users are willing to sign up for a newsletter held up by a cartoon mouse than they are for one emerging from the coils of a boa constrictor. When A/B testing is used in this way, a third or even fourth version of the page is often included in the test, which is sometimes called an A/B/C/D test. This, of course, means that traffic to the site must be split into thirds or fourths, with a lesser percentage of visitors visiting each site.

 ##### Advantages

 Simple in concept and design, A/B testing is a powerful and widely used testing method.

 Keeping the number of tracked variables small means these tests can deliver reliable data very quickly, as they do not require a large amount of traffic to run. This is especially helpful if your site has a small number of daily visitors.  Splitting traffic into more than three or four segments would make it hard to finish a test. In fact, A/B testing is so speedy and easy to interpret that some large sites use it as their primary testing method, running cycles of tests one after another rather than more complex multivariate tests.

 A/B testing is also a good way to introduce the concept of optimization through testing to a skeptical team, as it can quickly demonstrate the quantifiable impact of a simple design change.

 ##### Limitations

 A/B testing is a versatile tool, and when paired with smart experiment design and a commitment to iterative cycles of testing and redesign, it can help you make huge improvements to your site. However, it is important to remember that the limitations of this kind of test are summed up in the name. A/B testing is best used to measure the impact of two to four variables on interactions with the page. Tests with more variables take longer to run, and in and of itself, A/B testing will not reveal any information about interaction between variables on a single page.

 If you need information about how many different elements interact with one another, multivariate testing is the optimal approach!

 #### Multivariate Testing

 Multivariate testing uses the same core mechanism as A/B testing, but compares a higher number of variables, and reveals more information about how these variables interact with one another. As in an A/B test, traffic to a page is split between different versions of the design. The purpose of a multivariate test, then, is to measure the effectiveness each design combination has on the ultimate goal.

 Once a site has received enough traffic to run the test, the data from each variation is compared to find not only the most successful design, but also to potentially reveal which elements have the greatest positive or negative impact on a visitor's interaction.

 ![Browser](https://d1qmdf3vop2l07.cloudfront.net/optimizely-marketer-assets.cloudvent.net/raw/seo-pages/mvt_browser-table.png)

 ##### Common Uses

{'fraction_non_alphanumeric': 0.03917649410353788, 'fraction_numerical': 0.0026983809714171496, 'mean_word_length': 4.294708994708995, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5625692', 'n_tokens_mistral': 2412, 'n_tokens_neox': 2258, 'n_words': 1604}