Skip to main content

Managing A/B Tests

A quick guide to running your A/B Test and choosing a winning variant with confidence.

Written by Cassandra Campbell

Optimize your results by easily creating A/B Tests. After you tweak different versions of each campaign form (called variants), BDOW! will automatically balance the distribution according to your specifications and keep track of the performance.

From the Forms > Tests page, you can create a new test variant of an existing campaign form. The test will initialize by cloning the form, which you can then modify by navigating to Use This > Edit.

Running a successful A/B Test

Now that you've created your first A/B Test, let's go over how you can set up your testing parameters, monitor performance between your variants, and end testing when you've found the best variant for gaining the most conversions.

Adjusting the traffic percentages for variants

In a Test using two different variants, each will have a 50% chance of being shown by default. If you want to adjust how much each is weighted against your incoming site traffic, you can change the sliders under the % of Traffic column in the Tests Dashboard:

Understanding your stats

The Tests Dashboard is also where you can see a breakdown of different stats to gauge how your Test is performing.

Conversions and conversion rates

  • Views. Number of times each form variant was displayed.

  • Conversions. The number of times each form variant was successful (i.e., an email address was submitted or a Call to Action was clicked).

  • Conversion Rate. The percentage of conversions to views.

Statistical Significance

Very important! Statistical Significance shows the accuracy of your A/B Test results. You shouldn't end a test with lower than 95% Statistical Significance in order to make sure that the results are useful.

How long this takes will depend on:

  • How many variants you're testing

  • How much traffic you have seeing the test campaign

  • The conversion rate of your baseline (lower = slower significance increase)

  • The improvement of variants vs. the baseline (higher improvement = faster significance increase)

Improvement

Baseline is the original campaign form that you're testing against the variants. Each variant will show a percentage compared to the baseline's performance.

Ending a test and choosing a winner

Once your test has been running for a while and you're confident in the results (remember, wait until you have 95%+ Statistical Significance!), it's time to select a winner.

Open up the Tests view and click the Use This button next to the variant you want to select as the winner of the Test (the selection button is the dropdown "root").

Note: Be careful! Once you select a winner, all other variants in the Test will be lost and the winner will replace the Baseline campaign form!

What if I want to keep one of the variants?

Before picking a winner, make a copy of the lower-performing variant form if you want to preserve it. To do so, select Use This > Copy as Form from the dropdown menu next to it.

Did this answer your question?