So you’re doing some split testing right? I’m sure you are (or should be! I mean it!). Anyway, are you sure that the results you’re having are significant, statistically significant?
Here’s what I mean
Suppose you have two different banners. No matter if you’re a web designer or an internet marketer, either way, you want to know which one is better. The easiest thing to do is to perform some A/B split testing.
After the test is over you find out that 600 impressions of banner A resulted in 50 clicks. However, 600 impressions of banner B resulted in 70 clicks. Great! So banner B is the better one you may think. The only problem is that these results are not statistically significant. This means that the results should not really be considered when deciding which banner is the better one. It seems like banner B is better but the number of impressions and clicks is so small that the results might be completely accidental. How to find out whether or not the results after a split test are significant?
There’s a solution
You just have to get yourself a cool, easy to use mathematical tool.
Introducing the statistical significance checker:
It’s really easy to use. Just input the number of trials and the number of actions for both variants of your A/B split test and press the “calculate” button.
This tool will tell you whether or not your results are significant.
Here’s an example:
And another, significant one:
As you can see you will also get a level of certainty associated with your test. (A hint: everything above 95% is a great result.)
Now the best part. The tool is free. Just use the link below to download it. Have fun!
(Quick note. An internet connection is required for this tool to work.
The tool runs on Windows and needs a thingy called .NET Framework.)
Here’s a list of articles you may also enjoy:
- A/B testing – Wikipedia, the free encyclopedia – A/B testing or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates…
- The Ultimate Guide To A/B Testing – /B testing isn’t a buzz term. A lot of savvy marketers and designs are using it right now to gain insight into visitor behavior and to increase conversion rate…
- Perform Split Testing with phpA/B – Split testing, also called A/B testing, is a testing method where you compare a base sample with another test sample, changing one specific variable inside it.
- How to increase downloads for your app by 85%? (Hint: include more … – I ¦ Split Testing. and so should you. … Tags: A/B Split Testing, ab testing, increase downloads, software downloads, split testing. Posted in A/B Split Testing, Case Studies on August 2nd, 2010 | No Comments » …
- 30 Multivariate & A/B Split Testing Tools, Tutorials & Resources … – Crossroads by Captain DJ. Getting targeted traffic for a website is just the first step in a thorough SEO process. What comes next? Landing page.
- A/B Split Testing – A/B Split Testing — How to use A/B Split Testing to Increase Conversion Rates, Challenge Assumptions and Solve Problems…