“In marketing and business intelligence, A/B testing is jargon for a randomized experiment with two variants, A and B, which are the control and treatment in the controlled experiment.In online settings, such as web design (especially user experience design), the goal is to identify changes to web pages that increase or maximize an outcome of interest (e.g., click-through rate for a banner advertisement). Formally the current web page is associated with the null hypothesis.”
So, it’s where you as marketer/business owner get to play mad scientist with your web traffic in an attempt to improve your conversion. Cue evil laugh…
Switched-on marketers know how important A/B testing is for Conversion Rate Optimisation (CRO). We rely on it to make controlled, and statistically valid marketing decisions that can help boost business. That’s right, it’s a scientific way to get more sales!
Effective A/B testing lets us know what does work and what doesn’t. In a world where online competition gets tougher every minute, we need to be on guard and do WHATEVER it takes to protect & grow our business. In case you’ve never done any A/B testing for your website, here are some reasons why you must start now:
- You can never really know which pages of your website suck unless you start A/B testing them
- A/B testing proves or disproves any hypothesis based on eyent tracking or surveys you have on your website
- A/B testing output tells you why people act the way they do rather than just assuming how they think
- A/B testing is a powerful tool for landing page optimization and website conversion optimization.
On the other hand, if you’re already an A/B testing junkie and can’t get enough running awesome tests on your website to continually optimize conversion, we are happy for you! But the true measure that a “test” works is if it generates leads and increases ROI. At the end of the day, the goal of every business is to maximise profits. If you have done several A/B testing, but still lack results, maybe you are missing something. To further help you with your quest in finding the holy grail of CRO and A/B testing, below are 10 signs that you are doing it right:
- You forget everything you think you know about your customers. It’s always tempting to make assumptions about how your customers act and think. Although that may be helpful in creating a hypothesis, it is always best to strip yourself off of any assumptions and be unbiased towards A/B testing
- You are always establishing a baseline. No matter how quickly you want to see results and go after high stakes A/B testing, you know the importance of establishing a baseline to measure against.
- You know that while it worked for others, that doesn’t mean it will also work for you. Because if it does, there will be no more need for testing.
- You test one thing at a time. It is a must to test one variable at a time if you want your A/B testing results to be readable. If you change two things on your website and run A/B test, how would you know which one contributed to much of the conversions?
- You know it’s not done yet until statistical confidence is reached. It is easy to call it off once you see favorable results, but are you sure that the same results can be expected when you run the same tests? Statistical confidence is highly dependent on your sample size, margin of error, and population size.
- You learn along the way. One does not simply master A/B testing in a day and know the fail-proof secrets to CRO. It’s a continuous process and you will make mistakes and learn from them along the way.
- You value second opinion. Any feedback is better than no feedback. Use some other tools for feedback like Peek User Testing and Amazon Turk.
- You understand that user behavior data and customer survey data may conflict. Human decisions vary. Though survey plays an important role in marketing, sometimes people act differently than the way they think. If a person said she would rather make her own coffee, but bought a cup from a café instead, then there’s a conflict and you acknowledge it.
- You clearly define your success metric. CRO is about conversions. Make sure you measure your success accordingly and don’t be carried away by click-through rates, shares, or tweets.
- You don’t test everything! You don’t need to test every detail on your site to increase conversion rates. Trust your intuition, use your common sense, and don’t waste your time on unnecessary things.
Happy A/B testing, & feel free to drop us a line if you want more info or help. Cheers, Darren