Hey guys,
Today I am going to discuss one of CRO’s most commonly discussed and debated themes - website traffic. How many site visitors do we need to run accurate A/B tests, and what to do if you don’t have enough but still want to begin your CRO journey?
Weekly Insight
Fresh news
Additional Readings
When it comes to CRO, and specifically A/B testing, one of the most common questions that come up is “how much website traffic is required to run a successful CRO program?”
Depending on who you ask or which article you read, you’ll hear many different answers. Some might say traffic doesn’t matter and you can start with 1000 monthly unique visitors, another might say you need minimum 10k and a third might say unless you have at least 50k, you’re wasting your time.
The simple answer to explain all of the above has to do with two words:
Yes, I know it sounds fancy and reminds you of that dreadful maths course you failed in University, but let’s simplify it.
Statistical significance is a term used to know if a test result is likely real or just happened by chance. In simple terms, if you run a test and get a result that’s statistically significant, it means there's a high probability (usually 95% or higher) that the difference you’re seeing is genuine and not just a random fluke.
For example, in CRO, if you test two versions of a webpage and version B performs better than version A with statistical significance, you can feel confident that version B really is more effective, and it’s not just a coincidence based on the specific visitors who saw it.
Make sense?
In even simpler terms, when you run an A/B test, you need to allow a certain amount of website visitors to ‘pass through’ the test before you can conclude that the test results are accurate or happening by chance. Once enough visitors have passed through each variation, we can only then conclude which version performed better and move forward.
The goal is to run tests that reach statistical significance (typically 95%) within a reasonable timeframe, ideally under a few weeks. This ensures your findings are reliable without overextending the testing period.
Well, with very low website visitors (eg. 10k monthly), in order to allow each variation of the test to get at least 5k visitors passing through each variant (since we’re testing the entire 10k), you will need to wait the entire month (maybe even longer) until you have a statistically significant result.
Conversely, a website getting 100k monthly visitors might run a test where statistical significance is reached far quicker and tests can therefore run for much shorter periods of time, allowing for more tests to be run per month, resulting in more ‘winning’ tests, resulting in a greater ROI.
In summary, having fewer website visitors does not mean you cannot run A/B tests, it just means that you will only be able to run a limited number of tests per month as each test needs a longer time period to reach statistical significance.
At Hype, we try to set a threshold of around minimum 20k monthly visitors when taking on new clients, although more is always better. This allows us to still run at least 2 tests per month, with the option of scale as traffic increases through various traffic acquisition sources.
If you’re unsure if you have enough traffic to start testing, consider alternative approaches to testing while you focus on ramping up your traffic.
Broader conversion optimization methods, such as qualitative analysis (user feedback, heat maps) or testing on pages with the most traffic and impact (e.g., checkout or product pages) can still provide tons of insight and value without A/B testing.
Bottom line - traffic is important, but don’t turn your head away from CRO entirely just because you lack it. There’s lots you can do in the meantime while you put the necessary foundations in place to get your site to a place where testing can begin.
See some additional readings below which will give you some further strategies on what you can do while you work on increasing your traffic numbers.
CRO for Low Website Traffic: 7 Tactics for Optimizing Without A/B Testing by Mouseflow
Conversion Rate Optimization Research Strategies for Low-Traffic Websites by GrowthSavvy
6 Conversion Rate Optimization (CRO) Tips For Low Traffic Websites by Search Engine Journal
That’s it for now.
See you next week!
Aharon & The Hype Team
Who are we and how we can help you?
Hype Digital is a Conversion Optimization agency specializing in improving your website and in-app conversion rates.
We help you understand what your customers want and how to get them to do what you want them to do.
We've helped 100s of brands increase conversion rates through research and experimentation.
For a FREE chat, reach out to us at hello@hypedigital.co or contact us through our website -> www.hypedigital.coNo posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.