The group of users who take part in your experiment comprise the sample population.
When running an experiment, it’s important to have a large enough sample size so the portion of users (the sample population) accurately represents your entire audience.
If your sample size is too small, your test results will not be statistically valid or reliable at a high level of confidence. In other words, results may not be accurate because they won’t be representative of how your entire audience actually behaves.
As a very general rule of thumb, for a highly reliable test, you need a minimum of 1,000 visitors and 100 conversions per variant.
If you follow this guideline, you’ll typically achieve enough traffic and conversions to derive statistically significant results at a high level of confidence.
However, this amount is a suggested guideline. As this article explains, it's best to calculate your sample size requirements.« Back to Glossary Index