guess_the_test-white-green
Member Login
Member
Login
guess_the_test-white-green

Frequentist testing methodology

« Back to Glossary Index

As statistical method of calculation used in traditional hypothesis-based A/B testing.

In frequentist statistics, your aim is to prove or disprove the null hypothesis.

The null hypothesis assumes there is no difference between versions.

When you carry out an A/B test using frequentist statistics, your aim is to prove or disprove the null hypothesis.

The null hypothesis is disproven if a winning test result emerges showing a conversion difference between versions.

Although typically used in most A/B testing platforms, the drawback of the frequentist method is it requires a large sample size and a valid testing timeframe in order to declare statistically significant results with a high level of confidence.

In contrast, the Bayesian methodology enables testers to run an experiment with a much smaller sample size within a much shorter testing timeframe.

The tradeoff is a sacrifice of statistical rigour only possible through a properly carried out hypothesis-based A/B test experiment.

The reason is because a test run with the Bayesian method states only the likely probability the outcome will occur.

« Back to Glossary Index
hello world!
eyechevron-downarrow-circle-upcrossmenu-circlecross-circle linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram