Free Sign up
Member Login

Everything You’ve Learned About A/B Testing Is Wrong!

By: Deborah O’Malley| 2019

There’s some bad news and good news.

Let’s get the bad news out of the way. Then, go into the good news.

The bad news is: what you’ve been taught about how to calculate a valid A/B test is probably wrong!

That cause you’ve probably been taught using Frequentist Statistics.

Frequentist Statistics

While there’s nothing wrong with Frequentist Statistics, the model doesn’t work that well when applied to A/B testing.

The reason why is because the Frequentist approach can’t accurately answer the question “What’s the probability version B will perform better than version A?” Or vice versa.

In Frequentist statistics, the only way to validly address this question is by stating a null hypothesis. And here’s where it gets more mind boggling. . .

Null Hypothesis

Oh, snap! Only Pro Members Have Full Access To This Post

Unlock Access. Sign up to become a Pro Member. Get complete access to this helpful content, plus so much more.


Notify of
Newest Most Voted
Inline Feedbacks
View all comments
4 years ago

I was surprised to know that till now the marketers were carrying out the A/B testing in a wrong way. Still, I believe that the marketing automation platform tool which I use does it in a better idea as I can enhance the conversions.

Other Posts You Might Enjoy

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram