All About Bayesian: A Look At A New Statistical Model For A/B Testing

By: Deborah O’Malley| 2019

Bayesian Statistics isn’t new. It’s been around for a while. Since the 1770’s, in fact.

But, its just now making inroads as computing capabilities improve and marketers become more savvy about proper A/B testing techniques. Case in point, A/B testing software company, Visual Website Optimizer (VWO), recently started using Bayesian Statistics in their new SmartStats A/B testing engine.

What is Bayesian Statistics?

Bayesian statistics is a statistical model that can be applied to A/B testing to more accurately determine the precise range of your conversion rate, answering the question: “What’s the probability of version B beating A?”

Named after the British mathematician, Thomas Bayes, the model is based on “Bayes Theorem,” also called Bayes’ Rule. The theorem is expressed with this formula:

Breaking down the formula:

A and B are considered events
P(A|B) is the conditional probability of A occurring, given B is true
P(B|A) is the conditional probability of B occurring, given A is true
P(A) is the prior probability A will occur
P(B) is the prior probability B will occur

Is your head spinning yet? Stick with it, it’ll all makes sense in a minute. . .

First though, you need to accept the mindset that, in Bayesian reasoning, you can continually update your beliefs about data, as your gather evidence. In A/B testing, you gather evidence by running a test. After observing the evidence, your opinions may change. This is the idea behind probability.

Probability Defined

Oh, snap! Only Pro Members
Have Full Access To This Post

Unlock Access. Sign up to become a Pro Member. Get complete access to this helpful content, plus so much more.

Get Pro Membership

Pro Member Benefits

On top of valuable information, articles and resources,
Pro Members enjoy many exclusive benifits, including:

Full access to library of
Archived Tests

Get rich A/B testing insights, tangible
takeways, and valuable insights with full
access to the ever-growing library of
Archived Tests.

Real-life data from
big brands

Show bosses, web teams, and clients

real-life data on what works and doesn't

with testing, based on eye-opening results

from companies similar to yours

Personalized survey results
to keep track of your wins

Take your best guesses and track your
successes with personalized poll results.
Then, share your glory with contacts
and colleagues.

Communicate directly
with the experts

Take advantage of the exclusive
opportunity to speak directly with
testing strategists, agencies, and CRO
specialists through interactive
webinars and tutorials.