All About Bayesian: A Look At A New Statistical Model For A/B Testing

By: Deborah O’Malley| 2019

Bayesian Statistics isn’t new. It’s been around for a while. Since the 1770’s, in fact.

But, its just now making inroads as computing capabilities improve and marketers become more savvy about proper A/B testing techniques. Case in point, A/B testing software company, Visual Website Optimizer (VWO), recently started using Bayesian Statistics in their new SmartStats A/B testing engine. 

What is Bayesian Statistics?

Bayesian statistics is a statistical model that can be applied to A/B testing to more accurately determine the precise range of your conversion rate, answering the question: “What’s the probability of version B beating A?”

Named after the British mathematician, Thomas Bayes, the model is based on “Bayes Theorem,” also called Bayes’ Rule. The theorem is expressed with this formula:

Breaking down the formula:

  • A and B are considered events
  • P(A|B) is the conditional probability of A occurring, given B is true
  • P(B|A) is the conditional probability of B occurring, given A is true
  • P(A) is the prior probability A will occur
  • P(B) is the prior probability B will occur

Is your head spinning yet? Stick with it, it’ll all makes sense in a minute. . .

First though, you need to accept the mindset that, in Bayesian reasoning, you can continually update your beliefs about data, as your gather evidence. In A/B testing, you gather evidence by running a test. After observing the evidence, your opinions may change. This is the idea behind probability.

Probability Defined

Oh, snap! Only Pro Members
Have Full Access To This Post

Unlock Access. Sign up to become a Pro Member. Get complete access to this helpful content, plus so much more.

Group 4.1

Pro Member Benefits

On top of valuable information, articles and resources,
Pro Members enjoy many exclusive benifits, including:

Full access to library of
Archived Tests

Get rich A/B testing insights, tangible
takeways, and valuable insights with full
access to the ever-growing library of
Archived Tests.

Real-life data from
big brands

Show bosses, web teams, and clients

real-life data on what works and doesn't

with testing, based on eye-opening results

from companies similar to yours

Personalized survey results
to keep track of your wins

Take your best guesses and track your
successes with personalized poll results.
Then, share your glory with contacts
and colleagues.

Communicate directly
with the experts

Take advantage of the exclusive
opportunity to speak directly with
testing strategists, agencies, and CRO
specialists through interactive
webinars and tutorials.

Immediate answers
to questions

Reduce frustration by getting immediate

answers to specific technical questions

from top-world experts during live,

interactive "how to" sessions.

Access to the forum

Get special access to the Community

Forum - where you can post targeted

questions and get practical answers you

can't find anywhere else.

Rewards and discounts

Get exclusive members 'rewards and
discounts on upcoming industry partners'
courses, conferences, and events.

Get replays of webinars

Always get replays of webinars you
missed and the opportunity to post
webinar questions in advance, making sure
your specific questions are answered.

Conversion audit

Receive an informative conversion Audit

on your website, from our partner,

Convert Experts, giving you solid

suggestions to improve your website.

Start your no-risk, 14-day trial for just $1

Joining is risk-free. If you’re not completely satisfied, cancel at anytime.

Pricing Options

After your 14-day, $1 trial, select the plan that works best for you:

Group Plan

normally $997/yr

$497 /yr

  • (billed annually )
  • 3 team members for the price of 2
  • Unlimited access to all webinar, tests and resources
  • Exclusive offer - Get a free Conversion Audit on your site

Annual Plan

normally $697/yr

$247 /yr

  • (billed annually)
  • Save big! Enjoy 56% off by paying once, annually
  • Complete access to the full library of archived tests
  • Exclusive access to all webinars, articles, and resources

Monthly Plan

normally $797/yr

$47 /mo

  • (billed monthly)
  • No obligation, try monthly and enjoy
  • Full access to the complete library of archived tests
  • Complete access to all webinars, articles, and resources
All prices in $USD.
Group 2

What people are saying about us

Featured Test Brands

As a Pro Member, you'll get complete access to informative
A/B tests from brands and agencies around the world, including:

Group 4
Group 6
Group 6(2)
Group 4(7)
Group 4(1)
Group 6(1)
Group 6(3)
Group 4(8)
Group 4(2)
Group 4(4)
Group 6(4)
Group 4(9)
Group 4(3)
Group 4(5)
Group 4(6)
Group 4(10)

Sign up to get full access. Become A Pro Member