Results: Did one version outperform?

Similar "Download Trial" CTA

Click here to enlarge

Did one version outperform?

Loading ... Loading ...

Difference Between Versions:

Version A – “Download Trial” CTA
Version B – Similar “Download Trial” CTA


Key Performance Indicator (KPI): 

Orders/visitor


Test Goal:

Increase clicks on the "Download Trial" button

 

Traffic Source:

Direct

 

Audience: 

Included all visitors from the U.S. and Canada

 

Organization: 

Autodesk - Makers of AutoCAD, the 3D CAD modeling software

Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test
This week’s Featured Test is brought to you by:

Test Run By and For

Autodesk

Test Run On

Adobe Target

WINNING VERSION

B

Poll Results - The Best Guesses:

Did one version outperform?

  • Version A
  • Version B
Loading ... Loading ...

Guess what!?! There is no difference between the variants.

So, if you spent a lot of time go back and forth, squinting, trying to determine what changed between the two versions, and couldn’t find anything, worry not. You don’t need new glasses. Phew. 🙂

But. . . And, this is an important but. . . even though there was no difference between the versions, this test still achieved a statically significant difference in results!

Version B won.

Perplexed?

Read on to find out more. . .


Test Details & Background:

Autodesk, the makers of AutoCAD, the popular 3D CAD modeling software conducted this A+ A/A test in-house.

An A/A test test takes a different testing approach. It pits two of the exact same versions against each other. A/A tests are performed to validate the test is set-up and working properly — and that the data coming back is clean, accurate, and reliable.

As a mature, and established, testing organization, AutoDesk decided to conduct 4 separate A/A tests, on different pages, with different amounts of traffic.

 

Poll Results - The Best Guesses:

Did one version outperform?

  • Version A
  • Version B
Loading ... Loading ...
oh_snap-mobile

Sign-up for the newsletter to get FREE gamified A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test
archived_tests-paywall_image jpg

5
COMMENTS: Tell Us Your Thoughts.

avatar
3 Comment threads
2 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
3 Comment authors
Deborah OWardSteven Graff Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Steven Graff
Guest
Steven Graff

Often use A/A tests to confirm testing tool validity or test is set up correctly.

Ward
Guest
Ward

Being relatively new in the field, I was wondering how the sample size was determined? If I use an online tool, it seems that both variations are at least 1400 visitors short to be statistically siginificant.

Other Tests You Might Enjoy

Test Run By and For

Autodesk

Test Run On

Adobe Target