Results: What’s the optimal CTA number?

Difference Between Versions:

Version A – One-button CTA: “Shop Bath and Shower”
Version B – Three-button CTA: “Body Scrubs,” “Shower Oils,” “Gift Sets”


Key Performance Indicator (KPI): 

Average order value


Test Goal:

Increase CTR to the product page, and lift revenue

 

Traffic Source:

All except referral traffic

 

Audience: 

Primarily women, aged 20-50

 

Organization: 

Sabon NYC: Luxury bath and body products retailer

Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test
This week’s Featured Test is brought to you by:

Test Run By

UX Alley

Test Run For

Sabon NYC

Test Run On

Dynamic Yield

WINNING VERSION

B

Poll Results - The Best Guesses:

Should you go with less, or more CTAs?

Loading ... Loading ...

Test Details & Background:

Sabon NYC is a luxury bath and body care products retailer.

Looking to increase online product sales, they turned to UXAlley, a New York-based user experience agency, focused on optimizing websites and conversions.

After conducting a data-driven analysis, UXAlley’s testing team isolated Sabon’s homepage hero image banner as an area of potential optimization.

Noticing that shoppers were drawn to many of Sabon’s pampering products, the team wondered if it would work best to feature several product Call To Action (CTA) buttons in the homepage banner – rather than just one primary CTA.


Hypothesis:

The team suspected giving users more options to personalize their experience, and deep dive into product categories of interest, would work best.

As a result, showing three buttons on the homepage banner would increase website engagement, generating higher clickthrough rates (CTRs) to products, compared to the banner with just one button.

With higher CTRs, the team also expected revenue per session would increase.

However, they acknowledged that presenting too many competing options, through multiple CTAs buttons, could create confusion, or hesitation, diminishing conversions.

So, decided to test what worked best.


 

Test Set-up:

To determine which CTA button banner would win, the testing team set-up and ran an A/B test on the powerful personalization platform, Dynamic Yield.

The test ran for three weeks. During this time, 82,757 visitors took part. Traffic was split 50/50.

Half of visitors saw the original one-button CTA homepage banner. It looked like this:

version_a-flash_sale-lg

The other half of visitors saw the three-button CTA banner, which looked like this:

version_b-flash_sale-lg

CTRs to the product page, and revenue per session were tracked across both versions.


The Real-Life Results:

Winner: Version B – the three-button banner was the big winner.

Adding the additional CTA buttons worked wonders.

On desktop, the 3-button design generated a 7.37% higher CTR, compared to the 1-button banner.

On mobile, the CTR uplift was even higher, at 11.67%.

The increased CTR translated to significantly higher revenue per session. With the winning three-button design, revenue increased by 23%, compared to the one-button banner.

Results achieved 95% confidence.


How Trustworthy Are The Results?

Given that the sample size was large (82,000+ visitors), the test was run for a sufficient time period (3 weeks), and results achieved 95% confidence, the test appears to hold strong validity.

It’s interesting to note, following this test, the testing team tried the three-button vs. one-button approach on other banners and promotions, and continued to achieve similar success.

Based on continuous testing, the team is confident the multi-button approach works best for generating higher CTRs, and subsequent revenue.


Analysis:

Why did the three-button CTA design win? There’s likely a trifecta of reasons:

1. Immediately presented new offering to users who didn’t otherwise navigate

When users landed on the site, the banner design with three buttons immediately presented multiple product categories options.

Unless users took the time took explore the site – and surf through the top navigational menu – they wouldn’t have known these product options existed.

Adding multiple CTAs created an easy way to exposed new categories, and product offerings, to users who may not have otherwise navigated to them.

2. Helped shoppers hone in on their category of interest 

With multiple product categories right in front of them, more shoppers became aware of the product options.

With heightened awareness, they were able to delve into the specific categories of interest to them.

In the one-button design, not all shoppers were looking for bath and shower supplies. But, with the three button design, many more shoppers were grabbed by the popular product categories of body scrubs, shower oils, and gift sets, and continued exploring from there.

3. Provided an easy access shortcut

For return visitors, who already knew what products they wanted to explore, the multiple CTA design more easily facilitated the navigation.

It provided an easy access shortcut. Shoppers didn’t have to sift through the extensive dropdown top navigational menu, or fumble through the product filter system.

Rather, they could pinpoint an exact product of interest, and start their shopping journey from there.

The three-button design made the shopper’s desired navigational starting point that much simpler and obvious to find. With increased engagement, more shoppers were able to find products of interest, and ultimately purchase them.


Tangible Takeaway & Immediate Application

While we tend to believe users need to be directed to one clear, strong CTA, this test shows sometimes more is better – especially when the additional choice target users’ needs, and helps direct them to specific categories of interest.


What Do You Think?

Why do you think the three-button design won? Share your thoughts in the comment section below.

Poll Results - The Best Guesses:

Should you go with less, or more CTAs?

Loading ... Loading ...

Sign-up for the newsletter to get FREE gamified A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test

5
COMMENTS: Tell Us Your Thoughts.

avatar
3 Comment threads
2 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
3 Comment authors
Deborah ORyan RSandis Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Sandis
Guest
Sandis

Love it! Thank you, Deborah and UXAlley, for sharing this one.

I went for B as it showed segmentation within the same niche. It resembles the old-time classic issue of not having categories high enough vs fancy slider.

So far so good, but then I realized I have some work to do as well as a result of this!

Ryan R
Guest
Ryan R

I guessed A. I assumed the 3 categories shown were too limiting in their scope. First time I’ve ever been wrong.

Other Tests You Might Enjoy

Test Run By

UX Alley

Test Run For

Sabon NYC

Test Run On

Dynamic Yield