Difference Between Versions:
Version A – One-button CTA: “Shop Bath and Shower”
Version B – Three-button CTA: “Body Scrubs,” “Shower Oils,” “Gift Sets”
Key Performance Indicator (KPI):
Average order value
All except referral traffic
Primarily women, aged 20-50
Sabon NYC: Luxury bath and body products retailer
Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.
Become a More Successful Digital MarketerPro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage
This week’s Featured Test is brought to you by:
Poll Results - The Best Guesses:
Test Details & Background:
Sabon NYC is a luxury bath and body care products retailer.
Looking to increase online product sales, they turned to UXAlley, a New York-based user experience agency, focused on optimizing websites and conversions.
Noticing that shoppers were drawn to many of Sabon’s pampering products, the team wondered if it would work best to feature several product Call To Action (CTA) buttons in the homepage banner – rather than just one primary CTA.
The team suspected giving users more options to personalize their experience, and deep dive into product categories of interest, would work best.
As a result, showing three buttons on the homepage banner would increase website engagement, generating higher clickthrough rates (CTRs) to products, compared to the banner with just one button.
With higher CTRs, the team also expected revenue per session would increase.
However, they acknowledged that presenting too many competing options, through multiple CTAs buttons, could create confusion, or hesitation, diminishing conversions.
So, decided to test what worked best.
To determine which CTA button banner would win, the testing team set-up and ran an A/B test on the powerful personalization platform, Dynamic Yield.
The test ran for three weeks. During this time, 82,757 visitors took part. Traffic was split 50/50.
Half of visitors saw the original one-button CTA homepage banner. It looked like this:
The other half of visitors saw the three-button CTA banner, which looked like this:
CTRs to the product page, and revenue per session were tracked across both versions.
The Real-Life Results:
Winner: Version B – the three-button banner was the big winner.
Adding the additional CTA buttons worked wonders.
On desktop, the 3-button design generated a 7.37% higher CTR, compared to the 1-button banner.
On mobile, the CTR uplift was even higher, at 11.67%.
The increased CTR translated to significantly higher revenue per session. With the winning three-button design, revenue increased by 23%, compared to the one-button banner.
Results achieved 95% confidence.
How Trustworthy Are The Results?
Given that the sample size was large (82,000+ visitors), the test was run for a sufficient time period (3 weeks), and results achieved 95% confidence, the test appears to hold strong validity.
It’s interesting to note, following this test, the testing team tried the three-button vs. one-button approach on other banners and promotions, and continued to achieve similar success.
Based on continuous testing, the team is confident the multi-button approach works best for generating higher CTRs, and subsequent revenue.
Why did the three-button CTA design win? There’s likely a trifecta of reasons:
1. Immediately presented new offering to users who didn’t otherwise navigate
When users landed on the site, the banner design with three buttons immediately presented multiple product categories options.
Unless users took the time took explore the site – and surf through the top navigational menu – they wouldn’t have known these product options existed.
Adding multiple CTAs created an easy way to exposed new categories, and product offerings, to users who may not have otherwise navigated to them.
2. Helped shoppers hone in on their category of interest
With multiple product categories right in front of them, more shoppers became aware of the product options.
With heightened awareness, they were able to delve into the specific categories of interest to them.
In the one-button design, not all shoppers were looking for bath and shower supplies. But, with the three button design, many more shoppers were grabbed by the popular product categories of body scrubs, shower oils, and gift sets, and continued exploring from there.
3. Provided an easy access shortcut
For return visitors, who already knew what products they wanted to explore, the multiple CTA design more easily facilitated the navigation.
It provided an easy access shortcut. Shoppers didn’t have to sift through the extensive dropdown top navigational menu, or fumble through the product filter system.
Rather, they could pinpoint an exact product of interest, and start their shopping journey from there.
The three-button design made the shopper’s desired navigational starting point that much simpler and obvious to find. With increased engagement, more shoppers were able to find products of interest, and ultimately purchase them.
Tangible Takeaway & Immediate Application
While we tend to believe users need to be directed to one clear, strong CTA, this test shows sometimes more is better – especially when the additional choice target users’ needs, and helps direct them to specific categories of interest.
What Do You Think?
Why do you think the three-button design won? Share your thoughts in the comment section below.