Difference Between Versions:
Version A – Free shipping offer, for orders over $99
Version B – Same offer, but for order over $199
Key Performance Indicator (KPI):
Average order value
Determine if users will add more to their cart to receive free shipping
Direct, search, referral, organic, paid, affiliate, social, and existing customer email list
SpiritHoods: Animal-inspired faux fur clothing products
Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.
Become a More Successful Digital MarketerPro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage
This week’s Featured Test is brought to you by:
Poll Results - The Best Guesses:
Read on to learn more. . .
Test Details & Background:
Will users spend more to receive free shipping?
That was the question Brand Value Accelerator (BVA), the eCommerce optimization agency, set-out to test for their client, SpiritHoods, a unique retailer offering animal-inspired faux fur clothing and products.
Looking to increase SpirtHood’s sales, the testing team wondered what the optimal free shipping threshold was — to get people spending more, to qualify for free shipping.
The team suspected that the higher the shipping threshold, the more likely shoppers were to add products, to take advantage of the free shipping.
However, the team also acknowledged that if the free shipping offer was too high, customers might not convert. And, if it was too low, there could be a missed sales opportunity.
So, the team decided to test what shipping threshold worked best.
To determine the optimal price point to get shoppers adding items to take advantage of free shipping, an A/B test was set-up and run on Optimizely. The test ran for 2 weeks.
During this time, approximately 40 thousand visitors saw a banner with a different free shipping offer. Traffic was evenly split between the variants.
The banner with the $99 free shipping offer looked like this:
The banner with the $199 free shipping offer looked like this:
To determine the impact of a lower or higher free shipping threshold, Revenue Per Visitor (RPV) and Average Order Value (AOV) were tracked across the versions.
The Real-Life Results:
Winner: Version B – with the $199 shipping threshold was the big winner.
Making it so shoppers had to spend more to take advantage of free shipping had a hugely positive impact on sales.
The version with the higher shipping threshold led to a 24% increase in RPV, compared to the version with the $99 shipping offer.
As well, increasing the shipping threshold accelerated AOV by 5.6%, compared to the lower $99 offer.
Results achieved 90% confidence.
How Trustworthy Are The Results?
Given the sample size was large (40 thousand visitors), the test ran for a standard amount of time (2 weeks), and results achieved 90% confidence, it’s likely the study is highly valid.
However, it should be noted that 90% confidence means there’s a 10% probability the results are due to random chance, which is a higher odd than desired. Ideally, your test results should achieve 95% confidence or higher.
What does this test result show us about free shipping? There’s a couple insights:
1. Shoppers were motivated to spend more to get something for free
It’s important to note that this test did not look at whether a higher or lower shipping threshold attracted more customers, or brought more shoppers through the funnel.
Instead, it looked at the economic effect of increasing the shipping threshold, from $99 to $199, as measured by revenue and order value.
The study found it was most definitely worthwhile to set a higher price for free shipping, since customer were willing to buy more, in order to take advantage of the free offer.
This finding is extremely value, and can be applied to your own eCommerce site: shoppers will spend more money to get free shipping!
In fact, according to research looking at ways to influence online buying decisions, customers say free shipping is their top reason to shop online more. And, 93% of customers admit to buying more if they get free shipping.
2. Free shipping isn’t necessarily free
However, free shipping comes at a cost.
As a merchant, you might make more sales, but you also have to absorb the shipping charge.
That can hurt.
To put money back in your pocket, its worthwhile testing your site’s optimal free shipping threshold.
This test found, shoppers were likely spend more to to take advantage of the free shipping offer.
However, there’s likely a delicate balance in this equation.
If the free shipping threshold too high, shoppers might be not spring because they’ll recognize they have to buy too much, just to get free shipping.
On the other hand, if the free shipping threshold too low, you could end up losing out since shoppers may spend only what they need to get free shipping.
So, it’s best to test.
Then, once you’ve found your site’s sweet spot, it can be beneficial to continually optimize your free shipping price point.
During a time like COVID, for example, shoppers may feel they have less disposable income to spend. Your best bet may be responding by lowering your free shipping threshold.
If you look at SpiritHoods site today, you’ll see, they’ve lowered their shipping threshold from the previous $199 winning result to today’s price of $129.
Your optimal shipping threshold may be higher, or lower, depending on the average price of your products. And, depending on shifts in consumer demand.
As well, if you’re consider testing adding a free shipping notice on your site, it’s valuable to assess the optimal placement and formatting. This GuessTheTest case study offers some insight into what works well.
Tangible Takeaway & Immediate Application
Online shoppers will spend more to take advantage of free shipping.
To determine your optimal shipping threshold, its always best to test — and to continue testing as consumer spending fluctuates with shifting economic times.
What Do You Think?
Why do you think the version with the $199 free shipping threshold won?
And, why do you think SpiritHoods has since lowered their free shipping threshold to $129?
Share your thoughts in the comments section below.