Difference Between Versions:
Version A – Coupon offer, and fewer form fields
Version B – No coupon offer, and more form fields
Key Performance Indicator (KPI):
Average order value
Increase the number of leads filling out a form
Local small and medium-sized businesses (SMBs), in targeted U.S. cities
Thomas Printworks: 60-year old printing company doing large-color graphics like vehicle wraps, billboards and skyscraper banners
Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.
Become a More Successful Digital MarketerPro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage
This week’s Featured Test is brought to you by:
Poll Results - The Best Guesses:
Surprisingly, the longer form WITHOUT the coupon offer won!
Can’t believe it? Read on to learn more.
Test Details & Background:
Founded in 1965, Thomas Printworks has evolved from a small print shop in Dallas, Texas, to an industry-leading printing company with over 600 employees, across 28 U.S. locations.
Originally known as Thomas Reprographics, the company has rebranded, modernized, and optimized their website along the way.
Looking for expert direction, the printing company has turned to the Texas CRO agency, Reap Marketing, for help.
Reap began by creating a lead generation campaign for Thomas’ vehicle graphics – one of the print company’s highest profit margin products.
The campaign offered a lead magnet with a 25% off coupon. The discount was given once users submitted their contact info, through a short web form.
The lead generation technique worked well, generating many new contacts and quotes.
But, later analysis showed the print company was giving away a lot of money, in the form of discounts. So, Reap took a step back and reconsidered the approach.
They wondered if they could keep the same form conversion rate, but lower the discount amount offered.
They decided to test the effect of using a 10% off coupon, versus of a 25% off coupon.
Interestingly, they found conversions between the 10% and 25% off coupon were virtually the same!
That’s when the team knew they were on to something. . .
Through careful analysis, Reap’s testing team realized visitors were filling out the form because they were most interested in one thing: receiving a price quote.
Prospects wanted to know whether they could afford the high-quality printing service for vehicle graphics.
The team suspected, if they altered the form, and targeted it to provide leads with what they were after most – a price quote – they could generate similar form completion conversions, without needing to offer a discount.
Tossing aside form field best practices, the team decided to test the effect of making the form longer — instead of shorter — and removing the coupon offer altogether.
Despite a longer form, every form field was pin-point focussed on asking leads the information necessary to provide a price quote.
To definitively determine which form format worked best, the two coupon offers (25%) and (10%) were pitted against the version with no discount at all.
A split-test was set-up and run on Ion Interactive. The study ran for six-weeks.
During this time, traffic was split 30/30/40. Nearly a third of users saw the (control) version with the 25% coupon offer. It looked like this:
The other third of visitors saw the offer with the 10% off coupon code. Aside from the reduced discount amount, everything on the page was exactly the same. The page looked like this:
As well, 40% of visitors were directed to (version B) the page with the longer, but more targeted form that offered no discount at all. The page looked like this:
Form completions were tracked across all three versions.
The Real-Life Results:
Winner: Version B – surprisingly, the longer form – with no discount – wildly won.
Compared to the original 25% coupon offer (which converted at 2.2%), the winning variant, without the coupon offer, led to an incredible 140% (5.28% conversion rate) increase in form completions.
There was little difference between the 10% and 25% off coupon.
Results achieved 90% confidence.
Anecdotally, the printing sales team also remarked the leads coming from the winning form were better quality and more responsive a follow-up phone call.
On top of that, removing the discount offer saved the print company over a quarter million dollars a year.
The winning experiment literally became a license to print money! 😉
How Trustworthy Are The Results?
Given the study ran for an adequate period (six weeks), and achieved 90% confidence, results are likely quite valid. Although a sample size was not reported in the study, the fact that results achieved such strong confidence indicates there was enough traffic to confidently assert reliable results.
However, it should be noted, sample size can greatly sway results. So, it’s important to ensure your studies have adequate traffic to confidently assert validity. If you’re uncertain how much traffic you need to confidently state a conversion lift, you can use a sample size calculator, like this one.
Why did the version with the longer form format, and without the coupon offer win? There’s likely a couple explanations:
1. Best practices don’t ALWAYS hold true
The commonly accepted best practice that you should “keep forms as short as possible” doesn’t always hold up in every situation.
Sometimes, a longer form can convey a richer, higher quality response.
This fact especially holds true when the form fields are most relevant and targeted to what users need most from the site.
2. Discounts don’t always win
As this study showed, sometimes there isn’t much difference between the richness offered in a discount. Just the presence of a discount may be perceived as valuable.
That said, visitors may respond even more favorably to non-incentivized offers, like talking to a subject matter expert, or getting a price quote – if that’s what they truly want.
In this study, targeting the form to tune into prospects’ needs worked wonders. It saved the company money, and increased lead conversions.
With the targeted form, prospects were able to answer their ultimate question: can I afford this service from this company? And, in doing so cut out the ambiguity of a discount that would be applied to an unknown price.
Tangible Takeaway & Immediate Application
Discounts don’t always win. In fact, they can create a loss for you and your customers, if they’re not what the customer truly needs.
Also, form field best practices are just that – best practices. While they often work, they can’t be blindly applied to every situation.
To truly know what works best with your audience and website, you need to test.
What Do You Think?
Why do you think the longer form, without the coupon code won? Share your thoughts and reactions in the comment sections below.