Difference Between Versions:
Version A – Fillable field on donation form
Version B – Preset buttons on donation form
Key Performance Indicator (KPI):
Average order value
Increase the number of people completing the donation form
Sign-up to get FREE gamified A/B tests and more valuable content delivered to you each week.
Become a More Successful Digital MarketerPro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage
This week’s Featured Test is brought to you by:
Poll Results - The Best Guesses:
Test Details & Background:
CaringBridge is a not-for-profit social networking website service. It enables anyone facing medical issues to create a website, share health status updates with their network, post announcements and photos, set-up personal fundraisers, and manage tasks.
Since CaringBridge is funded by donor support, increasing online donations is key. For help, they’ve turned to NextAfter, the not-for-profit fundraising organization focussed on increasing donations and optimizing conversions.
This test was part of an on-going series, looking at ways to optimize CaringBridge’s main donation page.
Previous testing showed some interesting outcomes:
- Donor motivation on the main page was much different than those donors who gave money through the site’s “tribute” pages. Tributes are donations made in honor of a loved one. Most of the site’s revenue comes in through Tribute pages.
- Site users who donated on the main page were motivated to support the website and its mission, rather than giving a tribute to a specific person, in need of support.
- The main donation page had a high average donation amount.
As well, in a previous study, the testing team looked at whether three or six donation buttons performed better. The team found six donation buttons worked best. Too few buttons hindered gift giving and donation conversions.
With these findings in mind, the testing team wondered if the six-array preset button format was truly optimal. They questioned whether the buttons were creating “decision friction,” making it difficult to choose a donation amount.
The team wanted to evaluate if an open field — allowing the donor to decide on their own gift amount — would work best.
The testing team speculated providing an open field, where the donor could choose an amount would impact conversions. But, were uncertain whether the open field would outperform the button format.
So, decided to test.
To determine the optimal donation form format, an A/B test was set-up and run on Optimizely. The experiment ran for 24 days.
During this time, 785 visitors were directed to the version either with the open form field, or the preset donation buttons. Traffic was split 50/50.
Half of visitors saw the donation page with the open field. It showed a $100 amount, by default. But, this amount could be deleted and replaced with a different desired donation amount. The minimum donation contribution was $10. The donation page looked like this:
The other half of visitors saw the exact same donation page, but with an added six button array on top of the open donation amount form field. The donation page looked this:
Donation form completions were tracked across both variants.
The Real-Life Results:
Winner: Version A – with the open field was the definite winner.
Removing the preset donation buttons, and enabling donors to type in their chosen donation amount, increased conversions an incredible 125.9%. Results achieved 99% confidence.
See this results screenshot for more details:
How Trustworthy Are The Results?
Given the fact the test ran for an adequate time period (3+ weeks), and achieved 99% confidence, results appear trustworthy.
The sample size of 785 visitors is lower than ideal. However, according to this calculator, the sample size was just large enough to achieve valid results, assuming a baseline conversion rate of 3.3% and a minimum detectable effect of 7%.
Surprised by these results? If you thought the version with the preset buttons would’ve won, you’re not alone.
In fact, a similar study conducted by World Wildlife Fund (WWF), found providing preset amounts was indeed the way to go. In the WWF test, offering preset donation amounts reduced choice, which helped decrease hesitancy, lifting conversions.
So, why would the results be opposite in this study? Why was it better to give donors seemingly more choice, through an open form field?
There’s likely a couple factors at play:
1. Type of visitor donating
According to Brady Josephson, Vice President of Innovation & Optimization at NextAfter, “the open field format tends to work best with high-value donors, or repeat donors, but not as much with first time donors, or with those who haven’t donated in a while.”
Comparing this donation form test to the WWF test, the type of visitor donating may explain the difference between which format worked best.
It’s, therefore, crucial you know your audience — and their donation tendencies — in order to optimize the donation format on your site.
2. Donation frequency
Another factor influencing how donors will respond is the frequency of the requested donation. Is it a one-time, or recurring payment? This factor can make a big difference.
Donors making a one-time payment may be more apt to select their own larger amount, if they’re only donating once.
In contrast, if being asked to donate monthly, it may be optimal to present a smaller predetermined amount.
When testing the optimal donation format for your site, consider the donation frequency.
3. Amount of choice given
In this test, the preset button amounts were meant to reduce “decision friction.”
In actuality, they likely increased hesitancy because there were so many choices. It became difficult to decide whether to donate $25, or $500, or something in between this wide range.
Providing an open form field, with the default $100 amount, made the decision easier. Donors simply had to choose whether to give $100, or not. And if not, what amount they’d prefer. The decision-making process was simpler. As a result, decision friction decreased. And, conversions increased.
Had the box been entirely blank, results may have differed.
Tangible Takeaway & Immediate Application
The optimal donation form format is not one-size-fits all. The best format is one that reduces decision friction, catering to the type of donor, and the frequency of the requested donation contribution.
It’s important to fully understand your audience and their tendencies to create the optimal donation form format.
What Do You Think?
Why do you think the open field format won? And, what areas would you suggest for future testing?