With the pre-set buttons won.

But only by default. Read on to learn more. . .

Test Details & Background:

Yemeksepeti.com, the Turkish online food delivery company, conducted this tasteful test in-house, on their site.

Through an online wallet system, Yemeksepeti enables users to maintain a credit balance, within their account, making it faster and easier to pay online, for upcoming orders.

When the balance runs low, customers are encouraged to top-up the amount.

Yemeksepeti’s testing team has found prompting users to maintain a high account balance leads to more food orders, and food orders of greater value.

So, they deemed it highly beneficial to test the optimal top-up forma to encourage customers to top-up their credit card balance.

Originally, the Yemeksepti site featured a top-up screen with three pre-defined amounts (25, 50, 100 TL) as well as a blank field where the user could input any amount desired. It looked like this:

However, tThe testing team wanted to determine if removing the pre-defined amount buttons – replacing them with a bigger input box – would increase both the top-up rate as well as the top-up amount. So, they created a revised version which looked like this:

Hypothesis:

The team suspected removing the pre-defined amounts work well since it would give users the freedom to enter the exact amount desired.

As a result, the top-up rate and amount would increase.

However, the team also acknowledged leaving the amount field blank slate could overwhelm visitors with choice, to the point they’d make no decision at all – and wouldn’t fill in the amount box.

So, decided to test which format worked best.

Test Set-up:

To definitively determine the optimal top-up format, an A/B test was set-up and run on Optimizely.

The test ran for 40 days. During this time, over 20 thousand visitors took part. Traffic was split 50/50.

Half of visitors saw the original version (control), with the pre-selected amount buttons. The other half of visitors were directed to the version with just the blank field.

The metrics of top-up rate and top-up amount were tracked across both versions.

The Real-Life Results:

Winner: Version A, the pre-filled buttons, won. But, only by default. . . 

Contrary to the hypothesis, the new design, with the blank field did not work better.

Top-up Rate

In fact, it led to a -11.18% drop in the top-up rate! Results achieved 90% confidence.

Top-up Amount

That said, the blank field design did have some merit. It worked better at increasing the average top-up amount. But, because far fewer users took the action of inputting their top-up amount, this blank slate version, overall, far under performed.

The team declared the test unsuccessful, but have decided to do future testing on the pre-selected amount buttons.

How Trustworthy Are The Results?

Given the sample size was large (20 thousand users), and the test ran for a standard amount of time (40 days), it’s highly likely the study is valid.

However, it should be noted, the results only achieved 90% confidence, which means there’s a 10% chance results would not hold true in future studies. That’s a wide margin. Ideally, you should aim to achieve 95%+ significance in your studies.

As well, it should be noted, the top-up amount was not reported. Therefore, we have to take the testing team’s word for it when they say the losing variant overall underperformed.

Analysis:

What does this surprising study show us? There’s a couple important takeaways:

1. Amount of choice impacts conversions

As previously discussed, choice can impact conversions.

Too little choice, people feel disempowered and don’t act. Too much choice, people become overwhelmed with possibilities – and don’t convert.

On the version with the blank box, customers were faced with a lot of choices.

First, they had to decide whether or not they wanted to top-up their wallet balance.

And, if so, they then had to figure how much to add.

This choice was likely difficult for many hungry visitors who simply wanted to place their order and fill their bellies – not their online wallet.

Because there was no suggested figure to guide them, many users chose the easiest option: to not top-up at all.

More choice did not equal more conversions.

In contrast, the version that limited choice to a selection of clickable buttons helped guide hungry visitors. Interestingly, however, it also limited them; these users were less likely to top-up a high amount.

This test shows, when making a choice on whether or not to add money, most users need guidance. But, to truly optimize conversions the pre-determined amounts need to be carefully worked out so visitors give most.

2. Results are context specific and may differ – so test!

It’s interesting to note, not all the studies have arrived at the same finding.

In fact, two not-for-profit donation studies show markedly different results. In this test by the World Wildlife Fund, the pre-filled button amount was definitely the way to go. Just like this Yemeksepeti study.

However, in this donation test by Caring Bridge, the blank button format worked far more effectively.

Why the discrepancy across case studies?

There’s likely a couple factors at play:

1. The context of the contribution – In this Yemeksepeti study, anyone adding funds was putting money into their own account. They weren’t giving or donating to others. This simple fact may have dramatically changed user behavior. Therefore, it’s hard to draw trends across case studies that are in different industry verticals and have far different contexts.
2. Type of visitor – Additionally, anyone making a monetary contribution may either be doing so for the first-time or a repeat customer. The type of visitor may influence contribution behavior. A first-time user may not feel as comfortable forking over money as a person who’s established a trusting relationship with the brand.

These factors will influence how users behave, thereby affecting overall conversions.

Taken together, these studies show there’s not a right way to optimally format the contribution field on your site. The only right thing to do is test for yourself! 😉

Tangible Takeaway & Immediate Application

The amount of choice given, the context in which you’re asking for a monetary contribution, and the type of visitor (new versus returning) are some aspects that may determine optimal funding format.

Consider these factors when designing and testing your site’s optimal format.

What Do You Think?

Why do you think the version with the fillable field underperformed? What would suggest testing next?

Share your thoughts in the comments section below.