Results: Does segmenting by gender impact conversions?

Gender segmented email offer

Click here to enlarge

Standard email offer - not segmented by gender

Click here to enlarge

Does segmenting by gender impact conversions?

Loading ... Loading ...

Difference Between Versions:

Version A – Email offer segmented by whether the reader was male or female. Offer on the left sent only females; offer on the right, only to males

Version B – Standard email offer; same offer sent to both males and females


Key Performance Indicator (KPI): 

Lottery ticket orders


Test Goal:

Determine if gender segmentation impacted lottery ticket orders

 

Traffic Source:

Existing email list

 

Audience: 

Dutch lottery players, both male and female

 

Organization: 

Staatsloterij - the Dutch state lottery system

Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test
This week’s Featured Test is brought to you by:

Test Run By

Greenhouse Group

Test Run For

Staatsloterij

Test Run On

Oracle

WINNING VERSION

A

Poll Results - The Best Guesses:

Does segmenting by gender impact conversions?

  • Version A
  • Version B
Loading ... Loading ...

The segmented offer was the winner.

Read on to learn more.


Test Details & Background:

Greenhouse Group, a Dutch digital media agency, conducted this gender-specific case study for their client Staatsloterij, the Dutch national lottery company.

The study goal was to determine whether gender effected conversions to a lottery ticket bonus prize, sent by email.

To test, the audience was segmented by gender. Both readers who identified as either men or women received an email offer for a lottery prize. However in the variation, men saw a different featured bonus prize than females.

For those who identified as females, the advertised bonus was a chance to win tickets to a prestigious, sold out concert featuring the famous Dutch artist, Marco Borsato.

Initially, the team planned on offering this bonus to all players. But, after doing some research, discovered about 70% of all people who “liked” Marco Borsato on Facebook were women.

With this knowledge, the team suspected the bonus concert prize offer might not motivate the male audience segment to convert. So, the team decided to create a test variant for men, switching the concert ticket offer to a Jackpot lottery prize.


 

Hypothesis:

The team suspected segmenting the bonus offer by gender would lead to a higher overall conversion rate, increasing lottery ticket sales.

However, they also acknowledged that showing the concert ticket bonus offer only to women and the bonus jackpot prize only to men could backfire since gender might not be the only factor affecting preference.

So, the team decided to do some gender-bending testing to definitively determine what worked best.


Test Set-up:

To find out whether the gender-segmented offers lifted conversions, an A/B test was created using Siebel, a CRM application from Oracle. Siebel was used to set-up the gender targeting and track the email open rate. As well, Google Analytics was used to track conversion data, including shopping basket adds, orders, and product conversions.

In total, 132,000 people received an email offer over an eight-day period.

Offer sent to both men and women

Half the recipients received the control email offer, which featured concert tickets, as a bonus prize. This version was sent to both males and females. It appeared like this:

staatsloterij-version-b-lg

Gender segmented offer

The other half of recipients saw either the concert ticket, or bonus prize offer, depending on their gender. The campaign shown on the left was sent to those who identified as females; the version on the right was sent to those who identified as males. The gender-segmented offers looked like this:

staatsloterij-version-a-lg

Although the total sample population contained proportionally more men (62%) than women (38%), the test was constructed so both genders were evenly split across both the control and variant groups.

Conversions were tracked across both email campaigns.


The Real-Life Results:

Winner: Version A – the gender segmented version was the definite winner.

Segmenting the offers by gender lifted lottery ticket orders 46%. Results achieved 99% confidence.


How Trustworthy Are The Results?

Given the fact the sample size was large (132 thousand viewers), the test ran for an adequate time period (8 days — which is long for an email campaign), and results achieved 99% confidence, it’s highly likely the study findings are valid.

However, it’s difficult to determine whether the gender segregation was properly set-up, given the were more men than women in the test group. We can only assume splitting the sample population was indeed done proportionally and accurately.

As well, it’s important to note, gender can be a tenuous topic. Not all people identify as either male or female. When segmenting by sex, this test likely didn’t take into account those who identified neither as male or female.

Additionally, this test only showed that those who identified as male preferred the bonus prize offer, compared to the concert ticket offer. What it didn’t test is whether females would have also preferred the bonus prize offer, compared to the concert tickets. All females were only shown the concert ticket offer — whether they were in the control or gender segmented group.


Analysis:

Why did the version with the gender-segmented version win? There are likely a few reasons:

1. Not all visitors are the same

Your visitors all think, act, and behave differently. So, you shouldn’t treat them all the same way. A tailored tactic will likely be much more effective than a one-size-fits-all approach. Appropriately segmenting your audience may be key for conversions.

Gender is one obvious way to segment. However, to aptly segment by gender, you must intimately know your audience’s’ preferences, likes, and dislikes.

Deeply understanding your audience can give you insight to use in your A/B tests. For example, you can determine the preferences and behaviors of your audience by looking at a Facebook group, or doing consumer polls and surveys.

2. Relevant offers convert better

But, gender isn’t the only way to segment visitors.

You might also want to look at new versus returning visitors, or segment visitors by device type, or location.

For each segmented category, your goal should be to present highly relevant information, specific to that group. As this article in Forbes magazine explains, creating contextual relevance is a key way to appeal to customers. Offers seen as relevant are more likely to convert.


Tangible Takeaway & Immediate Application

Producing relevant content, and imagery, that resonates with your audience, and their preferences, will probably convert a whole lot better than generic material that doesn’t strike a chord.

To come up with powerful segmentation strategies, do some preliminary research about your audience. Facebook is one resource to tap, but you can gain insight into your audience many ways.


What Do You Think?

Why do you think the version with the gender-bending version won? Share your thoughts in the comments section below.

Poll Results - The Best Guesses:

Does segmenting by gender impact conversions?

  • Version A
  • Version B
Loading ... Loading ...

Sign-up for the newsletter to get FREE gamified A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test

7
COMMENTS: Tell Us Your Thoughts.

avatar
4 Comment threads
3 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
4 Comment authors
Deborah OFred van EngelenPatrickSteven G Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Steven G
Guest
Steven G

Without confirming a drop below the control for females we could just be looking at a better offer. The conclusion is tenuous at best and has indicators of confirmation bias. As working women often deal with wage disparity, it is possible they would have responded in a similar fashion or at a higher rate. (Something that should also be tested against a financial demographic, not just gender)

Patrick
Member
Patrick

So none of the woman got to see the right offer? Then it’s not really a gender based test.

Fred van Engelen
Guest

I hope the email images are a mockup instead of the real emails since both men and women are addressed with “Beste heer”, which translates to “Dear Mr.”. No doubt that would affect the conversion rate for women.

Other Tests You Might Enjoy

Test Run By

Greenhouse Group

Test Run For

Staatsloterij

Test Run On

Oracle