Results: Which nav bar format works best across device types?

Which nav bar format works best across device types?

Loading ... Loading ...

Difference Between Versions:

Version A – Persistent navigational bar. Displays at all times, at all places on the page, as the user scrolls up or down

Version B – Partially persistent nav bar. Disappears as the reader scrolls down the page, but re-appears for 6 seconds, upon scroll up


Key Performance Indicator (KPI): 

Engagement -- Measured by pageviews and time spent/visit


Test Goal:

Determine the optimal nav bar format: persistent (scrolls with user), partially persistent (appears upon up scroll), or pinned to the top (shows only when landing on page)

 

Traffic Source:

Direct, search, referral, and social

 

Audience: 

Online newspaper readers on either desktop computers, tablets, or mobile devices

 

Organization: 

The McClatchy Company - leading news and information provider

Sign-up for the newsletter to get FREE A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test
This week’s Featured Test is brought to you by:

Test Run By and For

The McClatchy Company

Test Run On

Adobe Target

WINNING VERSION

B

Poll Results - The Best Guesses:

Which nav bar format works best across device types?

  • Version A
  • Version B
Loading ... Loading ...

The partially persistent nav bar won, partially. . .

Read on to learn more. . .


Test Details & Background:

McClatchy, a media group with a wide array of online and print newspapers, in over 30 growing markets, and 65 million monthly readers, ran this neat newspaper navigation (nav) study in-house, on their own news assets.

Through previous studies, the innovative media provider has found the nav bar is a key element for readers to find and click articles of interest.

Wanting to increase reader engagement, the team wondered the best format for the nav bar display.

They considered 3 different nav display formats:

  • Persistent: showing at all times, at all places on the page, as the user scrolls up or down.
  • Partially persistent: disappears as the reader scrolls down the page, but re-appears for 6 seconds, upon scroll up.
  • Pinned: shows only above the fold, and doesn’t display once users scroll below the fold.

In addition to the optimal nav bar display, they questioned if findings would hold true across devices type. They considered the three main devices used:

  • Desktop
  • Tablet
  • Mobile

 

Hypothesis:

The team suspected user engagement would be impacted by the way the navigational bar was formatted on the device used.

However, they weren’t fully sure how results would play out.

They expected the partially pinned navigation bar would be the winner for mobile and tablet devices since it would give users the most flexibility on a small screen. But, were uncertain how it would perform on desktop devices.

So, they decided to test what nav format worked best, across all device types.


Test Set-up:

To definitively determine the optimal nav display, the team set-up and ran a complex experiment, using Adobe Target.

The study ran for 39 days. During this time a total of 745,144 desktop, mobile, or tablet visitors saw one of three different navigation bar experiences, with traffic equally allocated to the three experiences by device type.

Control – Top Navigation Bar

The control experience featured a “typical” navigation bar. It displayed when users first opened, or landed on the page, but disappeared as users scrolled below the fold. It looked like this:

mcclatchy_control

Version A – Partially Persistent Navigation Bar

The partially persistent navigation bar appeared at the top of the page, when users landed on the site, but disappeared upon scroll down. Then, reappeared upon scroll up, but disappeared 6 seconds later. It looked like this:

mcclatchy-partially_persistent

Version B – Persistent Navigation Bar

The persistent nav bar was pinned to the top of the page and stayed in place as users scrolled up, or down the page. It looked like this:

persistent_nav_bar

Engagement, as measured by page views/visit, time spent on site, was tracked across all variants, and by device type.


The Real-Life Results:

Winner: Version B – the partially persistent nav bar was the winner. But, just partially. . . It only won for tablet devices.

Results by Device Type:

Tablet

For tablet users, the partially persistent nav bar format increased page views 16.8% and viewing time 26%, compared to the control. Results achieved 99% confidence.

On tablet, the fully persistent nav format also worked well — but not quite as well as the partially pinned version. The fully persistent format increased pageviews 13.8% and time spent per visit by 17.4%, compared to the control.

 

Desktop 

While the partially pinned nav bar won for tablet, it was neutral for desktop viewers.

Compared to the control — the standard nav format — there was no statically significant difference. Neither a pinned, or partially persistent nav bar moved the needle. Therefore, the typical nav bar was declared the best experience for desktop readers.

 

Mobile

Interestingly, the partially persistent nav bar that worked so well on tablet devices lost on mobile. The partially pinned format created -5.9% drop in page views, at 99% confidence.

The fully pinned navigation format was the worst option for mobile users. It led to a -8.4% fall in page views and -7.9% drop in time spent on the site, compared to the control.

 

Overall Results

The partially pinned format was declared the study winner. Partially. It was named the winner with the caveat it really only worked best for tablet users.

Here’s a complete breakdown of the results:

results_chart

Nav Bar Clicks

Since this was a navigation test, the team also measured clicks on the nav bar itself. As expected, the more the nav bar was on the screen, the more clicks it got. However, clicks didn’t impact actual site engagement.


How Trustworthy Are The Results?

Given the fact the sample size was quite large (745,144 visitors, spread across three variants, and three device types), the test ran for an adequate time period (39 days), and results achieved 99% confidence, it’s highly likely the study findings are valid.

However, it should be noted, conversions were tracked by variant and device type, which drastically reduced the amount of traffic allocated to each version; even so, the sample size should still have been large enough to obtain statistically significant results.


Analysis:

Why did the version with the partially persistent nav bar win, but only on tablet devices? There are likely a couple reasons:

1. A one-sized fits all nav bar display doesn’t work

Different navigational experiences work best on different devices because users have different needs when consuming content on each device.

Screen real estate is a major factor that impacts user behavior. The more viewing room on a screen, the less impact a small design feature, like a navigation bar, will have on audience’s attention. And, vice versa.

In this test, the partially pinned navigational format probably worked best for tablet users because it helped remind readers they could easily navigate. But, since it disappeared upon scroll down, it wasn’t so intrusive that it distracted or blocked viewers from reading on a smaller screen.

In contrast, the fully pinned navigation bar likely impeded readers’ view on a small mobile screen and was seen as both intrusive and distracting.

2. Mobile-first doesn’t always win first place

When designing and creating content for conversions, it’s important to consider how your design will appear across different screen sizes. A mobile-first optimization approach is best only if the majority of your users are on mobile devices.

If your prime audience consumes content on tablet devices, then a tablet-first approach is needed. The same principle holds true is your traffic is primarily comprised of desktop viewers.

3. Cues need to match competing elements

On any device, visual distraction is important to consider for conversions.

If there are a lot of visually competing elements on your site, none will convert well — especially on a smaller screen. It’s important to strike the balance between prompting conversions and not being so in your face users become frustrated by, what they may view, as obtrusive or obnoxious reminders.

Building flexibility into your viewing platform may be the best way to go. In this test, the navigation bar that appeared, disappeared, and reappeared was fluid. By being flexible, it catered most to users’ needs. And, therefore, converted best.


Tangible Takeaway & Immediate Application

Cater to users’ device type. Viewers don’t always want a navigation bar in their face, especially on mobile devices with smaller-sized screens.

When possible, format and display content to be most responsive users, based on their screen size.

On small screens, you can optimize your navigational structure by making it flexible. It’s okay to allow the nav bar to disappear and re-appear as users move through the page.


What Do You Think?

Why do you think the partially persistent nav format won? Share your thoughts in the comments section below.

Poll Results - The Best Guesses:

Which nav bar format works best across device types?

  • Version A
  • Version B
Loading ... Loading ...

Sign-up for the newsletter to get FREE gamified A/B tests and more valuable content delivered to you every other week.

I agree to the Terms & Conditions.

Become a More Successful Digital Marketer

Pro Members get exclusive access to the full library of Archived Tests, helpful resources, plus personally connect with industry experts to get your specific questions answered. Get the Pro Advantage

Get Your A/B Test Featured!

Have your A/B test showcased. You'll get a great case study write-up, your logo prominently featured, linkbacks to your site, and social media promo, giving you valuable, free brand exposure. Submit Your Test

1
COMMENTS: Tell Us Your Thoughts.

avatar
1 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
1 Comment authors
Deborah O Recent comment authors
  Subscribe  
newest oldest most voted
Notify of

Other Tests You Might Enjoy

Test Run By and For

The McClatchy Company

Test Run On

Adobe Target