Children's Hospital Improves Email Program with A/B Testing

by Hannah Stuart, Copywriter - 06/07/17

Did you know a simple experiment can optimize your email marketing?


If you want to know which email subject line, article headline, arrival time, or even button color is most likely to garner results, let your readers decide. A/B testing is an easy, painless way to evaluate several approaches to sending out mass emails and determining which options provide you with the best results.

Increase Your Open Rate

The average email subscriber receives about 15 commercial emails per day—that’s more than 400 emails each month—from credit unions and banks, department stores, healthcare providers, and any other company with access to the subscriber’s email address. With that many emails competing for attention in your reader’s inbox, it’s critical that your message catches his or her attention and provides value. Here's where A/B testing can help.

A/B testing is a method used to determine which of two separate versions of your email is more successful. By dividing your email list in half and testing version A on half the list and version B on the other half, you can learn how differences in the email content can get you a higher open rate.

Getting Started with A/B Testing

First, Set Your Goal.

What aspects of your email do you want to test? Based on which variations perform best, you can continue to refine your email strategy based on what resonates with your unique audience.

K.I.S.S. a.k.a. "Keep it simple, somehow."

This mantra reminds you that for the most accurate results, you should only test one aspect of your campaign at a time.

Don't Drag it Out.

Set a timeline for your test emails that doesn’t drag on forever, or you might risk your campaign becoming less relevant, particularly if your topic is time sensitive.

Choose a Good Sample Size

If your emails are going out to only 10 people, five people getting one headline and five people getting the other isn’t going to give you a good picture of your audience’s preferences. Reserve A/B testing for email messages with the largest audience so the size can be kept statistically significant.

Measuring the Results: A Case Study

Once you look at open rates, click-to-open rates, and click-thru rates and learn whether your A option or B option garners better results, you can use the knowledge you’ve learned to send more effective emails in the future.

Email Subject Line Performance Comparison

In an A/B study of email click-thru rates for a children’s hospital, True North Custom compared the following two subject lines:

• Subject Line A—“February’s Guide to Good Health from XXX Children’s Hospital.”

• Subject Line B—“February’s eNews: Traveling with Kids, Dental Care Tips and More.”

For the purpose of this study, half the email subscriber list received the e-newsletter with subject line A, a traditional, generic email subject line.

The second half of the subscriber list received the e-newsletter with subject line B, which teased article content in the subject line.

The results of the A/B testing were impressive. Subject line B had a significant increase in click-thru rates. In the case of the children’s hospital, the more detailed headline provided significantly higher click-thru rate for the “Keep Your Child Safe When Flying” and “Dental Health Starts Early”—both articles that were teased in the subject line. 

Refining Via Results

What we learned from this test is that readers in the children’s hospital audience are more likely to read articles teased with a subject line that is descriptive of the articles contained inside, rather than a generic “Here’s our monthly e-newsletter” subject line.

It’s important to note that this only tells us what this particular audience prefers; a different hospital’s e-newsletter readers might have a different result. However, going forward, we know we can use descriptive subject lines for the children’s hospital’s e-newsletter with the knowledge that we are reaching a better percentage of our audience than before.

The Bottom Line

Don’t let your A/B testing be a one-and-done endeavor. Keep refining your content! After you test subject lines, move on to headlines or article titles within your email. You can also test out your send time to determine whether the time of day has an affect on if your email is opened or not. The possibilities are endless.

Want to learn more about True North Custom's e-newsletters? 

Contact Us 

Topics: Digital Marketing, Healthcare Case Studies

Get Blog Updates by Email

Healthcare Insight Magazine

The latest issue of our guidebook for healthcare marketers offers fresh ideas and expert perspective to inform your consumer and physician engagement strategies. 
Sign Up and Start Reading Now!

Insight-Magazine-Winter-2020 copy


Popular Articles

Follow us on Twitter