Whether you’re a seasoned veteran or relatively new to email marketing, you’ve got to think outside the inbox when it comes to A/B testing. If you’re not already running A/B tests, you’re missing out on a huge opportunity to optimize your email campaigns. If you are conducting A/B tests, don’t get complacent. Test various elements to ensure that your content continues to resonate with your subscribers.
However, resist the temptation to test multiple elements at once. To avoid skewing the results, test one element at a time. For instance, test a subject line in one email campaign, a call to action in another.
Also keep in mind that you should test a large-enough email sample to make the test statistically significant. What exactly constitutes a statistically valid sample size? Most email marketers will suggest at least 1,000 email addresses. That doesn’t mean you need to run every test on your entire email list. If your list is smaller than 1,000, test 85 percent to 95 percent of the list to ensure valid results.
For larger lists, you may want to test using the 20/80 rule. Send Version A of your email to 10 percent of your list, and Version B to another 10 percent. A couple hours after the email is deployed, the email with the most opens is determined the winner, and that version is sent to the remaining 80 percent of your list. Most email service providers (ESPs) have built-in functionality to automate these types of tests.
Now that we’ve got the technical aspects out of the way, let’s take a look at some of the more creative aspects of A/B testing. Following are three concepts to consider the next time you launch an A/B email test.
1. Go “old school”
Pit a text-only email against an HTML email and see which one converts better. Take a look at your inbox. You’ve undoubtedly got dozens and dozens of HTML-designed emails, filled with eye-catching graphics. When is the last time you remember seeing a text-only promotional email? The absence of distracting graphics may actually cause a text-only email to perform better than its image-filled counterpart. A case in point:
Note that a text-only email, such as the one above, doesn’t mean you should forsake all branding. Including your organization’s logo at the top of the email lends credibility and recognition to your communications.
Special characters in subject lines are so “yesterday,” which is why you just might want to test them again. This site is a good place to start if you need some ideas on what icons are available. The charts also include both Unicode and UTF-8 codes, one of which your developers will need for production. Here’s an excerpt from one of the charts:
If you’re targeting a B2B audience, though, you should know that Outlook and Gmail will display the special characters slightly differently. As with any email, you should do test sends with various email clients and browsers to make sure the email renders properly before sending.
And while personalization is nothing new, when done right it’s worth testing. Remember, personalization is not limited to the subject line. The following email also inserts the first name into the body of the email:
Be sure to use best practices when adding personalization to an email. Always include a generic subject line version in case the email record in your database does not include a first name. For example, the generic version of the Ticketmaster subject line could be “Your Ultimate Live Event Guide.” If you are personalizing the body copy, the name should be the same font/size/color as the rest of the body copy. Otherwise, it will look out of place and defeat the whole purpose of personalization.
Another throwback approach would be to use the tried-and-true direct mail technique of adding a “P.S.” line. According to Professor Siegfried Vogele in Handbook of Direct Mail, over 90 percent of people read the P.S. first in a direct-mail letter. Granted, a recipient may have to scroll down to see the P.S. line in an email, but a strong P.S. can reinforce the call to action.
2. Take advantage of technology
If you don’t think you have the bandwidth to conduct A/B tests on a regular basis, think again. The technology is out there to make A/B tests as simple as A-B-C. Take advantage of these tools of the trade (both free and inexpensive), and take the pain out of email testing.
The email isn’t the only thing you should be testing. If you want to test conversions, then you need to test the landing pages, too. Unbounce, as it states, lets you build, publish and test landing pages without the need for IT. Rates are as low as $49/month. The site also offers landing page templates.
Spaces’ Image Resizer makes the task of resizing images a no-brainer. It’s not only free, it’s handy for both A/B testing and for sharing your email content — especially when an offer is involved — on social media.
Email on Acid, which conducts email rendering tests (as recommended above) for its clients, has provided a list of sites that offer free email templates. The company offers its own free templates as well, including responsive templates such as these:
Any email you develop, whether from scratch or using a template, should be responsive. Mobile-friendly emails are a must, as more than half of all emails are opened on mobile devices.
You can test your email and landing page designs outside your own lists with UsabilityHub’s Five Second Test. Pricing is on a subscription or test basis. How it works: Your design is shown to testers for 5 seconds. Then the testers are asked questions to see what they remembered. It’s a way to get a quick read on different treatments of an email, landing page, etc.
3. Test. Analyze. Repeat.
Over time, your list demographics will change and it pays to re-test elements that you previously tested. Or your product/service offering may vary, necessitating new tests. So what else can you test in your emails? The options are seemingly endless:
- Preheader text
- Email segmentation
- Number of links
- Colors
- Navigation elements
- Video
- “From” name
- Testimonials
- Incentives
- Day/time you send email
Regardless of what you ultimately decide to test in your emails, it’s important that you test. It’s equally important to analyze the results, and then apply your findings to future email campaigns.
By Darcy Grabenstein, senior copywriter at Annodyne
