Recently Return Path released a new ebook entitled All About A/B Testing: How to test, What to Test, and How to Analyze Your Results. The ebook provides readers with a thorough overview of how to get the most out of your A/B tests. Some of the topics that are covered include setting up and running your A/B test, testing ideas, and testing best practices. In most cases, you can create your control group and monitor results with your company’s internal deliverability tools. However, there are additional tools you can use to gain a different perspective.
Enter Inbox Insight.
Before we discuss how you can use Inbox Insight to your advantage with A/B testing, first let’s discuss the product.
Inbox Insight is Return Path’s competitive intelligence tool that allows you to increase subscriber engagement through competitive insights while also having visibility into engagement data from real email users from Return Path’s Consumer Network. The Consumer Network consists of over 2 million email subscribers who voluntarily agree to exchange their email data in return for services, like, for example, using our Return Path developed app Shopami. With the data we collect from the Consumer Network and the sending domain of any company, you are now able to see engagement data in the product.
What to test
Inbox Insight can help you determine what you would like to test by having the visibility into your competitors emails all in one location. For example, if your company was interested in sending out a new labor day campaign, you can use Inbox Insight to review Labor Day campaigns from the previous year to see how those campaigns performed. This information could be used to help define the new “Labor Day” campaign your company is interested in deploying.
Additionally, if you were not sure exactly what you wanted to test, you could use Inbox Insight to review industry and competitors subject lines over the last 18 months. You can also use Return Path’s Subject Line Optimizer to review subject lines as well.
Control group and test group
Now that you have determined what the subject or previous email campaign will be for your control group, you have to determine what exactly are you going to test. As a company, you could add personalization to your message or you could make a very specific change. For example, you might exchange commas in your subject line for line separators (|). In Inbox Insight you could create a tag for your control group (Tagging is a functionality in Inbox Insight that allows to you to group and mark a subset of emails to see how those specific campaigns are performing). Secondly, as you deploy your test campaigns, you can create an additional tag labeled “test.” For additional ideas on testing, take a look at this blog that lists 50 ideas to help get you started.
Analyzing your results
Okay, so you have determined what you are going to test, created a control and test group, and now you want to see how your campaigns performed. Inbox Insight can help with that as well. Using the “trends” page in Inbox Insight you can track the “tags” for both the test and control groups. Using the data in the trends page you can see how each group performed with metrics like “Read %” and “Forwarded %” while also determining which day of the week, day of the month, and month each campaign performed its best.
With A/B testing, the key is to continually test new subject lines and tactics. However, with Inbox Insight, you can continue to track and compare previous tests with new tests over an 18 month time period while also comparing your tests with competitor and industry metrics.
Moral of the story, continue to test and let Inbox Insight help you evaluate subject lines and modifications to your subject line format. Then view the metrics over time to see how the changes you have made are improving your subscriber engagement.