Return Path’s Sample Size Calculator: Your Key to Testing

Have you ever put together the perfect test campaign, only to run into the issue of how many people should receive it? You know you can’t send it to your whole list, but is an A/B split the right option? Look no further than Return Path’s Sample Size Calculator.

Return Path’s Sample Size Calculator takes all the guess work out of determining the percentage of subscribers who should be receiving that message. Return Path can help you find out how large your sample population should be to accurately reflect the true population of your list. All you need to do is simply input your confidence level, list size, and acceptable margin of error and SHAZAM, sample size calculated.

This allows you to avoid using equations that look like this:


When you select a confidence level, you are selecting the probability that the statistic you are seeing from the sample population falls within a certain percent of the true value. So the higher the confidence level the more accurate your statistic will be. This; however, will require a larger sample size because the closer your sample statistic is to the actual value, the closer your sample size will have to be to the actual population size (list size).

The second piece of information is easy — just enter the list size!

The last element—acceptable margin of error—allows you to choose the acceptable range of values for how close the figure needs to be to the true value. If you choose a margin of error of percent you are accepting values that are within five percent of the true value.

The output you receive from inputting the above values is the sample size. Didn’t see that coming, right? The sample size provided will produce values that are within the acceptable margin of error to the true value with a probability equivalent to the confidence level the user chose!

Below is an example of how the Sample Size Calculator can be used:
Bob wants to see how his new creative will perform with the 2000 subscribers on his list. He doesn’t want to mail to too many people until he understands how his subscribers will react. Using the Return Path Sample Size Calculator, Bob inputs a confidence level of 95 percent, list size of 2000 and an acceptable margin of error of five percent into the Sample Size Calculator. See below.


The Sample Size Calculator calculated that a sample size of 322 would be sufficient to produce values that are within five percent of the actual value the 2000 subscribers produce with a probability of 95 percent. Would you have guessed that on your own? Or would you have over sent?

Testing is very important in the world of email. You don’t want to be too stagnant and your subscribers become bored and unengaged. You also don’t want to go to the extreme and constantly change your emails and lose people who are confused by your ever changing branding. This confusion can lead to complaints which are even more detrimental to your email program. The best way to go about testing is finding the right audience to test. It’s easy to do a 50/50 test or even a 10/10/80, but if you really want to take the guesswork out of testing then the Sample Size Calculator is the way to go!

minute read

Popular stories



BriteVerify email verification ensures that an email address actually exists in real-time


The #1 global data quality tool used by thousands of Salesforce admins


Insights and deliverability guidance from the only all-in-one email marketing solution

GridBuddy Cloud

Transform how you interact with your data through the versatility of grids.

Return Path

World-class deliverability applications to optimize email marketing programs

Trust Assessments

A revolutionary new solution for assessing Salesforce data quality


Validity for Email

Increase inbox placement and maximize subscriber reach with clean and actionable data

Validity for Data Management

Simplify data management with solutions that improve data quality and increase CRM adoption

Validity for Sales Productivity

Give your sales team back hours per day with tools designed to increase productivity and mitigate pipeline risks in real-time