John Young, Ph.D.
Director of Product Analytics
We encourage receiving networks to share data with us at Return Path so that we can in turn provide solutions and information that will help their filtering decisions. We believe that you can learn from another company’s mistakes and success. And, when working in a collaborative environment, receiving networks can learn from cases where one system accepted mail that another system was blocking erroneously or vice versa.
We decided to dig into our data to find out if Internet Service Providers (ISPs) treat IPs differently. We took a random sample of 400,000 IPs that attempted to send messages to four different receiving networks in early 2009. The ISPs used from our network consisted of two webmail providers, one cable operation, and a hosted business email provider.
By looking at IPs that mailed to all four networks, it became clear to us that receiving networks make extremely different decisions about how to treat those mailers.
From the data, we found that when ISPs make decisions on what to do with inbound mail, they cannot agree on IPs from smaller volume mailers, especially when that IP has no rDNS. Also noteworthy was a 7X difference in accepted mail from the same set of IPs when comparing their behavior at two ISPs. Now, that’s not as interesting until you note the variance in complaint rates across ISPs, which vary as much as 3X between them.
One would assume that the more email sent, the higher the probability of a user complaining. However, for this set of IPs and their corresponding data, that is not always the case; especially when looking at those smaller volume IPs with no rDNS. For example, one IP sent ~100 messages to ISPs A and B. The IP had ~90% delivered rate at ISP A, 4% Delivered rates at ISP B, but less than 0.2% complaint rates at either.
Another interesting pattern indicated agreement in the treatment of IPs with a lot of trap hits. However, there was a significant percent of IPs with a large amount of traps who ended up in the disagree buckets respectively. Again, it was mostly smaller volume mailers averaging less than ~200 messages a week.
It is evident that ISPs could benefit from sharing their experiences and their data in their fight against spam. By doing so, they could minimize the amount of spam impacting their systems, and the overall costs associated with filtering.
We are currently in the process of conducting a larger study to see if the data we found in our initial analysis holds true. If you are a receiving network who would like to participate, please contact us. In exchange for participating, you will receive detailed reporting on how your filtering differs from other ISPS as well as reputation data that would greatly improve your filtering and blocking decisions.