MAAWG's Latest Documents Improve Accuracy of Reputation Systems

The Messaging Anti-Abuse Working Group (MAAWG), of which Return Path is a very active participant, met recently in Heidelberg, Germany. Among other exciting projects, they finished two new best practices documents which have been lauded in the press as a big step towards stopping botnet spam.

(“Botnets” are networks of computers infected by viruses or other malicious software, invariably without the owner’s permission or knowledge, which are used to engage in criminal activities like sending spam or attacking web servers.)

Neither document, however, is actually about botnets – that’ll come from the next meeting, which has a botnet theme. Instead, both describe simple ways to improve classification of mail sources, so that reputation scoring may be applied more accurately and effectively. I’ll explain this further towards the end.

Email Forwarding Best Practices, edited by two of our friends at Comcast , describes a problem which only affects a small percentage of users – but for those who are affected, it’s a big problem. Forwarding, in this context, is when a message is sent to an address which is configured to resend all mail to another address – for example, [email protected] may have his mail forwarded to [email protected]

The problem, as always, is spam. When more than 90% of all email is spam, then more than 90% of mail sent to [email protected] is spam – so more than 90% of what Stanford forwards to Comcast is spam. Comcast’s spam detection systems will notice that 90% of what they get from alumni.stanford.edu is spam – in other words, a very bad reputation – and will block all mail from that system.

Forwarding has been around pretty much since the beginning of internet email, though not all sites offer it today. The way it works in most places is almost appallingly simple: a message is received, and is immediately sent back out. There’s very little processing involved.

As Return Path is constantly advising clients, any legitimate sender needs to avoid looking like a spammer. So do forwarders.

MAAWG, in this document, recommends that forwarders engage in more processing before resending a message. They suggest to forwarders that they catch as much spam as possible, rather than blindly forwarding all of it, and ensure that both the systems they use for forwarding and the forwarded messages themselves are clearly labeled. They further suggest that anti-spam systems should look for these labels, and treat forwarded mail differently from other sources.

The second document is dryly titled Methods for Sharing Dynamic IP Address Space Information with Others. “Dynamic IP Address Space” refers to IP addresses which are dynamically assigned, such as to dial-up, cable, or most DSL connections. These consumer-grade services are how most people access the internet from home, and home computers are statistically extremely likely to be infected – thus, most botnets consist of computers on dynamic addresses.

MAAWG previously published a recommendation that ISPs should take steps to restrict or otherwise control port 25 connections from dynamic addresses, in order to reduce bots’ ability to send email. In those cases where this is not possible – and to assist with non-email-related attacks from botnets – MAAWG recommends clearly labeling such dynamic addresses, and keeping them separate from static (non-changing) addresses. The document goes on to list some common labeling methods and styles.

Obviously home users should be able to send email, but their legitimate messages are sent through their ISP’s mail servers – often using the submission port, 587 – rather than directly to the recipient’s server on port 25.

So, what do the recommendations in these documents have to do with reputation?

In both cases, the clear labeling and transparency make it easier for anti-spam systems to determine which thresholds are appropriate for that type of mail. Dynamic addresses, with few exceptions, shouldn’t be sending mail directly at all – thusany volume is suspicious, even before there are complaints or other data to mix in. Forwarding servers, because they’re likely to be forwarding some spam no matter how hard they try to catch it, can’t be judged solely on IP reputation – but should still have content filters applied.

Reputation systems adjust for other categories, too: the normal behavior of one of Comcast’s mail servers will be very different from one of eBay’s. Some give greater leeway to ESPs and other commercial senders who subscribe to feedback loops, because the feedback allows them to take action quickly. Others will vary based on the country of origin of the message, knowing that a particular set of users is unlikely to want email written in a language they don’t know how to read.

This variety of categories benefits users, because it increases the likelihood of catching unwanted mail while decreasing the likelihood of misplacing something the user actually wanted. However, it frustrates senders who’d prefer to have a single, simple numeric goal that they don’t have to think about. (Since the ISPs work for their users, you can guess whose preference wins out.) In any case, one reliable rule is that any behavior outside of the norm – no matter which category’s “norm” is used – is considered suspicious.

minute read

Popular stories

Products

BriteVerify

BriteVerify email verification ensures that an email address actually exists in real-time

DemandTools

The #1 global data quality tool used by thousands of Salesforce admins

Everest

Insights and deliverability guidance from the only all-in-one email marketing solution

GridBuddy Cloud

Transform how you interact with your data through the versatility of grids.

Return Path

World-class deliverability applications to optimize email marketing programs

Trust Assessments

A revolutionary new solution for assessing Salesforce data quality

Solutions

Validity for Email

Increase inbox placement and maximize subscriber reach with clean and actionable data

Validity for Data Management

Simplify data management with solutions that improve data quality and increase CRM adoption

Validity for Sales Productivity

Give your sales team back hours per day with tools designed to increase productivity and mitigate pipeline risks in real-time