Microsoft and others finally endorse an anti-spam framework…

Microsoft has revealed that it will merge its Caller ID for Email technology with a competing scheme for email authentication, known as Sender Policy Framework (SPF).

The bringing together of these two technical specifications could be a critical step in eliminating the spam problem and provide a huge win for email users worldwide and help in restoring user trust and value to email.

The combined technology proposals will be merged into one single proposal that will be submitted to the Internet Engineering Task Force (IETF) standards body for consideration as an Internet standard. Microsoft had previously planned to propose its Caller ID for Email as a standard, but had noted that the technology was complementary to other similar proposals. The backers of both anti-spam proposals now say they will work together to provide a single standard that will make it easier for Internet providers to block unwanted junk e-mail. Both aim to weed out fake e-mail addresses used by spammers to cover their tracks.

How can spammers get away with their fraudulent behaviour and how do these two technologies combat their efforts?

Both SPF, also known as "Sender Permitted From," and Caller ID attack a fundamental weakness in the ubiquitous Simple Mail Transfer Protocol: that is, those receiving E-mail have no easy way of determining whether senders are in fact who they say they are.

This is an especially serious problem for Internet service providers, all of whom would very much like to stop fraudulently addressed, or "spoofed," e-mail long before it gets delivered to their users in-boxes, especially if it’s stopped before it's sent.

So the combined proposal will form the basis for a technology that stop spammers from spoofing domains and sending email messages that appear to be from a trusted sender. The two methods are designed to ensure that the sender's return email address is real. They allow Internet service providers to check the authenticity of incoming email by verifying it with records from the domain name system database.

What is Spoofing again?

Spammers often appropriate the e-mail addresses of others in order to slip through content filters, a tactic known as "spoofing." So what we're trying to do here is to tell if an incoming e-mail is really coming from where it says it's coming from.

Spoofing is especially popular with scam artists who pose as companies like Banks and on-line auction sites such as eBay etc. in an attempt to collect credit card numbers and other sensitive information.

Email messages from these spoofed email addresses are very often used to re-direct users to malicious Web sites that then launch so-called phishing attacks. Spoofed email is also used to deliver viruses and other electronic attacks directly to users' computers.

What is phishing (pronounced “fishing”)?

It will also help combat the growing problem of phishing where criminals forge addresses of financial institutions and retailers in order to trick people into divulging financial and personal information. This is where con artists convince people to hand over user names, passwords and credit card numbers by posing as a legitimate business. That con is made easier, because SMTP lets e-mail senders claim to be anyone.

Where do the two methods differ in their approach?

The combined SPF and Caller ID, which have yet to be given a name, each evaluate a different part of the email to verify its authenticity. SPF examines the envelope information, sometimes called the header, whereas Caller ID looks at the content of the email to establish identity. A melding of the two specifications should produce a stronger authentication standard.

The proposals would help solve the domain-spoofing problem, which accounts for at least fifty percent of all spam. "Spoofing", as mentioned, is a tactic used by spammers to make return addresses appear legitimate to the recipient's spam filters.

Are these the only methods of controlling Spam at the moment?

E-mail authentication proposals have been around since at least 1998, but during the past year, spam has exploded to account for more than 83 percent of all Internet traffic, and so experts have given the concept more attention.

Technical proposals abound for fixing the authentication problem. A recent collection focused on the idea that ISPs could publish the range of Internet Protocol addresses associated with their e-mail domains. That way, an e-mail recipient's service provider could check the sender's stated domain (address) against the published IP address. If there's no match, the recipient's ISP can fairly safely assume that the message is spam, or at least suspiciously addressed.

Meanwhile, Yahoo has proposed another approach that would use digital signatures to authenticate e-mail. Their technology has the same objective as Caller ID but through a different system. DomainKeys matches digital signatures between the email and the server to gain admission to a person's inbox.

How will the spam actually get blocked at the users end?

Both Microsoft's Caller ID for e-mail and Sender Policy Framework will allow Internet Service Providers to check that a message that has been sent from, say, someone@example.com actually comes from the numerical addresses used by example.com.au's actual e-mail servers. Mail that does not match up could then be safely identified and rejected as spam.

Both Caller ID and SPF will confirm a sender's domain or Internet address. Anti-spam filters can then be created by an organisation to block messages with From addresses that don't match the real sending domain.

Because the merged specification will allow testing for spoofing both at the message transport (SMTP) level, and in the message body headers, this will allow administrators to block some spam before it's sent, while the content examination will deeply probe messages to detect phishing attacks.

So is this going to mean a major change in the way we use Email?

Microsoft's proposed changes to the email infrastructure are quite small and should not be disruptive. The standard would pose few difficulties for most companies that handle e-mail, and individual users would not have to make any changes at all.

Both of these solutions should be seen in the context of a stepping-stone to more technical solutions over time. However, there could be a two-fold benefit. As the IP addresses of many spammers are picked up and become known to Internet service providers, they can then be further blocked.

So do we at last have a fix for Spam?

Although the proposed merger is welcome news some analysts warn that we should not to get too excited too quickly. We may not to see anything happening overnight.

Some are saying, "Microsoft is going to have to do a lot of hustling in the next few months with SPF, and Yahoo too, to come to some sort of agreement." This is one of those technologies that unless the majority of the world uses it, may not reach its full potential.

Down the road, however, things may be completely different. If a single standard is adopted, much of the kind of spam that hounds us today will be a thing of the past and be blocked by these very techniques.

On the other hand some analysts believe that only a dramatic reworking of the entire email infrastructure can turn the tide against spammers. Whether these methods will have any more success against the spam plague than earlier solutions, such as antispam filters and legislation, they say, is still unclear.

When is this technology likely to be taken up?

The aim is to submit their combined proposal to the standards-setting Internet Engineering Task Force for approval in a month, with the hopes that it could be widely adopted in a year or so.

DomainKeys is said to be the longer-term approach; SPF the short-term, if all goes well, they should meet in the middle and squash spammers like a bug.

Arthur Hissey
Computer Research & Technology
www.crt.net.au