February 14, 2009

How do Spam Filters Work?

Over 95 percent of all email sent today is spam. That means if you have nothing filtering the email you read, only 5 of the 100 emails you look at will be legitimate. Within those 95 pieces of junk mail, several will contain tempting attachments that are really viruses, even more will contain letters of sorry trying to get you to send money, and even more will contain advertisements for fly by night pharmaceutical companies trying to sell you miracle drugs. How has time to deal with all of them? No one, and that is why Spam filters are becoming more and more important.

But how do spam filters know which emails are real and which are spam? Well, like any good security system, they run each message through several layers of tests and checks to ensure the email you get is not dangerous.

  1. Black Lists: The first layer of protection that a spam filter offers utilizes a black list. A black list is a list of IP address and domain names that in the past have sent out spam. It is similar to the black list used by clubs and casinos to keep out trouble makers. Where at a club, every person has their ID checked, spam filters do the same thing and look at where the email is coming from. If the sender's domain or mail servers IP address is on a black list, the message is refused. Some of the most popular blacklists that people use are Spam Cop, Barracuda Central and The Spamhaus Project, although there are hundreds of others, many of which use each other to increase their effectiveness.
  2. Key Word Searches: Spammers know what attracts peoples attention, they know that people love money, people love good looking men and women, people like exciting news, and because of this, they use the same type of word combinations to get people attention. This makes them predicable, and because of that, spam filters can look for key words and phrases and if present, block a message. For instance, if a message says something like "FREE VICODIN!!!!" its probably not legitimate, so the spam filter will mark it as spam. Some of the most common words used in spam are "millionaire" and "sex"
  3. Mail Formatting: Spam filters can also look at the design of an email. Is it all text? is it one big image? are their tons of links to other things. is any of the text hidden? By using characteristics like this, the spam filter can make an educated guess as to what is real and what isn't.
  4. Attachment Scanning: Spam filters will not only scan the email it self, but good ones will scan the attachment as well. This allows the filter to catch viruses and Spyware before they get to your computer.

When using any of the methods above on their own there is a good chance that you will both miss a lot, and let a lot through, so many spam filters use a point system to rate emails. For instance, a key word search might find a few things that look like spam, but could also be real, so it will give it 2 points, the formatting also looks bad, so it gives it another 2 points. The message comes from an IP that is not black listed, so no points given, and there are no attachments, so no points there wither. this is 4 points. So if the spam filter is only configured to block things that get 5 points or more, then the mail would go through, if it is configured to only allow 3s and below, it would be blocked.

Of course, no scan is ever going to be perfect. because of this good spam filters will have a way for the email recipients to see all mail that was filtered so they mark it as not spam so the spam filter doesn't make the mistake again, and also have an option to mark things it did let past as spam so the filter can better recognize the spam in the future.

          Bookmark this post:
          StumpleUpon DiggIt! Del.icio.us Yahoo Technorati Reddit Google