Spam Filtering Explained

What follows is a basic run-down of what happens when an email is received on a mail server and it passes through an anti-spam filtering product.  Anti-Spam products vary from one manufacturer to another but the essence of the job that they perform is to work out if a message is from a genuine sender or a potential spammer and to either accept or reject the message accordingly.

Some Tests that anti-spam software can perform on inbound mail:

  • DNS Whitelist
  • DNS Blacklist
  • Auto Sender Whitelist
  • Reverse DNS
  • HELO domain blacklist
  • SPF Check
  • IP Blacklist
  • IP Whitelist
  • Sender Blacklist
  • Recipient Blacklist
  • Directory Harvest Protection
  • Honeypot Check
  • Greylisting
  • Keyword Blacklist
  • URL Domain Blacklist

DNS Whitelist

If you have your name down on a Guest-List at a night-club – you can walk past the queues of people waiting outside and you are immediately let into the night-club without any delay.  Being on a DNS Whitelist is the equivalent of having your name down on the guest-list.  If your IP address is found on a DNS Whitelist, your emails are simply and quickly allowed through the filtering software because you are considered a safe source.

DNS Blacklist

Using the same analogy – This is the opposite of the Guest List – it is the “Never Let Them In To The Night-Club” list or the “Permanently Barred” list.  If your name is down on this list, the bouncers at the door will not let you in, no matter how much you try and persuade them to let you in.  Being on a DNS Blacklist is every Email Administrators nightmare.  If your mail server’s IP Address pops up on a DNS Blacklist – then there is a high probability that your email will get rejected because you are not trusted and have most probably been sending out spam or because your server is not configured properly.

Auto-Sender Whitelist

Some Anti-Spam software can automatically add the email addresses of people you send emails to onto a list that allows their return emails back into your server with minimal checks simply because you have emailed them and thus have a higher trust level for their emails over and above emails from people you have not emailed (and theoretically don’t know).

Reverse DNS

A bit of background info here on communications between mail servers.  When a sending server connects to a receiving server, the first thing it does (to be polite) is to say “Hello”.  The command it usually uses to do this is HELO followed by the FQDN of the Sending Server e.g., HELO  When the Receiving Server accepts the connection from the Sending Server, the Receiving Server knows the IP Address of the Sending Server and is then told its HELO name.  It can then perform a quick check to see if the IP Address of the Sending Server is configured correctly with Reverse DNS (the usual process of DNS is to resolve a FQDN to an IP Address – Reverse DNS does the opposite and resolves an IP Address to a FQDN).  So, if the FQDN of the Sending Server and its IP Address do not match when performing a Reverse DNS check, then the Receiving Server can reject the connection and refuse any mail.

HELO Domain Blacklist

The HELO blacklist checks to see that the FQDN of the Sending Server is configured properly. would be an example of a correctly configured FQDN.  MAILSERVER would be an example of an incorrectly configured FQDN.

Most spammers will use a hijacked PC to send out spam and these usually have names like HOMEPC or something similar, not  If Anti-Spam software sees the FQDN of the sending server as HOMEPC – it can choose to reject it based on this because it is not configured correctly and any mail being sent from this computer is most likely spam.

SPF Check

SPF Stands for Sender Policy Framework.  This is a recent addition to the fight against spam and when configured (properly) can be used to reject mail from spammers.

An SPF record can be configured on a domain by domain basis.  The job of the SPF record is to tell the world which mail servers are allowed to send out mail for that particular domain.  When a Receiving Server opens a connection to a Sending Server and learns its IP Address and FQDN – it can lookup the domain name and see if there is an SPF record setup.  If one is setup – it then has to work out if the Sending Server IP Address is allowed to send mail on behalf of the domain based on the results of the SPF record.  If no SPF record exists, then the SPF check neither passes or fails and the Anti-Spam software will let the mail through.  If the SPF check fails – any mail received will be rejected and likewise, if the SPF check passes – the email will be allowed to continue on its way to pass any additional Anti-Spam checks.  If the SPF record is configured incorrectly, emails can be rejected because of false results, so anyone setting up an SPF record needs to set it up correctly otherwise they may have problems sending mail.  No SPF record is much more preferable to an incorrectly configured record!

IP Blacklist

This is usually a list that can be added to manually.  There can be many reasons for adding IP Addresses to an IP Blacklist – the usual reasons are because you have received lots of spam from an IP Address or range of IP Addresses and your Anti-Spam software is not catching it.

IP Whitelist

If you have people / customers that have a fixed IP Address and you don’t want to have any of their emails stopped by any Anti-Spam checks, adding their IP Address to this Whitelist will allow their emails straight in to your server.  The only problem with this is that if their server becomes infected, you could allow Spam straight through as a result.

Sender Blacklist

If you receive spam from a particular email address or domain, adding an email address or domain name to this type of list will block all mail from the address or domain immediately.

Recipient Blacklist

If you want to block all mail sent to a specific email address, you can add the email address to this list and all mail will be rejected.

Directory Harvest Protection

What spammers want is to gather as many email addresses as they can so that they can send out mail to those addresses.  The spammers will have programs running on computers that scour the Internet for email addresses and when found, will be recorded.  They also will try to suck out the names and email addresses from mail servers by sending a server a message and changing the To: address.  When their programs receive a response from a mail server saying “Invalid Recipient” – they know the email address they are using is wrong.  If they don’t get an Invalid Recipient” response, they will know that the address is valid and will add it to their list.

The Job of the Directory Harvest Protection check is to restrict the number of email addresses they can check in any one connection and if exceeded, the software can tell the server to drop the connection and potentially not to accept any more connections from that IP Address for a specified period of time.

Honeypot Check

A Honeypot is essentially an email address that either has never been setup or it has been expressly setup and advertised (in a hidden form on the World Wide Web) for spammers to find so that any email received to that email address will flag the sender up as a spammer.  It is essentially a ‘Trap’ and when a spammer falls into the trap – it triggers an immediate logging of the sending IP Address as a spammer.  DNS Blacklists usually add IP Addresses as a result of an email being received in such a trap (honeypot).


If Whitelisting is the equivalent of having your name on the ‘Guest-List’ and Blacklisting is the equivalent of being on the “Barred’ list, then Greylisting is somewhere between the two.

What happens if Greylisting is enabled is the first connection attempt from any new IP Address / Email Sender will be automatically rejected for a short amount of time.  Any further attempts to connect from the same IP Address will not be Greylisted, unless the next connection is later than the Greylisting timout period.  The reason for this is that Spammers usually want to just connect to your mail server, send you their spam emails and then disconnect and move on to the next mail server.  If Greylisting is enabled, the spammer will be told to “Go Away and Come Back Later”.  Most of them will never return, so Greylisting can be a simple but very effective Anti-Spam technique.

Keyword Blacklist

This list can be used to block specific words such a Viagra / Vi@gra / Vi@gr@ etc.  Care has to be taken when adding words to such as list as a word such as ‘Sex’ will catch out valid emails with the word Essex and Wessex etc.  Adding things such as XXX (meaning Adult) – can catch out someone sending three kisses at the end of an email.

URL Blacklist

Buried within lots of spam emails are often URL links to websites that can automatically make you think your computer is infected with a virus or are Phishing Websites setup to fool you into thinking you are on a genuine website.  Such emails are the usual “Access to your bank account has been restricted – please click the following link to verify your security details” type email.  All that happens if you visit such a site and confirm your security details is that your bank account will get emptied, so stopping such emails reaching your inbox will protect you from innocently giving away access to your bank account.

The checks above are just some of the ways that emails can be checked to see if they are spam or not.  A well configured server with a good anti-spam product installed should not reject messages incorrectly and ideally should not let much spam through to your inbox, but as can so easily happen, a well-configured server can have its security breached and thousands of spam messages can be sent out before the server administrators realise it and stop it, resulting in spam being received because the sender / sending IP is either trusted, whitelisted or not usually sending out spam.