Sieving through the Waves of Spam

posted in: 3DN, Projects | 0

This post is also available in: nlDutch

Sieve Introduction

 

Sieve is a script language for filtering email messages. For a long time at 3DN a mismash of separate tools has been used to try and deal only somewhat successfully with spam email. Now this has all changed through implementing sieve on our mail server.

Our mailserver has been running for years on the Debian variant of Linux. Debian is a highly reliable open distribution of Linux which comes standard with the Exim MTA. Exim is really quite difficult to configure. This is no big surprise as in fact it’s quite difficult to run a complicated mailserver these days. Still, after many years of muddling around with Exim we decided to change MTA to Postfix. Postfix in its own is also hardly simple to configure but as it turns out it has some features that make it easier to integrate with some other important tools related to running a mailserver.

Our goal was and is to have a mailserver which:

  • Receives email for multiple domains. As a business 3DN own several domains itself which made it necessary to set up the MTA for multiple domains. From this it has turned out to be a relatively easy step to host email for customer domains as well.
  • Provide a secure IMAP environment for multiple people to read their mail safely.
  • Provide an MTA through which multiple people from multiple domains can safely send and relay emails.
  • Reduce the vast amount of spam email.
  • Reduce the amount of virii being spread.

One of the great problems with Spam is that it cannot with 100% certainty be established if an email IS actually spam, this is why typically Spam gets filtered into a separate mailfolder. We can then periodically search through this spam folder an adjust our filters should there be any false positives.

Without describing our full mailserver setup I would like to elaborate somewhat on using Sieve in our mailserver environment.

Chapter 1. Sieve Plugin for Thunderbird

 

We have in the past used Thunderbird extensively for email. It has always been a fantastic tool which has its own spam filtering functionality. We have used Thunderbird’s functionality often to automatically file certain emails into their own folders. This is mostly based on a situation where everybody has their own desktop system where those filters can be setup easily. However, when we sit down behind another desktop, or, as happens much more frequently, our mobile phones, these filtering rules are no longer instantly accessible. Sieve is a script language that’s intended to run on the mailserver itself so email can be moved to folders as they are being delivered. As it turns out, Thunderbird even has, by means of a separate plugin, an interface to Pigeonhole/Sieve.

While this article is about how to setup the initial Sieve filters, Thunderbird is no longer required after this initial setup, it merely provides a convenient method of adding more filters at a later time.

In order to install Sieve in Thunderbird, we need to first download the Sieve plugin from the GIT development site. Unfortunately this is required as some changes in Sieve are not backwards compatible. It is therefore important to make sure we’re running a recent version of Thunderbird first.

After that, the Sieve plugin can be download here. When the file has been downloaded it can be installed into Thunderbird by going to the Add-Ons menu in Thunderbird. From this menu choose ‘Extensions’ -> ‘Install Add-On from File’ and choose the just downloaded XPI file. After the plugin is installed Thunderbird does not need to be restarted, we can simply go into the account settings and among the ‘Server Settings’, ‘Copies and Folders’ etc. we will find the ‘Sieve Settings’ there now.

Chapter 2. Sieve Settings

 

When we go into the Sieve Settings, we can choose safely ‘Yes, manage Sieve scripts for this account’. The defauls account settings can be left unchanged and we can now choose ‘Edit Filters’. We will then first get a message saying ‘Your mail server attempts to identify itself with invalid information’. It’s okay to continue here as this is caused by using a so-called self-signed security certificate. So click ‘Continue’ on this dialog and you should see a tab in Thunderbird with ‘Sieve Filters’ as its title. Right under the ‘Donate’ button there are some buttons that say ‘New…’, ‘Edit…’ etc.

We’re now going to create our first Sieve filter through clicking on ‘New’. A popup window will ask for the name of the new script, enter ‘Spam’ here and click on ‘Ok’.In the editor dialogue now enter the following:

require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
 fileinto "Sievespam";
}

After saving this Sieve script the 3DN mailserver will now look into your messages if there’s an email header in there that’s called ‘X-Spam-Flag’. This email header is added by Postfix after it has first sent the email through Spamassassin. Spamassassin does a lot of checks to determine if email may actually be spam and on determining that it is Spam it will set the value of the X-Spam-Flag header to ‘YES’.

After first receiving, then sending it through Spamassassin, and doing a range of other checks, Postfix finally hands over control to Dovecot’s LDA (local delivery agent). Dovecot’s LDA will then run the Spam Sieve script on the message and it will test for us if the X-Spam-Flag is set to ‘YES’; If this is the case it will automatically put the email into the folder ‘Sievespam’.

Using this mechanism, a LOT of functionality can be added on the server-side to your mailbox. We recommend that you read up on the Sieve website about this.