Filtering mail server side with Sieve, part 1

Armijn Hemel, August 31, 2010, 7897 views.

Like us, you might receive a lot of mail and read it from different locations, using different mail programs, on different kinds of devices, including mobile. If not all mail is relevant at all times, you might want to look at server side filtering instead of client side filtering.

Tags: , ,

Without bragging: I get a lot of mail. Apart from the usual spam there are chain letters with jokes that are not funny, legitimate commercial e-mail, mailinglists, mail from bug trackers and regular work mail. Needless to say I try to be ruthless in filtering. Everything that distracts from the absolute essential is filtered out by my mail programs (Thunderbird and Evolution). But there are good reasons why you would not want to do this on the client, but on the server as soon as mail arrives.

Why filter on a server instead of a client?

Filtering of messages into different folders is usually done client side: the mail program downloads headers and possibly the mail body via IMAP and applies local filters, after which mails are moved, deleted, flagged, and so on.

The processing of e-mails is done by the mail program (clientside). If you have enough bandwidth, the device your mail program is running on has enough processing power and you just use one program this is probably not much of an issue.

It is getting far more likely that this is not the situation.

Low or expensive bandwidth

It is true that bandwidth has increased a lot over the years, but there are still places where you have just slightly more bandwidth than dial up, or just exactly that, with GPRS for example. This bandwidth often is expensive as well, compared to `normal' broadband. If you just want to check if there has been some important mail, but don't care about all the mailinglists and other things you would normally filter it is irritating to first have to download all the headers and filter them. Doing it on a server is a lot more convenient and saves you the burden of first downloading mail headers (and possibly mails).

No powerful device

A lot more people are moving to smaller, less powerful devices, like smart phones. Of course these are still as powerful as the computers of, say, 8 years ago, but filtering is still not something I want to do on my mobile phone.

Multiple mail programs

If you read mail from several locations (home, work, phone, Internet cafe) you need to make sure that your rules are in sync, so your mails are always filtered in the same way. Server side filtering makes this a non-issue.

Server side filtering

A few years ago I mainly used procmail to filter incoming mail to various mbox mailboxes. It worked quite well, but the setup on that machine was that mail was delivered locally to Unix accounts. These days I'm using a setup which uses virtual users and there are no corresponding Unix accounts.

Although procmail works great it is hard to configure for use with virtual accounts. There are people who got it working, but their experiences clearly showed it is a bit of a hack to get it to work.

Introducing Sieve

I was very pleased to learn that there is something called Sieve. Sieve is a standardized mail filtering language which hooks into your IMAP server and filters incoming mail. The language has its origins in the Cyrus IMAP server, but other IMAP servers, like Dovecot, support it as well. The language has been defined and extended in a few RFCs.

A difference between procmail and Sieve is that with procmail you can run external commands. In a virtual setup where all mail is owned by one user this could be problematic or even downright dangerous if you let users decide what commands to run, so Sieve does not allow external commands, but is kept simple instead. Several extensions to Sieve allow more functionality, such as vacation messages, notifications if certain mail arrives, regular expressions, and so on.

Sieve scripts are executed for a user every time a mail arrives. Sieve scripts are simple text files with commands in it. Every user can have their own collection of Sieve scripts. Usually there is one Sieve script active per user, although there is a linking/chaining mechanism

ManageSieve

Sieve scripts need to be stored on a server before they can be executed. There are various ways, such as uploading using conventional methods, but these might not be suitable if you have virtual mail accounts. A special protocol for uploading and managing Sieve scripts called ManageSieve is defined in RFC 5804. Server support for ManageSieve is quite good, at least in Dovecot, where we tested it. Client support for ManageSieve is still a bit shaky.

In a few upcoming blog articles we will look at some simple scripts, look at how to configure Dovecot for it and test with a few clients that support Sieve.

Social networking: Tweet this article on Twitter Pass on this article on LinkedIn Bookmark this article on Google Bookmark this article on Yahoo! Bookmark this article on Technorati Bookmark this article on Delicious Share this article on Facebook Digg this article on Digg Submit this article to Reddit Thumb this article up at StumbleUpon Submit this article to Furl

Talkback

respond to this article