July 20, 2021

Filtering mail with Lua using imapfilter

Recently I came across imapfilter, and it's what I've always been searching for. The problem it solves is pretty simple: Client-side E-Mail filtering with something like Thunderbird is flawed because rules only apply while the application is running, which gets annoying when you also view mail on other devices.

Big E-Mail services of course noticed this issue and offer you to configure server-side filtering rules, however there is no standardization and the extent of options varies greatly.

Imapfilter solves this problem in a very simple way: It is very minimal, really only handling IMAP and exposes APIs for you to interact with it via Lua. Then you can run imapfilter with your config on an always-on machine like a SBC or a VPS and everything just works.

You don't really need to know Lua at all in order to do basic filtering, however it can help if you want to get more advanced.

Getting started

By default imapfilter reads from ~/.imapfilter/config.lua. Here's a minimal example:

options.timeout = 120
options.subscribe = true

account1 = IMAP {
    server = 'imap.mail.server',
    username = 'user@mail.server',
    password = 'secret1',
}

messages = account1["INBOX"]:contain_from("mailinglist@example.com")
messages:move_messages(account1["example-list"])

I think you get the gist of how imapfilter works by just looking at this example. It moves all mails in INBOX which are from mailinglist@example.com to the example-list folder. To test this config simply run

imapfilter

Sometimes the list address will only be in the CC field, we can just combine messages like so:

messages = account1["INBOX"]:contain_from("mailinglist@example.com") + account1["INBOX"]:contain_cc("mailinglist@example.com")

Imapfilter exposes many other functions aside from contain_from and contain_cc, they are documented in man imapfilter_config.

Always-On filtering using IMAP IDLE

I have imapfilter running in a daemon on a Raspberry Pi. To achieve this we use imapfilter's idle functionality:

while true do
	account1["INBOX"]:enter_idle()
	messages = account1["INBOX"]:contain_from("mailinglist@example.com")
	messages:move_messages(account1["example-list"])
end

Here the enter_idle function will hang until new mail is added to INBOX, then the rules are applied and because of the endless loop it then goes into a waiting state again.

My config

Sadly since everything is single-threaded you can't idle in more than one account at once. We solve this by just spinning up one imapfilter process per account using its own config file.

In common.lua I have a few shared configs and functions:

-- Options
options.timeout = 120
options.subscribe = true

-- Functions
function mailinglist(i, address, folder)
    messages = accounts[i]["INBOX"]:contain_cc(address) + accounts[i]["INBOX"]:contain_to(address)
    messages:move_messages(accounts[i][folder])
end

function do_idle(a, f)
    while true do
        accounts[a]["INBOX"]:enter_idle()
        f()
    end
end

Accounts are configured in accounts.lua:

-- Accounts
accounts = {}
credentials = {
    ["account1"] = {
		server = 'imap.mail.server',
		username = 'user1@mail.server',
		password = 'secret1',
    },
    ["account2"] = {
		server = 'imap.mail.server',
		username = 'user2@mail.server',
		password = 'secret2',
    },
}
function use_account(i)
    accounts[i] = IMAP{
        server = credentials[i].server,
        username = credentials[i].username,
        password = credentials[i].password,
        ssl = 'auto',
    }
end

Then each account gets a config file, so for example account1.lua:

package.path = os.getenv("HOME") .. "/.imapfilter/?.lua;" .. package.path

-- Accounts
require "common"
require "accounts"
use_account("account1")

-- Filtering
function filter()
    mailinglist("account1", "list1@example.com", "cool-list")
    mailinglist("account1", "list2@mailserver.org", "other-list")
end

filter()
do_idle("account1", filter)

I wrote a systemd service in /etc/systemd/system/imapfilter@.service:

[Unit]
Description=IMAPFilter for account %I
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/imapfilter -vc .imapfilter/%I.lua
WorkingDirectory=/home/imapfilter
Restart=always
RestartSec=60

SystemCallFilter=@system-service
PrivateTmp=yes
NoNewPrivileges=yes
ProtectSystem=strict

User=imapfilter

[Install]
WantedBy=multi-user.target

This way you can set up a daemon for account1 by simply running

systemctl enable --now imapfilter@account1

Filtering mail with Lua using imapfilter

Getting started

Always-On filtering using IMAP IDLE

My config

Further reading