Personal Postfix + Dovecot mailservers
09 August 2024For most of my domains that I use for email there is a simple forwarding rule that sends emails to my main email address, which in turn splits emails off to separate folders based on the address. This forwarding is typically done by the “parked page” facilities of the DNS registrar but when looking into DNSSEC hosting one thing I had to consider was how to forward emails since the likes of deSEC in their simplicity do not have such facilities. The only real solution would have been to run my own forwarding servers. In the end I opted for CloudFlare that does have the option of forwarding emails, but decided to try out setting up my own self-contained email infrastructure anyway with the following requirements:
- Accept emails from multiple domains
- Have per-domain inbox folders
- Be able to access the folders via secure IMAP
- Redundancy by having a backup relay server
pgu.org.uk
as a place-holder which is one of the disused domains I own and used for testing purposes, but have since been removed from the servers and no longer point anywhere.
It happened to be far simpler to use a domain that is not intended to be used for production use rather than going through the recorded steps and replacing them with the classical example.com
place-holder, that latter of which is anyway needed as the palce-holder for a secondary domain.
Setting up Let's Encrypt
In this day and age use of SSL encryption is now standard and an essential ingredient of this is an appropriate security certificate, and these are available for free from Let's Encrypt. One down-side is the certificates they issue are only valid for three months rather than the normal twelve although they do provide ways for the certificates to be automatically issues and renewed, but for testing purposes this limitation is not an issue. However here the manual process using DNS validation will be used as it needs the least up-front infrastructure.Getting and setting up certbot
There are a load of different ways of obtaining Let's Encrypt certificates but the one used here is via the command-line Python version of the certbot tool which rather conveniently is available from the Python Package Index so can be installed using thepip
command.
python3 -m venv envLetsEnc . ./envLetsEnc/bin/activate pip install certbot
My preference is against running things as root so some option parameters are needed to redirect some built-in paths, so I wrote a script that instead uses the current directly and creates sub-directories as needed.
It then runs the certbot tool in manual mode using DNS validation of domain name ownership.
It is assumed this script is named certbot.sh
.
#!/bin/bash BASE_DIR=$(pwd) echo "Using base: ${BASE_DIR}" mkdir -p "${BASE_DIR/etc}" "${BASE_DIR/var}" "${BASE_DIR/log}" certbot \ --config-dir "${BASE_DIR}/etc" \ --work-dir "${BASE_DIR}/var" \ --logs-dir "${BASE_DIR}/log" \ --preferred-challenges=dns \ --manual certonly \ $*
Getting the certificate
It is well advised to do a dry-run using the--dry-run
options which takes you through the domain validation procedure but does not create the certificate itself, since it is very easy to make a mistake or two going through the interactive instructions.
If the -d
parameters are omitted the tool will prompt for the domains instead:
$ ./certbot.sh --dry-run certonly -d pgu.org.uk -d mail.pgu.org.uk
In the process the tool will ask for TXT record(s) to be added to the domain name record and it is these that verify ownership of the domain.
Here the DNS name-server service being used is deSEC which is a free provider of DNSSEC-enabled DNS hosting which is also a lot easier to use than most other DNS record interfaces.
Note that certbot makes reference to _acme-challenge.pgu.org.uk.
but when adding the DNS record the domain part (i.e. pgu.org.uk
) is omitted.
Unlike some other DNS providers deSEC requires TXT records to be quoted, and it will display an error if they are missing.
For the sub-domain where certbot mentions _acme-challenge.mail.pgu.org.uk.
only the _acme-challenge.mail
part is put in the TXT record.
While deSEC updates its name-server very quickly the changes can take a while to propagate and one way of checking whether propagatioon has happened is via the dig
command.
The tool itself will emit a URL that can also be used for checking, and an example session is given in the appendix.
$ dig _acme-challenge.pgu.org.uk TXT +short
If all works out the location of the private key & certificate will be displayed along with information related to expiry and the future renewal.
Keep the files in the ./etc
sub-directory around as they will be needed.
If this directory is moved then file paths within ./etc/renewal/*.conf
might need to be updated.
Setting up the mail server
Handling of incoming mail is done using Postfix which is a drop-in replacement for the now often-shunned sendmail. Setting up Postfix or any similar mail handling software needs care because any security lapse caused by misconfiguration such as acting as an open relay will sooner or later be exploited for nefarious purposes resulting in a world of pain. It is why many internet provides disallow them by blocking the relevant ports. Firstly a setup that receives email targeted at@pgu.org.uk
which is delivered to a local account;
secondly further setup so that email can be received from the virtual domain @example.com
;
and finally separating out mail sent via these two domains into folders.
Basic mail-server setup
The first stage is to get up and running something that will securely accept and deliver inbound email targeted as a specific Linux account, which is what the example configuration file shown below will do. Parameters that have been omitted will take on default values and just the ones of interest are shown, so themain.cf
on the server only needs to contain what is shown here.
myhostname = mail.pgu.org.uk mydomain = pgu.org.uk myorigin = $mydomain mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain mynetworks_style = host local_recipient_maps = unix:passwd.byname $alias_maps ## virtual_alias_domains = example.com ## virtual_alias_maps = hash:/etc/postfix/virtual.cfg alias_maps = hash:/etc/aliases alias_database = $alias_maps smtpd_relay_restrictions = permit_mynetworks, defer_unauth_destination smtpd_recipient_restrictions = smtpd_tls_cert_file=/etc/postfix/ssl/fullchain.pem smtpd_tls_key_file=/etc/postfix/ssl/privkey.pem smtpd_tls_security_level = may smtpd_tls_auth_only = yes compatibility_level = 3 unknown_local_recipient_reject_code = 450 soft_bounce = yes ## mailbox_command = /usr/bin/procmail -a "$EXTENSION"
The commented-out lines will be mentioned later when further features are enabled, and the other parameters are as follows:-
myhostname
- Address of the server
mydomain
- Domain name server's address is within.
myorigin
- Domain for local mail.
mydestination
- Domains & sub-domains the server is the final destination for, and hence will pass emails onto the local delivery system for dispatch to user mailboxes.
Aside from
$mydomain
the other parameters are supposedly to stop local mail loops. mynetworks_style
- Only the local host is considered “trusted” and hence able to have email relayed to third-party destinations. An alternative is to set
mynetworks
manually. local_recipient_maps
- Where to check if a local recipiant is valid. In this case it checks local user accounts and things covered by
alias_maps
. alias_maps
- Aliasing of local mail recipiants. See below for details.
alias_database
- This lists files that should be reindexed when
newaliases
is called. This is basically all entries inalias_maps
starting withhash:
,btree:
, ordbm:
. smtpd_relay_restrictions
smtpd_recipient_restrictions
- These two parameters between them handle what emails are accepted and rejected, and they are evaluated in order with
permit_mynetworks
allowing anything thatmynetworks
permits, anddefer_unauth_destination
rejects anything that neither relaying nor local delivery will accept. In production the harsherreject_unauth_destination
may be preferrred to the latter. smtpd_tls_*
- SSL-related parameters.
compatibility_level
- This is present for backward-compatibility and decides which parameters are permitted and what default values to use.
soft_bounce
- If this is set Postfix treats emails that would normally bounce more forgivingly, such as keeping them queued rather than doing an immediate return to sender, and using non-permanent status codes. Useful when trying to sort of configurations but should not be left on long-term.
unknown_local_recipient_reject_code
- The SMTP status code to use for unknown local recipients.
/etc/postfix/ssl/
for SSL to work properly.
At this stage it should be possible to start Postfix up and fire off emails to @pgu.org.uk
and their reception into a user's mail spool can be checked using pine
from the console.
Checking relay security
One of the absolutely first things to do upon the server going live is to check with a tool such as those from MX-Toolbox or App River to make sure it will not relay unauthorised emails. If it does act as an open relay and you do not discover it at this point, someone you don't want to have any dealing with very quickly will. It is also worth checking with SSL Tools that the certificate setup is all well and good.System aliases
System aliases reroute local mail to Linux accounts that will actually be read, and some of these such as email to postmaster should be present.# Person who should get root's mail. Don't receive mail as root! root: remy # Basic system aliases -- these MUST be present MAILER-DAEMON: postmaster postmaster: root
Handling “virtual” domains
Virtual domains are what Postfix considers to be domains that it accepts email from but then selectively forwards rather than acting as the final destination, and hereexample.com
is used as the canonical example.
In main.cf
shown above the lines virtual_alias_domains
and virtual_alias_maps
need to be uncommented.
The first parameter lists the virtual domains to process and the second specifies the file that contains the list of valid addresses and where to route them to, the latter file having the contents like this shown below.
Replace the usernames with whatever local Linux account(s) should receive the respective emails.
The last rule is a catch-all and where it appears should not matter.
foo@example.com foo bar@example.com bar @example.com remy
Splitting mail into folders
Rather than delivering emails directly user mailboxes Dovecot is setup to hand them off to Procmail which in turn delivers it to the respective folders based on per-user configuration. Withinmain.cf
use of Procmail is enabled by uncommenting the line mailbox_command
and the following ~/.procmailrc
within the destination's user's home directory does the mailbox routing.
This recipe filters out emails based on the X-Original-To
header since this catches blind carbon-copied emails.
MAILDIR=$HOME/Mail DEFAULT=$HOME/Mail/ :0: * ^X-Original-To: list@example.com .MailingList/ :0: * ^X-Original-To: (.*)@example.com .example/
The prefixed dot is part of the Maildir directory structure and the trailing slash indicates that Maildir folders are to be used rather than the older Mbox.
The first rule filters emails to list@example.com
into a folder called MailingList
and the second filters everything else to that domain into example
.
Procmail uses and stops at the first rule that matches unless it has options used to do otherwise.
Using the Mutt configuration file given in the appendix it is possible to use Mutt to check that emails ar going into the correct
Setting up a backup relay
For purposes of redundancy a domain accepting email should have more than one MX server and the usual practice is to specify two, although Google is known for having as many as five for load-balancing purposes. The architecture used here is the backup server storing any messages it receives in a local queue until it is able to forward them to the primary server, and the configuration here is based on Postfix's remote site example.myhostname = mx2.pgu.org.uk mydomain = pgu.org.uk myorigin = $mydomain mynetworks_style = host mydestination = relay_domains = pgu.org.uk, example.com relay_recipient_maps = hash:/etc/postfix/relay_addrs.cfg local_recipient_maps = local_transport = error:local mail delivery is disabled virtual_alias_maps = hash:/etc/postfix/local_addrs.cfg smtpd_relay_restrictions = defer_unauth_destination smtpd_recipient_restrictions = smtp_tls_CApath = /etc/ssl/certs/ smtp_enforce_tls = yes smtpd_tls_cert_file=/etc/postfix/ssl/fullchain.pem smtpd_tls_key_file=/etc/postfix/ssl/privkey.pem smtpd_tls_security_level = may smtpd_tls_auth_only = yes soft_bounce = no compatibility_level = 3
Don't forget to add mx2.pgu.org.uk
as an MX record with lower priority on all domains, including any virtual domains.
Parameters that are unchanged compared to the primary mail server configuration have been omitted from the list of details:-
mydestination
- Don't accept any mail for local machine. Important bit however is the absence of
$domain
as the local machine is not the final destination forpgu.org.uk
emails. relay_domains
- List of domains to be accepted for forwarding.
relay_recipient_maps
- Destination addresses that will be accepted for relaying. See below for details.
local_recipient_maps
local_transport
- There are no local mail recipients.
virtual_alias_maps
- Routing of Postmaster email to somewhere it can be read. See below for details.
smtpd_relay_restrictions
- Reject emails that are not for an authorised destination. In practice this means anything inbound that does not match a domain listed in
relay_domains
. Some other guides mention the deprecatedpermit_mx_backup
going here. smtpd_recipient_restrictions
- Empty since
smtpd_relay_restrictions
has done all the filtering work already. smtp_tls_CApath
- Points to the system certificate authority respository, so the certificate of the primary server can be checked.
smtp_enforce_tls
- Enforce validation of server certificate when connecting to primary server.
postqueue -h
can be used to show what deferred emails are in the queue waiting delivery, and postqueue -f
can be used to flush the queue if the primary server is known to be available.
Authorised email address list
Having a backup mail server store emails with invalid addresses for later rejection and bouncing when the primary server comes back online is a source for back-scatter which will lead to trouble, so the relay needs a list of valid addresses so it can immediately reject invalid destinations. This is where/etc/postfix/relay_addrs.cfg
comes in and it lists all addresses that should be accepted.
The @example.com
entry is a domain-wide catchall.
postmaster@pgu.org.uk Ok remy@pgu.org.uk Ok @example.com Ok
The Ok
is due to the Postfix file format expecting something to be there even though it is not used in this case, and if left out postmap
will choke whwn re-indexing the file.
Routing postmaster messages
Email to certain local system destinations should be routed somewhere sane, and by using full email addresses/etc/postfix/local_addrs.cfg
effectively sends them to a mailbox on the primary mail server.
MAILER-DAEMON postmaster@pgu.org.uk postmaster postmaster@pgu.org.uk
Check open relay status!
As with the primary server it is advisable to use an online tool to check whether it acts as an open relay or not. During testing the logs indicated there were unauthorised relay attempts within five minutes of the backup server going live.Installing & setting up Dovecot
Once emails are making their way into Linux accounts they are then accessed via IMAP which is served using Dovecot, and mercifully does not need that much configuration as well as not having quite the same misconfiguration hazards as Postfix. At least on Slackware Dovecot's configuration is split into lots of files and the snippets below are the individual files requiring changes with all the comments and empty lines filtered out. Coincidentally the Postfix has the commanddoveconf -nP
that outputs a single configuration consisting of all non-default options.
Enforce use of SSL
Dovecot can use the same certificates that Postfix uses sothe- paths inconf.d/10-ssl.conf
are set to point at them.
There is no reason to allow unencrypted connections so use of SSL is also set to required rather than yes.
The <
is part of Dovecot's configuration syntax and is not a typo.
ssl = required ssl_cert = </etc/postfix/ssl/fullchain.pem ssl_key = </etc/postfix/ssl/privkey.pem
Setup mail folder locations
The mail folder locations inconf.d/10-mail.conf
need to match up with where Procmail delivers messages.
The parameter mailbox_list_index_very_dirty_syncs
cam be set to yes to increase performance but this is best done once everything else is working.
mail_location = maildir:~/Mail/ namespace inbox { inbox = yes } mailbox_list_index_very_dirty_syncs = no protocol !indexer-worker { }
Setup ‘special’ folders
Withinconf.d/15-mailboxes.conf
are special use folders for things like draft emails.
The snippet below sets up the four folders that Thunderbird uses: Archive, Draft, Send, and Templates.
namespace inbox { mailbox Drafts { auto = create special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Archive { auto = create special_use = \Archive } mailbox Templates { auto = create } }
In this case all the folders are automatically created on the server but only the Sent folder is automatically subscribed to by email clients.
Setting up PAM
Since in this setup Dovecot uses Linux accounts, PAM is used for authentication which requires/etc/pam.d/dovecot
to have the following content.
There seem to be non-PAM options for Dovecot to authenticate with Linux logins but they did not work in the short amount of time they were tried.
#%PAM-1.0 auth required pam_unix.so account required pam_unix.so
Appendix
Setting up Mutt for Maildir mailboxes
The following~/.muttrc
sets the console-based Mutt email client to access the Maildir-type mailboxes under ~/Mail
which is useful for testing prior to installing Dovecot.
Mutt is far from the friendliest of email programs but at least on Slackware it is installed by default.
The bottom three lines set things up so pressing the C button brings up a list of mailboxes found under ~/Mail
.
set mbox_type=Maildir
#set mbox="~/Mail"
set folder="~/Mail"
set mask="!^\\.[^.]"
set spoolfile="~/Mail"
mailboxes `echo -n "+ "; find ~/Mail -maxdepth 1 -type d -name ".*" -printf "+'%f' "`
macro index c "
The story behind PGU
Just over five years ago I was in the process of clearing out unused domains and due to their sky-rocketing prices UK domains were mostly culled, butpgu.org.uk
was kept since domains that short usually have value financially and in terms of convenience.
I obtained it when I ran the Post-Graduate Union website at University and in protest at how the undergraduate sabbaticals rather acromoniously defunded and disbanded the PGU in 2009 due to financial problems, myself and the other person who maintained it decided to keep it up as a memorial.
For experimental purposes it seemed better to use a domain that is both unused and not expected to be used any time soon, but above all are also significantly different for in-use domains — using remy.org.nz
would be asking for trouble given how similar it is to remy.org.uk
.
Also for testing purposes another domain which previously was for a now-discontinued open-source software project was used as the secondary virtual domain but since that domain is due to lapse later this year the relatively few references to it were all changed to example.com
.
Using real values in examples always carries risks so at the very least they should not be ones that in the near future may be owned and used by someone else.
Example Certbot transaction
$ ./certbot.sh -d mx2.pgu.org.uk Using base: /home/remy/LetsEncrypt Saving debug log to /home/remy/LetsEncrypt/log/letsencrypt.log Requesting a certificate for mx2.pgu.org.uk - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Please deploy a DNS TXT record under the name: _acme-challenge.mx2.pgu.org.uk. with the following value: ln-XbC-z_hXmDVRJeAw0wQMNHOyiWmvs0lCebtQpeRQ Before continuing, verify the TXT record has been deployed. Depending on the DNS provider, this may take some time, from a few seconds to multiple minutes. You can check if it has finished deploying with aid of online tools, such as the Google Admin Toolbox: https://toolbox.googleapps.com/apps/dig/#TXT/_acme-challenge.mx2.pgu.org.uk. Look for one or more bolded line(s) below the line ';ANSWER'. It should show the value(s) you've just added. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Press Enter to Continue
At this point the DNS TXT record is added to the domain name. If there is m ore than one domain Certbit will ask about each one at a time.
Successfully received certificate. Certificate is saved at: /home/remy/LetsEncrypt/etc/live/mx2.pgu.org.uk/fullchain.pem Key is saved at: /home/remy/LetsEncrypt/etc/live/mx2.pgu.org.uk/privkey.pem This certificate expires on 2024-11-08. These files will be updated when the certificate renews. NEXT STEPS: - This certificate will not be renewed automatically. Autorenewal of --manual certificates requires the use of an authentication hook script (--manual-auth-hook) but one was not provided. To renew this certificate, repeat this same certbot command before the certificate's expiry date. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If you like Certbot, please consider supporting our work by: * Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate * Donating to EFF: https://eff.org/donate-le - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -