Email on Gentoo

I run an email server, web server, a couple web applications, MythTV, Bind, DHCP, firewalling, a filtering web proxy, NFS, and a Samba workgroup, just off the top of my head, on Gentoo. The biggest pole in the tent, as it were, is the email serving. Since I have a cable internet connection, I don't have a fixed IP address. I could pay $100/mo and get the business class service if my wife would let me, but, right now, it's the $35/mo basic service, and, in addition to not having a fixed IP address, I'm not allowed by my TOS to run any sort of servers on the connection. So I pay $8/mo for a Linux-based web host. It receives all my email and hosts my real site. I merely bounce all the email from this host to my machines at home. Once there, I can do whatever I want with it.

I like to use IMAP. It keeps all my mail handy no matter where I am. (With POP, once it's downloaded, it's only available on your local machine.) Also, I encrypt all my mail traffic with SSL, both sending and receiving. I don't feel I have anything to hide, but I want to at least encrypt passwords on public networks. The fact that I'm maintaining my privacy is a bonus, and I don't have to think about censoring conversations.

These capabilities of being able to securely send and retrieve email, no matter where I am, is not without some cost. As you can see below, it's quite a hairball. There's no end of howto's on the internet for configuring all of this. I read through many while setting up this mess. But none of them have a diagram of how email flows through the system. I wouldn't say I'm all that visual of a person, but sometimes I really need this, and this was one of those times. So I hope that this can be a help to someone else.

Note that this is a fine setup for a local network, when you want to use the /etc/passwd accounts for things like Samba and NFS as well as email. But this approach would require so many changes to be used when you had virtual domains or lots of non-OS users that I think it might be appropriate to start somewhere else if that's what you're shooting for. I suppose, though, that one advantage of that sort of method over this one would be that you wouldn't have to necessarily fool with SMTP over SSL. You could easily use some of the other authentication methods available for starting a SASL session (using a secondary account database) and get logged in without sending your passwords in the clear over the network.

There are 3 ports listening on the other side of the firewall: 25, 465, and 993. Port 143 listens only internally.

Port 25 will relay mail from my local network without encryption or authentication, but only from the local network.

/etc/postfix/main.cf:

mynetworks = 127.0.0.1, 192.168.0.0/24, 192.168.1.254

Port 25 will also receive mail meant for “me,” both from the outside world, and from internal, cron-type automated emails from my other machines. Note that entries in /etc/aliases are setup to bounce mail to the root account on my local machines to my personal account on my mail server.

/etc/postfix/main.cf:

mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain, mail.$mydomain, davidkrider.com, mail.davidkrider.com, davidkrider.homelinux.net, enterprise.$mydomain, reliant.$mydomain

To send email from my laptop while on the road, or a generic machine on the internet, I've set up both authentication and encryption. To get the encryption, you have to enable TLS on the SMTP port.

/etc/postfix/main.cf:

smtpd_use_tls=yes smtpd_tls_auth_only = yes smtpd_tls_key_file = /etc/ssl/dk.key smtpd_tls_cert_file = /etc/ssl/dk.crt smtpd_tls_CAfile = /etc/ssl/dk.pem smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s tls_random_source = dev:/dev/urandom

To get authentication, you need to configure Cyrus SASL (Simple Authentication and Security Layer). As you can see, I use “plain” and “login” mechanisms. This is primarily why I actually use encryption. Other forms of login use some sort of native encryption in the authentication method, like checking a pre-encrypted password against another database. I don't want the extra complexity; I just want to use the same database as the system user account. So I wind up using plain-text authentication, and SSL encryption over the whole connection.

/etc/conf.d/saslauthd:

SASLAUTHD_OPTS="${SASLAUTH_MECH} -a pam"

/etc/sasl2/smtpd.conf:

pwcheck_method: saslauthd mech_list: plain login

In a move that I actually applauded, even though it causes me some inconvenience, the company I work for has blocked outgoing connections on port 25. So I additionally set up the SMTP daemon to listen on the standard SSL-encrypted port 465.

master.cf:

smtps -o smtpd_tls_wrappermode

After the machine accepts email for sending, I have it use my ISP's relay. I didn't always do this. However, because of spamming, a lot of people sign up for blackhole lists. One of these lists will kill all mail from dynamic IP address ranges. I'm in that group, and I've seen my mail bounced because of it. So, I just use the relay. It sort of bothers me, a little, to not be a purist about this, but it's the right thing to do in my situation.

main.cf:

relayhost = [smtp.comcast.net]

Both received and sent mail get filtered through Amavis for protection against viruses, worms, and phishing attacks. (Yes, I run Linux on my desktop, but my wife doesn't, and I may not while on the road. And phishing doesn't care what OS you're running.) Amavis runs on port 10025, but the trick to getting it running with Postfix is to setup another SMTP daemon on another port to receive the cleansed emails back from Amavis.

/etc/postfix/main.cf:

content_filter = smtp-amavis:[127.0.0.1]:10024

/etc/postfix/master.cf:

smtp-amavis unix - - n - 2 smtp -o smtp_data_done_timeout=1200 -o smtp_send_xforward_command=yes 127.0.0.1:10025 inet n - n - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_client_restrictions= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 -o strict_rfc821_envelopes=yes -o smtpd_error_sleep_time=0 -o smtpd_soft_error_limit=1001 -o smtpd_hard_error_limit=1000

I choose NOT to have Amavis filter the email through Spamassassin. If I did this, there would be only one Spamassassin configuration for the whole machine, and I want per-user configuration capability. Also, I want Amavis to go ahead and pass on everything it marks so that I can see how it's doing.

/etc/amavisd.conf:

@bypass_spam_checks_maps = (1); # uncomment to DISABLE anti-spam code $final_virus_destiny = D_PASS; # (defaults to D_DISCARD) $final_banned_destiny = D_PASS; # (defaults to D_BOUNCE) $final_spam_destiny = D_PASS; # (defaults to D_BOUNCE) $final_bad_header_destiny = D_PASS; # (defaults to D_PASS), D_BOUNCE suggested

After Amavis is done with it, and passes it back to Postfix, Postfix then hands off the email to Procmail for delivery. This is where it gets passed through Spamassassin.

/etc/postfix/main.cf:

mailbox_command = /usr/bin/procmail
home_mailbox = .maildir/

(Set ".maildir/" to something you'd like in your setup. I wanted it to be hidden when I was working on that server, hence the dot.)

~/.procmailrc:

MAILDIR=$HOME/.maildir
#LOGFILE=$HOME/procmail.log
#VERBOSE=yes

:0 fw
|/usr/bin/spamc

:0:
* ^Subject: SPAM ([0-9]*)
.Spam.Caught/

I first set system-wide defaults for Spamassassin in the usual place. Note, significantly, that I can leave out SA's blackhole checks because I get my mail bounced to my server from a trusted source. There's no sense in checking the sending IP address against a list of possible bad guys.

/etc/mail/spamassassin/local.cf:

rewrite_header subject SPAM (_SCORE(0)_):
use_bayes 1
skip_rbl_checks 1
use_razor2 1
use_dcc 1
use_pyzor 1
bayes_auto_learn 1
report_safe 0

Then I can change the maximum “score” to count something as spam in my local user's account, but leave it at the default for my wife. (And, I suppose, eventually for my kids...)

~/.spamassassin/user_prefs:

required_score 3.5

On the pickup side, things are easy and simple inside the firewall. I just use Courier IMAP with my standard OS accounts. This didn't need any configuring; the defaults work fine. For pickup outside of the firewall, you have to use a second daemon running IMAP over SSL. This needed one tweak for me.

/etc/courier-imap/imap-ssl:

IMAPDSSLSTART=1

To train my Spamassassin filter, I collect uncaught and misidentified spam messages in a folder until I'm ready, and then run a script like this:

#!/bin/bash
echo "Learning about uncaught spam..."
sa-learn --spam --showdots /home/david/.maildir/.Spam.Uncaught/cur/
echo "Learning about misidentified spam..."
sa-learn --ham --showdots /home/david/.maildir/.Spam.Misidentified/cur/
echo "Cleaning misidentified spam..."
cd /home/david/.maildir/.Spam.Misidentified/cur && for i in *; do spamassassin -d < "$i" > "../../.Spam.Clean/cur/$i"; done && cd
echo "Learning about normal ham..."
sa-learn --ham --showdots /home/david/.maildir/cur/

I used to have instructions here about configuring the system-wide /etc/procmailrc file with the filter for spamc. However, in the many months I've been running like this, I came to realize that I wasn't getting very good learning of my spam. When I finally checked it all out, I discovered that I wasn't taking advantage of training my filters at all! When you configure Postfix to deliver mail via procmail, and set the filter to spamc in the system config file, spamd gets invoked as root, and never sees your personal Spamassassin Bayes database, no matter how well trained it is. In my defence, the docs for spamassassin repeatly tell you that you should run as root so that you can drop privileges to the individual user. However, at this point in the chain of processes, that makes it seem -- to Spamassassin -- that you're filtering mail for root. Anyway, the flip side of this is that I now need to create a .procmailrc file with the call to the spamc filter for every user on the system. The advantage is that I can still configure per-user treatment of spam. I toyed with moving all spam-handling into Amavis, but per-user configuration there would have been more difficult. Different strokes for different folks!