Merge branch 'master' into terminal

content/blog/mail_server_alpine_postfix_dovecot_tutorial/index.md

@@ -94,10 +94,18 @@ The mail server will be composed of the following software:
         <td>Mail delivery agent</td>
         <td colspan="2">Dovecot</td>
     </tr>
+    <tr>
+        <td>SPF authentication</td>
+        <td colspan="2">postfix-policyd-spf-perl</td>
+    </tr>
     <tr>
         <td>DKIM authentication and signing</td>
         <td colspan="2">OpenDKIM</td>
     </tr>
+    <tr>
+        <td>DMARC authentication</td>
+        <td colspan="2">OpenDMARC</td>
+    </tr>
     <tr>
         <td>Spam filter</td>
         <td rowspan="2">Amavis</td>
@@ -216,6 +224,7 @@ following TCP ports are open on your firewall:
 | 465  | Email message submission over TLS |
 | 587  | Email message submission          |
 | 993  | IMAPS (IMAP over TLS)             |
+| 4190 | ManageSieve                       |
 
 ## Obtain a TLS certificate
 
@@ -801,6 +810,8 @@ checks.
 
 ## Sender Policy Framework
 
+### Set up your DNS record
+
 Add a TXT record for your root domain with the contents `v=spf1 mx ~all`, like:
 
 ```dns
@@ -863,6 +874,68 @@ Breaking down the TXT data:
     </tr>
 </table>
 
+### Get Postfix to validate SPF
+
+We're going to use a Postfix SMTPd policy server called postfix-policyd-spf-perl to check SPF of incoming emails.
+postfix-policyd-spf-perl is very simple and requires almost no configuration.
+
+Install `postfix-policyd-spf-perl` and create a user, `policyd-spf` for it:
+
+    # apk add postfix-policyd-spf-perl
+    # adduser -S -s /sbin/nologin -h /dev/null -H policyd-spf
+
+Explanation of `adduser` flags:
+
+<table>
+    <tr>
+        <th>Option</th>
+        <th>Explanation</th>
+    </tr>
+    <tr>
+        <td><code>-S</code></td>
+        <td>Create a system user</td>
+    </tr>
+    <tr>
+        <td><code>-s /sbin/nologin</code></td>
+        <td>Set shell to <code>/sbin/nologin</code> so the user doesn't have a shell</td>
+    </tr>
+    <tr>
+        <td><code>-h /dev/null</code></td>
+        <td>Set home directory to <code>/dev/null</code></td>
+    </tr>
+    <tr>
+        <td><code>-H</code></td>
+        <td>
+            Don't create a home directory (if you try to create <code>/dev/null</code> and assign it to
+            <code>policyd-spf</code> there will be all sorts of permissions issues)
+        </td>
+    </tr>
+</table>
+
+Now edit `/etc/postfix/master.cf` to tell Postfix to start up the postfix-policyd-spf-perl daemon:
+
+```conf
+policyd-spf   unix -     n       n       -       0       spawn
+  user=policyd-spf argv=/usr/bin/postfix-policyd-spf-perl
+```
+
+Now get Postfix to use postfix-policyd-spf-perl in `/etc/postfix/main.cf` by adding the following lines:
+
+```conf
+smtpd_recipient_restrictions =
+    permit_mynetworks,
+    reject_unauth_destination,
+    check_policy_service unix:private/policyd-spf
+policyd-spf_time_limit = 3600
+```
+
+postfix-policyd-spf-perl is now set up, and you can test it by sending yourself an email from a mainstream email
+provider (which ought to have an SPF record) and checking for the presence of this header:
+
+```
+Received-SPF: pass (protonmail.com: Sender is authorized to use 'revsuine@protonmail.com' in 'mfrom' identity (mechanism 'include:_spf.protonmail.ch' matched))
+```
+
 ## DomainKeys Identified Mail
 
 ### Configure OpenDKIM
@@ -1065,6 +1138,11 @@ non_smtpd_milters = $smtpd_milters
 This uses the Milter extension, which is something that can be used to process mail; in this case, to add headers to
 emails relating to DKIM.
 
+You can, again, test this on both incoming and outgoing mail. On outgoing mail, there should be a `DKIM-Signature:`
+header present. On incoming mail from domains implementing DKIM, there should be a
+`Authentication-Results: master.revsuine.xyz;` header (obviously replacing `master.revsuine.xyz` with your hostname)
+indicating whether or not the email has passed DKIM authentication.
+
 ## Domain-based Message Authentication, Reporting, and Conformance
 
 ### Ensure your domains are aligned in email headers
@@ -1197,6 +1275,129 @@ The `fo` tag indicates when you would like to receive reports. The options are:
     </tr>
 </table>
 
+### OpenDMARC
+
+We can use software called OpenDMARC to enforce DMARC policies for incoming mail. OpenDMARC is another milter. Let's
+install it and enable its service:
+
+    # apk add opendmarc
+    # rc-update add opendmarc
+    # rc-service opendmarc start
+
+Edit the OpenDMARC config at `/etc/opendmarc/opendmarc.conf`.
+
+Change
+
+```conf
+AuthservID HOSTNAME
+```
+
+to
+
+```conf
+AuthservID OpenDMARC
+```
+
+This is so that the `Authentication-Results` header from OpenDKIM authentication. This will also make it clear which
+program adds which `Authentication-Results` header.
+
+Add the following line, replacing `mail.domain.com` with your *hostname* (so in [my
+instance](#a-note-on-my-dns-records), this is `master.revsuine.xyz`).
+
+```conf
+TrustedAuthservIDs mail.domain.com
+```
+
+This specifies that OpenDMARC should trust authentication results from `mail.domain.com`. Otherwise you would get the
+following error message in your syslog:
+
+    ignoring Authentication-Results at 1 from mail.domain.com
+
+Enable `RejectFailures`, which means your server will comply with `p=reject` in DMARC DNS records.
+
+```conf
+RejectFailures true
+```
+
+You also probably want to enable `RequiredHeaders`, which rejects emails that don't conform to RFC5322 standards, e.g.
+are missing a `From:` header.
+
+```conf
+RequiredHeaders true
+```
+
+In case external SPF validation fails (as in, no SPF results are placed in the message header), you probably want to
+add
+
+```conf
+SPFSelfValidate true
+```
+
+which tells OpenDMARC to perform the SPF check itself if it can't find SPF results in the message header.
+
+Now provide OpenDMARC with a socket to use for communication with sendmail. We will use a TCP socket on port 8893:
+
+```conf
+Socket inet:8893@localhost
+```
+
+For a Unix socket, you'd use the following format:
+
+```conf
+Socket local:/var/run/opendmarc/opendmarc.sock
+```
+
+By default, you will have the line
+
+```conf
+IgnoreHosts /etc/opendmarc/ignore.hosts
+```
+
+in `/etc/opendmarc/opendmarc.conf`. This tells OpenDMARC to not authenticate the list of hosts in
+`/etc/opendmarc/ignore.hosts`. An example `ignore.hosts` is
+
+    127.0.0.1
+    93.113.25.226
+
+Keep in mind that if you have specified `IgnoreHosts`, this file needs to exist in order for OpenDMARC to run. If you
+have the option set, make sure to `touch /etc/opendmarc/ignore.hosts` (or whatever filepath you've specified).
+Alternatively, comment out this option in order to use the default, which is to not authenticate mail coming from
+127.0.0.1.
+
+Restart OpenDMARC for these changes to take effect:
+
+    # rc-service opendmarc restart
+
+To have Postfix use the OpenDMARC milter, it's simple as adding the socket to the `smptd_milters` and
+`non_smtpd_milters` variable in `/etc/postfix/main.cf`:
+
+```conf
+milter_default_action = accept
+milter_protocol = 6
+smtpd_milters = inet:127.0.0.1:8891,inet:127.0.0.1:8893
+non_smtpd_milters = $smtpd_milters
+```
+
+Restart Postfix for the changes to take effect:
+
+    # rc-service postfix restart
+
+And when you receive emails from a legitimate source that implements DMARC, you should see the following headers in
+your emails:
+
+```
+Received-SPF: pass (protonmail.com: Sender is authorized to use 'revsuine@protonmail.com' in 'mfrom' identity (mechanism 'include:_spf.protonmail.ch' matched)) receiver=master.revsuine.xyz; identity=mailfrom; envelope-from="revsuine@protonmail.com"; helo=mail-40130.protonmail.ch; client-ip=185.70.40.130
+DMARC-Filter: OpenDMARC Filter v1.4.2 master.revsuine.xyz 88CFF1288D1
+Authentication-Results: OpenDMARC; dmarc=pass (p=quarantine dis=none) header.from=protonmail.com
+Authentication-Results: OpenDMARC; spf=pass smtp.mailfrom=protonmail.com
+Authentication-Results: master.revsuine.xyz;
+        dkim=pass (2048-bit key; secure) header.d=protonmail.com header.i=@protonmail.com header.a=rsa-sha256 header.s=protonmail3 header.b=nc4YWVM/
+```
+
+<!--
+    TODO: switch SPF filter to a milter e.g. https://www.acme.com/software/spfmilter/ so that SPF isn't checked twice
+-->
+
 ### Test SPF, DKIM, and DMARC
 
 You can use [mail-tester.com](https://www.mail-tester.com/) and send an email from your domain to check that SPF, DKIM,
@@ -1283,6 +1484,9 @@ smtp-amavis   unix -     -       n       -       2       smtp
   -o smtpd_client_connection_count_limit=0
   -o smtpd_client_connection_rate_limit=0
   -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_address_mappings
+  # avoid double dkim signing by setting smtpd_milters to empty
+  # otherwise will run all milters again after amavis
+  -o smtpd_milters=
 ```
 
 The first block tells Postfix to send emails to Amavis, and the second block tells Postfix to run an extra smtpd daemon
@@ -1425,6 +1629,136 @@ X-Spam-Status: Yes, score=999.802 tagged_above=2 required=6.2
  URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
 ```
 
+# Pigeonhole
+
+Dovecot can do server-side mail filtering with sieve scripts. These are user scripts that can perform actions on mail
+based on particular criteria, e.g.
+
+```sieve
+require "fileinto";
+
+if address :is "to" "postmaster@revsuine.xyz" {
+    fileinto "Postmaster";
+}
+```
+
+Places mail in the `Postmaster` folder if the `To:` field is `postmaster@revsuine.xyz`. You also can do things
+unconditionally, like
+
+```sieve
+redirect postmaster@revsuine.xyz;
+```
+
+unconditionally redirects all mail to `postmaster@revsuine.xyz`.
+
+Sieve scripts can be both per-user and system-wide.
+
+For more examples, [this page](https://doc.dovecot.org/main/howto/sieve.html) has some good examples.
+
+## Installing and setting up Pigeonhole
+
+To use Sieve, install `dovecot-pigeonhole-plugin`:
+
+    # apk add dovecot-pigeonhole-plugin
+
+Then edit `/etc/dovecot/conf.d/20-lmtp.conf`, and add the `sieve` plugin like so:
+
+```conf
+protocol lmtp {
+  # Space separated list of plugins to load (default is global mail_plugins).
+  mail_plugins = $mail_plugins sieve
+}
+```
+
+To configure Pigeonhole and sieve, edit `/etc/dovecot/conf.d/90-sieve.conf`. Sieve's options will be configured in the
+`plugin {}` block in this file.
+
+We can set the location of user sieve scripts with the `sieve` option.
+
+```conf
+sieve = file:~/sieve;active=~/.dovecot.sieve
+```
+
+means that `~/sieve` is a directory of sieve scripts, whilst `~/.dovecot.sieve` is a symlink to the "active" one, e.g.
+
+```
+sieve
+├── script1.sieve
+├── script2.sieve
+└── script3.sieve
+```
+
+could be your `~/sieve/` directory, and to make `script2.sieve` active, you would do
+
+    $ ln -s ~/sieve/script2.sieve ~/.dovecot.sieve
+
+`sieve_before` defines a directory of sieve scripts which will be executed *prior* to any user scripts. e.g.
+
+```conf
+sieve_before = /etc/dovecot/sieve
+```
+
+means that the sieve scripts in `/etc/dovecot/sieve` will be executed first, then the user's personal scripts at
+`~/.dovecot.sieve`.
+
+You can specify multiple directories in order, like so:
+
+```conf
+sieve_before = /var/lib/dovecot/sieve.d/
+sieve_before2 = ldap:/etc/sieve-ldap.conf;name=ldap-domain
+sieve_before3 = /etc/dovecot/sieve
+```
+
+etc. The `sieve_after` option also exists, and works the same way.
+
+This is not the same as `sieve_default`, which is *overridden* by user sieve scripts and only executes when a user has
+no sieve script.
+
+## ManageSieve
+
+Users can configure their own user sieve scripts using a protocol called ManageSieve. Like how IMAP allows users to
+read their emails without having shell access to the mail server, ManageSieve allows users to write sieve scripts
+without requiring shell access.
+
+To enable ManageSieve, edit `/etc/dovecot/conf.d/20-managesieve.conf`. Make sure the following line is uncommented:
+
+```conf
+protocols = $protocols sieve
+```
+
+By default, ManageSieve will listen on port 4190.
+
+## Sieve scripts for spam filtering
+
+Let's use a system-wide sieve script to file SpamAssassin-marked spam into a Spam folder. Create an
+`/etc/dovecot/sieve/` directory, and add it to your `sieve_before` settings:
+
+```conf
+plugin {
+  ...
+  sieve_before = /etc/dovecot/sieve/
+  ...
+}
+```
+
+Now create a new sieve script, `/etc/dovecot/sieve/spam_folder.sieve`:
+
+```sieve
+require ["fileinto", "mailbox"];
+if header :contains "X-Spam-Flag" "YES" {
+  fileinto :create "Spam";
+}
+```
+
+Replace `"Spam"` with the name of your spam folder. If it's a subfolder, e.g.
+
+```
+Inbox
+└── Spam
+```
+
+you would write `"Inbox.Spam"` in that case.
+
 # Miscellaneous suggestions
 
 You may want to get your domain whitelisted on [dnswl.org](https://www.dnswl.org/), an email whitelist service where