Email delivery headaches
Email delivery used to be easy. Server A connects to Server B over SMTP, states that it has a message for Bob from Alice, then sends the message text ("We meet at midnight"). This worked fine until spam came along; to cope with a deluge of spam, an extra step was added, namely "Server B decides whether it trusts Server A to provide email from Alice for Bob". Even then, it wasn't too bad; sure, wide swaths of IPv4 address space were blacklisted, but if you had a server at a reputable ISP, you would probably not be on any of those lists. Then cloud computing happened.With the launch of Amazon EC2 in August 2006, life suddenly became much easier for spammers: If they found that their IP address was blacklisted, all they had to do was pay another $0.10 to spin up an EC2 instance with a different IP address to keep sending spam. While Amazon started out by disabling spammers' accounts, this quickly proved infeasible; and before long, the entire EC2 address space was on "dynamic IP space" blacklists. When I launched Tarsnap this wasn't a problem for me since I sent all my outgoing email through a server at RootBSD; but a few years ago I decided to move everything to Amazon Web Services, for ease of management, and so I had to stop sending email out directly.
My first choice was to use Amazon's Simple Email Service — after all, it was designed specifically for the use case of "company with servers in EC2 needs to send email". Alas, SES couldn't satisfy my needs: For spam and scam prevention reasons, SES applies strict filters to outgoing mail. Not only are SMTP envelope sender addresses required to be on a pre-verified list, but message From: addresses also need to be pre-verified. For emails from Tarsnap to Tarsnap users this wouldn't be a problem — but Tarsnap also has several public mailing lists, and when Tarsnap users post to the lists, their email goes out (via Tarsnap's mail server) with their original From: header.
Having decided that I could not use Amazon SES, I turned to another email-delivery-as-a-service company, SendGrid. For two years I was generally satisfied with them; I occasionally had Tarsnap users report that gmail has marked messages as spam, but I chalked that up to Google's spam filters being overly aggressive and decided there wasn't much I could do about it. A couple days ago, however, a Tarsnap user reported to me that the DKIM headers being added by SendGrid were invalid — not good! — and worse, a significant proportion of email he received via SendGrid had been failing DKIM checks for months. While SendGrid replied via twitter to say that they were aware of the issue and working on a fix, this was enough to make me reconsider my email-sending strategy.
My new strategy is to combine Amazon SES and SendGrid. When outgoing email needs to be sent by qmail, it no longer goes straight to the standard qmail-remote binary; instead, I replaced that with a shell script which examines the sending address. If that address is in the list of addresses I have white listed with Amazon SES, a qmail-remote-ses binary is invoked to pass the message to Amazon; I trust Amazon to do a good job of making sure email gets delivered (since, after all, they have a lot of experience with email from their retail business), and this will take care of all the most important Tarsnap email. Messages going out with other addresses — mainly email from the public mailing lists — are passed instead to a qmail-remote-smtp binary (which is in fact the old qmail-remote binary) and make their way out via SMTP to SendGrid.
Will this be good enough? I certainly hope so; considering how essential email has been to the development of the internet, and how much it is still relied upon by other services — an email address is a de facto internet-wide user name — it seems absurd how far one needs to go to get email successfully delivered. And if I find it this much of a hassle... well, I'm sure I can't be the only person suffering this headache.