Understanding SPF, DKIM, and DMARC: A Simple Guide

This is a great guide but from my experience, even if you configure it 100% correctly, email services like Gmail may still classify your emails as spam for no apparent reason while not being on any IP or domain blacklist. I tried for hundreds of hours to get around it with no avail, and my emails to Gmail always went to spam unless it was a response to an email from a Gmail address. Had to go back to a 3rd party hosted service (iCloud) because of it.

I have had a Gmail account from the days when it was invitation only. The inbox contains spam and my test emails and nothing else!

I've run tiny smtp systems for 25 years or so. It can be done. I am based in the UK but at least one of my domains is a .net jobbie, so nominally American. That one still works fine and it is my (ltd) company domain, so all good. The MX records etc have moved around a bit but always very carefully.

It all starts around the IP address you are using. Is it "tainted"? is it in a tainted block? If it is then you need to either go elsewhere or clean it up and that takes a bit of time. By clean it up I mean apply for removal from the usual suspect's blocklists - Spamcop (lol), Spamhaus and all the rest that you can find.

Now setup PTR records. That has to be done by your ISP. If they can't do it for you, then find a new ISP. If you can't get PTR records to match A records then you may have to give up. One of the first checks an anti spam system will do is reverse look up an incoming IP address and compare it. Also that should match the HELO/EHLO announced by the SMTP MTA:

SMTP connection from IP address 12.13.14.15 HELO (my name is) smtp.example.co.uk

Receiver will check: smtp.example.co.uk == 12.13.14.15 AND 15.14.13.12.in-addr.arpa == smtp.example.co.uk.

Everyone gets their knickers in a twist about SPF, DKIM and DMARC but if you do not get the prior basics of IP -> A -> HELO -> PTR sorted out first then you will fail sooner or later. I also recommend that you ensure your MX records (receiving) match up too with your sending records. It means you can use mx is SPF, for example.

If you have multiple internet connections and IPs then be absolutely certain that your inbound and outbound IPs for SMTP match up.

Sorted all that? Cool, now proceed to SPF.

Most people fail at the PTR stage. If your ISP will not do PTR for you then you are probably screwed for self hosted SMTP. If you cannot change ISP to one that will, then you are really screwed. Sorry. In that case you will have to engage a service that will route SMTP on your behalf. It won't cost much but you won't own it and you will have to pay someone to do it. Soz.

Came here to say this, plus add a little personal insight to the future of email.

I've run 5 or 6 different mail servers over the past 10 years. Originally before O365 I was an exchange admin, then postfix, iRed, mailcow, mail gun, you name it. Hosted on every cloud provider, even in our colo with part of a private /24 allocation with good reputation (built since 1997, gawdamn). Every sort of header combination, tls setup, and no blacklists. Always 100% alignment, including strict rejection policy (best results even over quarantine).

Does not matter, if you're sending from custom domain not handled through a big name, expect the spam box with Gmail. Yahoo and Outlook are fine, but Gmail is the bane.

I've spent maybe 100 hours of my own over this last year and know what I realized? Nobody cares about email anymore, except for automated account management stuff (login, PW reset). Businesses pay the $3 /mo / seat for fastmail and don't think twice.

But the current trend is toward social chat (discord or Whatsapp) and most the people who own an iPhone just use their apple ID email for everything.

Although I am a fervent supporter of open protocols and believe email (with pgp signing) is an awesome long form communication format... Face it, it's going the way of the fax machine.

Reverse pointer is pretty easy with some hosting (Linode) and painful with others, but that's pretty basic knowledge. Same with managing IP reputation. Heck, mail gun helps warm up IPs for you (but if you're not email marketing it's ridiculous to maintain that).

What really gobbles my bobble is BIMI. Even without the paid-for certificate ($1500 is absurd), you can set it up to show your logo, and works on some providers (like yahoo). But careful, you have BIMI without the cert set up? Gmail spam-cans it.

Same with pgp, if you include your signature a lot of providers will immediately increase it's spam rating, usually high enough to land in spam (+7 pts usually), even though I doubt any spammer or scammer is inviting you to encrypted chats.

Email is broken because we all signed up for Gmail and didn't know better at the time.

It's getting pretty expensive to rent one IPv4 address per domain these days. You also don't always control every address in a block, which means there may be nothing you can do about your reputation no matter where you go.

Perhaps it is a volume issue? You need steady and significant volume of emails to maintain reputation at gmail and friends.

Guilty until proven innocent, an excellent initial position.

I’ve had to relax my SPF record to include the entire mail pool of my ISP to be able to send to anything hosted by Microsoft. I tried to liaise with them directly, and through Linode, but they refused to exclude the IP from their opaque blocklist. Their proposed solution was to change the IP of the VPS, but that’s just agreeing to play whack-a-mole with a bad faith actor.

There should be a path to greater transparency and accountability from the SMTP cartels, but I’m at a loss as to how that can manifest.

>Spamcop (lol)

What is lol about Spamcop?

Email as a communication method with your friends/family — absolutely, this has been dead for over a decade I’d say.

However, email has basically evolved into the way you communicate with “systems” and I’m kind of happy about it. Communication with companies outside your network, e-commerce accounts/purchases, communication with government systems, schools, banking, airlines, concerts/events, restaurants, etc. Hell, even RSS is now basically in email — newsletters are growing fast as a medium, not shrinking.

You just book a hotel in Nairobi? It’ll be in your email. No other communication method even comes close for this use case.

Social/chat apps will never unseat this because they’re social. Like nightclubs, the trendy ones come and go. Come back when you’ve set up an interoperable network of virtually every person on earth. Then we’ll talk about email being dead.

> Email as a communication method with your friends/family — absolutely, this has been dead for over a decade I’d say.

Email is the primary way that I stay in touch with my extended family and friends. At least for us, it's very much alive.

You only need one IP per MTA not per domain. I have a "vanity" email system that I run at home. I run it for my mates too. I have around 10 domains inbound. It all works fine.

SMTP and SIP are often held aloft as fucked up. My Dad's home telephony runs off a RPi and a Yealink DECT station and a dynamic DNS.

The modern internet might look a bit fucked up if you only look at the X/Facebook/webby wankery stuff but the real internet is functioning quite happily.

Did you send out mails directly from your ISP? (Than I could unterstand Microsoft.) Only your MX must send out mails. And only it should be in the SPF record.

It would be crazy of Microsoft to look at all Received-Hearders and want everything mention there in the SPF record. If that is what really happens, than you should exclude this information from the header. (mask-src on Opensmtpd. Pretty sure Postfix has that option to, but haven't used it in a decade, so I can't tell you the syntax.)

I originally had only my MX hosts listed in my SPF record, and configured to accept them exclusively. Microsoft unilaterally blocked the entire /23 my VPS resides within, and refused to exclude my configured IPs from their block.

The only way I’m currently able to deliver anything to domains hosted by Microsoft is to expand my SPF record to include my ISP’s mail hosts, and route delivery through them.

Microsoft, and the other SMTP cartel thugs, are undermining the protections these protocols were designed to provide.