MTA-STS and TLS-RPT: Enforcing Encrypted Email Delivery

The Gap That SPF, DKIM, and DMARC Don't Close

SPF, DKIM, and DMARC are well understood by most email administrators. They authenticate the sender, protect against spoofing, and give receiving servers a policy to apply when authentication fails. But they don't address one specific attack: an adversary who intercepts the SMTP connection between two mail servers and strips the TLS encryption before delivery takes place, exposing the message content in transit.

MTA-STS (Mail Transfer Agent Strict Transport Security) and TLS-RPT (TLS Reporting) were designed to close exactly this gap. Together, they let a domain declare that inbound SMTP connections must use TLS, and they give you a reporting mechanism to surface failures before they become delivery problems.

What MTA-STS Does

MTA-STS works by publishing a policy that tells sending mail servers two things:

1. You must connect using TLS when delivering to this domain
2. You must verify the TLS certificate — don't deliver if the certificate is invalid or untrusted

Without MTA-STS, SMTP connections are opportunistic: if TLS is available, it's used, but if a network attacker strips the TLS offer (a STARTTLS downgrade attack), the connection silently falls back to plaintext. The sending server has no way to know this was malicious, and delivery completes over an unencrypted channel.

With MTA-STS in enforce mode, the sending server refuses to deliver if it can't establish TLS with a valid, trusted certificate. The message doesn't get delivered — and it doesn't get intercepted either.

How MTA-STS Is Published

MTA-STS requires two components: a DNS TXT record and a policy file served over HTTPS.

The DNS Record

Published at _mta-sts.yourdomain.com:

_mta-sts.example.com.  IN  TXT  "v=STSv1; id=20240601000000"

The id value is a string that changes every time you update your policy. Sending servers cache your MTA-STS policy and use the id to check whether their cached copy is still current. Change the id whenever you modify the policy file.

The Policy File

Hosted at https://mta-sts.yourdomain.com/.well-known/mta-sts.txt:

version: STSv1
mode: enforce
mx: mail.example.com
mx: *.example.com
max_age: 86400

The key fields:

• mode: enforce rejects delivery if TLS validation fails; testing allows delivery but reports failures; none disables the policy
• mx: One or more authorized MX hostnames. Wildcards are permitted.
• max_age: How long sending servers should cache the policy, in seconds

Deploying MTA-STS Safely

Never start with mode: enforce. Always begin with testing. In testing mode, sending servers apply the policy logic but still deliver even if TLS validation fails — they simply report the failure. Run testing mode long enough to catch any legitimate TLS issues on your receiving infrastructure before those failures start blocking real mail.

Before switching to enforce, verify your MX server certificates:

• Issued by a publicly trusted certificate authority
• Not expired (set renewal reminders well before expiry)
• The certificate's Subject Alternative Name must match the MX hostname exactly

A certificate issue that surfaces in testing mode is just a warning. The same issue after you've switched to enforce mode causes silent delivery failures for senders that respect MTA-STS policy.

TLS-RPT: Visibility Into What's Happening

TLS Reporting (RFC 8460) is a companion standard that sends you structured data about TLS connection failures to your domain. It's published as a TXT record at _smtp._tls.yourdomain.com:

_smtp._tls.example.com.  IN  TXT  "v=TLSRPTv1; rua=mailto:tls-reports@example.com"

Supporting sending servers send daily JSON reports to your configured address, summarizing successful and failed TLS connections along with the failure type and message count. These reports are particularly valuable in testing mode — they show you which senders are running into TLS issues before you've committed to enforcement. After moving to enforce, they help you catch any ongoing TLS failures that might be quietly dropping legitimate mail.

MTA-STS vs DANE

DANE (DNS-Based Authentication of Named Entities) solves the same problem through a different mechanism: it publishes certificate fingerprints in DNSSEC-signed DNS records, allowing sending servers to validate TLS certificates independently of public certificate authorities. DANE is technically more robust but requires DNSSEC on your domain, which adds operational complexity that many organizations aren't ready for.

MTA-STS requires only HTTPS hosting and standard DNS — the same infrastructure most domains already have. The two standards aren't mutually exclusive, and some high-security deployments implement both, but MTA-STS is the practical starting point for most senders.

The Implementation Path

1. Verify your MX server certificates are valid, trusted, and match the MX hostnames
2. Set up HTTPS hosting for mta-sts.yourdomain.com — this needs a valid TLS certificate itself
3. Publish the policy file at the correct path in testing mode
4. Publish the DNS TXT record with an initial id value
5. Configure TLS-RPT and monitor incoming reports for two to four weeks
6. Resolve any TLS failures the reports surface
7. Update the policy file to mode: enforce and increment the id

MTA-STS fits naturally alongside the rest of your authentication stack. If you're configuring DMARC, DKIM, and SPF alongside MTA-STS, the MailDog DNS security guide covers all of these. The technical documentation includes details on how MailDog's SMTP infrastructure handles TLS on inbound connections. For questions about your specific DNS and certificate setup, reach out directly or explore more authentication articles on the MailDog blog.