STARTTLS Explained: Opportunistic Encryption for Email in Transit

If you've ever dug into email server logs or scrolled through the output of an openssl s_client command, you've probably seen the word STARTTLS. It appears everywhere in email infrastructure documentation, but the name is genuinely confusing. It sounds like it starts TLS immediately — it doesn't. STARTTLS is actually an upgrade command that tells a mail server to convert an already-open plaintext connection into an encrypted one.

Understanding how STARTTLS works — and where it falls short — is important for anyone building or auditing email infrastructure. Encryption in transit isn't optional anymore. Mailbox providers actively factor TLS support into how they evaluate inbound connections.

What STARTTLS Actually Does

When two mail servers connect, they start by exchanging pleasantries over an unencrypted channel on port 25. The sending server greets the receiving server, which responds with a list of capabilities it supports. One of those capabilities is STARTTLS.

The sending server then issues the STARTTLS command. If the receiving server agrees, both sides perform a TLS handshake and the rest of the session — including the email content — travels over an encrypted channel.

The critical distinction: STARTTLS is opportunistic by default. That means if the receiving server doesn't offer it, or if the negotiation fails, most mail servers will fall back to sending the message in plaintext. No error. No bounce. The email just goes out unencrypted.

STARTTLS vs Implicit TLS (Port 465)

There are two ways email connections can use TLS:

• STARTTLS (port 587 or 25): The connection starts plaintext and upgrades to encrypted mid-session.
• Implicit TLS (port 465): The TLS handshake happens immediately — before any SMTP commands are exchanged. There's no plaintext phase at all.

For client-to-server connections (your email app sending mail), implicit TLS on port 465 is the stronger option. The session is encrypted from byte one. If you're configuring a mail client or an application, prefer port 465 with SSL/TLS rather than port 587 with STARTTLS, though both are acceptable when configured correctly.

For server-to-server connections (MTA-to-MTA delivery), port 25 with STARTTLS is standard. You don't get to choose the port when receiving mail from the internet — other servers will connect on port 25.

If you want a deeper look at how the ports compare, the SMTP ports guide breaks down each one in detail.

The STARTTLS Stripping Attack

Because STARTTLS is opportunistic, it's vulnerable to a downgrade attack. An attacker positioned between two mail servers — on a compromised network or a rogue router — can strip the STARTTLS capability advertisement from the receiving server's response. The sending server never sees the STARTTLS offer, assumes the receiving server doesn't support it, and sends the email in plaintext.

This isn't theoretical. It has been observed in the wild, particularly on networks where traffic passes through infrastructure that can manipulate TCP streams.

The fix is enforcement: telling sending mail servers that your domain requires TLS and that they should refuse delivery if a valid certificate can't be verified. That's what MTA-STS does.

MTA-STS: Enforcing What STARTTLS Can't

MTA-STS (Mail Transfer Agent Strict Transport Security) is a standard that lets domain owners publish a policy declaring that their mail servers require TLS and that senders should authenticate the certificate. When a sending server fetches your MTA-STS policy and finds mode: enforce, it will not deliver mail if it can't establish a valid, verified TLS connection.

A companion standard, TLS-RPT, lets you receive reports when TLS negotiation fails for your domain — giving you visibility into connection attempts that fell back to plaintext or failed entirely.

Together, MTA-STS and TLS-RPT close the gap that STARTTLS alone leaves open. The article on enforcing encrypted email delivery with MTA-STS and TLS-RPT walks through the full setup.

How to Verify STARTTLS Is Working

You can test STARTTLS manually from the command line using openssl:

openssl s_client -starttls smtp -connect mail.yourdomain.com:587

If STARTTLS is working, you'll see the TLS handshake complete and the server's certificate details. If it fails, you'll get an error — which is far more useful than silently falling back to plaintext.

For a quick check without the command line, tools like MXToolbox's SMTP Diagnostics and mail-tester.com can confirm whether your outbound server is advertising and completing STARTTLS.

Also check your mail server's logs. A successful STARTTLS upgrade will show something like TLS connection established in the log output. If you're seeing plaintext delivery to major providers, that's a configuration issue worth investigating.

STARTTLS and Your Certificate

STARTTLS is only as good as the certificate behind it. A self-signed certificate will cause strict receivers (and MTA-STS-enforced domains) to reject the connection. You need a valid certificate from a trusted CA, issued for the hostname your mail server announces in its EHLO greeting.

Common problems:

• Certificate hostname doesn't match the EHLO name
• Certificate has expired
• Intermediate certificate chain is incomplete
• Certificate is self-signed

If you're using MailDog's SMTP relay, certificate management is handled for you — your connections to MailDog's infrastructure use valid, trusted TLS automatically. The documentation covers how to configure your application to connect securely.

What STARTTLS Does Not Protect

STARTTLS encrypts the connection between mail servers. It does not encrypt the email at rest on either server. Once the message is delivered and stored, it sits in the recipient's mailbox unencrypted — unless end-to-end encryption like S/MIME or PGP is used.

TLS in transit is table stakes — necessary but not sufficient for high-security email. For environments where end-to-end confidentiality matters, layered encryption is required.

The Bottom Line

STARTTLS is the mechanism that makes encrypted email delivery possible across the internet, but its opportunistic nature means it's not a complete security solution on its own. Pair it with MTA-STS enforcement, a valid certificate, and TLS-RPT reporting, and you have a solid foundation. Skip those layers, and you're relying on hope rather than policy.

If you're evaluating your email infrastructure's security posture, the DNS security tools and SMTP relay security checklist are good places to continue.