SMTP Authentication Troubleshooting: Fixing AUTH Errors and Relay Rejections

SMTP authentication errors are one of the most common — and most frustrating — problems email administrators face. A misconfigured AUTH setting, a wrong port, or an outdated credential can silently stop outbound mail or produce cryptic error messages that don't immediately explain what went wrong. This guide walks through the most frequent SMTP authentication problems and exactly how to fix each one.

How SMTP Authentication Works

When your email client or application connects to an SMTP server to send mail, the server requires it to prove its identity before relaying messages. This is the AUTH step in the SMTP handshake. The client offers an authentication mechanism — most commonly PLAIN, LOGIN, or OAUTH2 — and provides credentials. If the credentials are accepted, the session continues and the message is sent. If not, the server rejects the connection with an error code.

Understanding this flow makes troubleshooting much faster. Most AUTH failures happen at one of three points: wrong credentials, wrong mechanism, or a port mismatch that prevents AUTH from being offered at all.

Common SMTP Authentication Errors and Their Fixes

535: Authentication Failed

The 535 error means your credentials were rejected. SMTP 5xx errors are permanent failures, so the sending system won't retry automatically. Common causes include:

• Incorrect username — for most SMTP servers, the username is the full email address, not just the local part before the @
• Wrong password — check for copy-paste issues, leading or trailing spaces, or an expired password
• Account requires an app-specific password — common when MFA is enabled on the mailbox
• Account locked after too many failed attempts

Fix: Verify credentials by logging into the webmail interface with the same username and password. If webmail works, the issue is with how credentials are being passed to the SMTP connection, not the credentials themselves.

530: Authentication Required

A 530 error means the server requires authentication but none was provided — or authentication was skipped. This often happens when a client connects on port 25 (reserved for server-to-server traffic) instead of the submission ports.

Fix: Switch to port 587 with STARTTLS or port 465 with SSL. Most outgoing mail from clients and applications should use one of these two ports, not port 25. Confirm that AUTH is enabled on the port you're connecting to.

454: Temporary Authentication Failure

A 454 error is a temporary failure — the server couldn't complete authentication right now but may succeed later. Unlike 535, it's not necessarily a credential problem. Common causes include:

• TLS negotiation failed before AUTH could be attempted
• Server-side rate limiting or throttling on failed attempts
• Temporary database or account lookup failure on the server

Fix: Verify that your TLS configuration is correct and that your SSL certificate is valid. If TLS is failing, AUTH won't be offered over a plain connection — which is by design for security reasons.

AUTH Not Advertised in EHLO Response

Sometimes a client connects successfully but the server doesn't include AUTH in its EHLO capability response at all. This almost always means the connection isn't encrypted yet. Most SMTP servers won't advertise AUTH mechanisms over a plain connection — they require STARTTLS first.

Fix: Enable STARTTLS in your client before authentication. Issue the EHLO command, then STARTTLS, and only after the encrypted session is established should AUTH appear in the response. If AUTH still doesn't appear after STARTTLS completes, it may be disabled in the server's configuration for the port you're using.

Testing SMTP Authentication From the Command Line

When a client or application is failing but the root cause isn't clear, testing the SMTP connection manually isolates the problem by removing client-side variables.

openssl s_client -connect smtp.example.com:587 -starttls smtp

After the TLS handshake completes, type:

EHLO yourdomain.com

Check the response for AUTH LOGIN PLAIN in the capability list. If AUTH isn't listed, something is preventing the server from offering it. If AUTH is present, proceed:

AUTH LOGIN

You'll be prompted for a base64-encoded username and password. This raw test bypasses client libraries entirely and confirms whether the issue is with the credentials, the server configuration, or just your client's handling of the connection.

Application-Specific Authentication Issues

Web Applications and CRMs

Applications that send email via SMTP — WordPress, Magento, Salesforce, ticketing systems — have their own SMTP credential fields that are separate from your email client settings. After changing an email password or rotating SMTP credentials, update every application. These configs are easy to miss, and a 535 error from a forgotten application often looks like a platform outage until you trace it back.

OAuth vs Password Authentication

If your email provider uses OAuth 2.0 — common with Microsoft 365 and Google Workspace — basic password authentication to SMTP may be disabled by policy. You'll need to generate an app password specifically for SMTP use, or configure your application to use OAuth. Attempting to authenticate with a regular password when OAuth is required produces a 535 error that looks like a wrong password even when the credentials are correct.

Relay-Specific Authentication

If you're using a dedicated SMTP relay rather than connecting to a mailbox server, authentication uses relay credentials rather than mailbox credentials. These are different things and are easy to confuse. The relay may use API keys, token-based authentication, or a dedicated username and password that has nothing to do with any individual email account.

Check the MailDog documentation for relay-specific authentication formats, and ensure credentials are entered exactly as provided — no extra spaces or line breaks. The SMTP relay dashboard surfaces active connection logs and authentication attempts, which makes it straightforward to distinguish between a credential problem and a network-level issue. For configuration patterns that help prevent authentication failures before they happen, the relay security checklist covers access controls and hardening steps that reduce the surface area for misconfiguration.