Diagnostics of mail servers plays a significant part in ensuring the correct email operation. It allows to identify and resolve problems related to sending, receiving, and processing email messages. One of the key factors of diagnostics is the use of filters on mail servers, which provide protection against spam, viruses and other security threats.

This article will consider both external services that help checking the mail server operation from the sidelines, as well as internal tools that allow to run diagnostics directly on the server. All actions, as an example, will be performed on a private server powered by Ubuntu 20.04.6 OS with a configured solution in the form of Postfix and Dovecot, however, the presented methods are applicable to almost any operating system and mail clients.

Online services for mail server checking

The first and most important step is verification using exterior online services. This way, you can check mail server for exposures, SPF, DKIM and DMARC settings, and also check the reputation of IP addresses from which email is sent. In most cases, using only these tools may lead to desired results.

Let's consifer in detail the main verification services and their functionality:

MxToolBox allows us to check almost all available mail servers parameters. The service provides tools for checking all domain records, general domain availability, checking SSL certificates, IP address status and much more. To diagnose, you need to go to the service page, enter the IP address or domain, select the required tool and run the check

A detailed list of all available tools is shown in the screenshot:

As seen above, the website also checks for the presence of an IP/domain in blacklists. All major sources are checked: Spamhaus, Barracuda, SURBL and many others. If the address is in any database, the service will provide information about the reasons for being blacklisted. This way, you will be able to take measures to correct the problem.

MailTester is the second most popular tool for checking mail server. This service allows users to send a test email to a unique address, and then receive a detailed report on the quality of the sent message. Unlike MxToolBox, this resource is more focused not on diagnosing the mail server, but on possibly email improving. However, this does not mean that the service is not capable of conducting a full analysis of the sender’s server. It tests mail delivery, conducts research and gives recommendations for improving the mail service as a whole.

In order to complete the check, just go to the resource’s website and get a unique email address to which you want to send a email. After submitting, you must select “check assessment” and wait for the page to refresh. A good result looks like this:

In case of any problems or recommendations, the service will report this in the appropriate section of the report.

The above-mentioned services are sufficient for full check of the mail server from the sidelines. They allow you to identify potential problems with email delivery, check your security settings, and also receive recommendations for resolving possible errors. Next, we will perform testing on the server side.

Mail server settings check

Checking DNS records

One of the most common problems on the mail server side is incorrect DNS records configuration. You can check their correctness via the aforecited third-party services. However, in some cases, the email may simply not be delivered to a third-party mailbox. In this case, you should manually check all entries. To do this, go to the DNS editor and launch checking. As an example, let's take the following as the source data: domain profit.com, IP address 11.22.33.44, where @ can be used as the domain name, if the registrar does not allow filling in this form. Don't forget to change the values to your own. Don't forget to change the values to your own.

A records define the IP addresses of mail servers. Simply said, they direct the domain to the mail server address. It should look like this:

MX records are the most important for a mail server, they are responsible for the mail delivery route. In other words, they direct the mail to a mailbox.

SPF records point to servers that are able to send emails from a specific domain. Please note: they are published as TXT. Only one of the possible values is indicated. 

A DKIM record is used to verify the authenticity of an email. Must contain the generated public key. Likewise, only one of the value options is indicated.

DMARC recording is the next and final protection stage. It applies to emails that have not passed SPF and DKIM checks.

Also, one of the most important records for a mail server is the PTR record. It can be said that it works inversely to the A record, that is, it connects an IP address with a domain. This type of record can only be added through a request to the technical support of the hosting provider, and can be checked with the command: nslookup IP, where IP is your server address. The response should display the linked domain.

Port checking

Problems with closed ports can also lead to a complete mail server malfunction. First of all, it is necessary to clarify with the provider whether there are any restrictions on the necessary ports on its side. We do not limit clients to the number of open ports on any dedicated or virtual servers. However, many hosting service providers act differently.

You can check the open ports using the pre-installed netstat utility. It is enough to enter the command:

netstat – nat

With the SMTP/IMAP/POP3 server installed and configured, we will see the corresponding open ports:

As we may see on the screenshot, the main mail client ports are open, namely: SMTP (25) / IMAPS (143, 993) / POP3S (110, 995). You can also check the mail service on each port individually. The telnet tool will help you with this. Let's look at the example of POP3, that is, port 110:

telnet mail.yourdomain.com 110

We exit the tool with the exit command and check the remaining necessary ports according to the same principle. Note that this tool is also gives an opportunity to test sending emails without using an additional interface. This can be useful in cases where the user just needs to set up the server and plans to send mails locally.

In the event of difficulties when opening ports, you need to pay attention to the installed Firewall. Most distributions come with Iptables/Firewalld pre-installed.

For iptables we use the following commands:

iptables -A INPUT -p tcp --dport 25 -j ACCEPT
iptables -A INPUT -p tcp --dport 110 -j ACCEPT
iptables -A INPUT -p tcp --dport 995 -j ACCEPT
iptables -A INPUT -p tcp --dport 143 -j ACCEPT
iptables -A INPUT -p tcp --dport 993 -j ACCEPT

For firewalld:

firewall-cmd --permanent --add-port=25/tcp
firewall-cmd --permanent --add-port=110/tcp
firewall-cmd --permanent --add-port=995/tcp
firewall-cmd --permanent --add-port=143/tcp
firewall-cmd --permanent --add-port=993/tcp

In some cases, the SMTP service also requires ports 465 and 587 to be opened for correct operation. Opening these ports occurs in the same way. Restart the Firewall service after adding new rules.

Checking SSL/TLS certificates of the mail server

SSL/TLS certificates are used on the mail server for ensuring secure data transfer between the mail client and the server, and also confirm the authenticity of the sender's mail server itself, eliminating the possibility of man-in-the-middle attacks. However, they may also cause problems with sending or receiving mails. To launch diagnostics, you need to find out whether certificates are installed on the server’s side. Let's check the certificate presence with the following command:

openssl s_client -showcerts -server mail.profit.com -connect IP:port

In this command, you need to replace the values with your own: "mail.profit.com" is the address of the mail server; IP:port is the server data. By way of example, let's check port 993, which belongs to the IMAP protocol. Other protocols are checked in the same manner.

In response, the server must send the certificate data:

Most mail clients install certificates automatically. However, in some cases, it is necessary to manually release and add them to a specific client. different platforms may demand individual installation and configuration, so we recommend you to refer to the instructions for a specific solution. We recommend using Let’s Encrypt as a certificate, and using Certbot as an auxiliary tool for its installation and configuration.

Logs check

Finally, we have come to a solution that helps in most if any cases - checking the mail server logs. The majority of users delude themselves, thinking about the absence of their problem’s solution. However, modern mail clients store a large amount of information in logs:

1. Date and time of sending and receiving mails.
2. Address of the sender and recipient.
3. Filtering results for spam and viruses.
4. The state of the queue for sending and receiving mails.
5. Actions of administrators and users related to the mail server (for example, creating, deleting mailboxes, changing settings).
6. Errors and problems that arose during the processing and delivery of emails.

The standard directory in which logs of most commands are stored is located at: /var/log/. Depending on the solution used, the name of the specific log file may vary. For example, the mail.log file is located in the same directory for Postfix. We not recommend you to neglect this diagnostic method and use the logs as soon as the first signs of poor mail client performance appear. Please read our manual how to read Linux logs.

Understanding how to check mail server

We conducted a detailed diagnosis of the mail server. During testing, both online tools for detailed research and local solutions for detecting possible problems were affected. After conducting all the necessary tests, a complete picture of the current state of the mail server was obtained, as well as recommendations for eliminating potential difficulties.