Reducing server load

In this article, we will delve into why increased server load occurs and discuss various ways to optimize high-load processes. Special attention will be given to code optimization in Apache/Nginx and MySQL, we will talk about caching as an auxiliary tool, and also consider possible external threats, such as DDOS attacks, and ways to prevent them.

Why Server Load Occurs

Before proceeding to server optimization, it is necessary to conduct a thorough analysis of the current load on resources. This includes measuring CPU load, RAM usage, network activity, and other key parameters. Understanding the dynamics and peak loads allows identifying bottlenecks and optimizing resource allocation, thus increasing the stability and performance of the server infrastructure.

For initial troubleshooting of high server load, we recommend conducting a general server diagnostics. If this is insufficient, a more detailed analysis of resources is necessary. As an auxiliary tool, exploring the logs of the Linux server can be helpful, as this is where the source of the problem is found in most cases.

Optimizing Apache/Nginx Server

Increased Server Load Due to Indexing

Increased load due to indexing on the server can occur, for example, when search engines scan a large number of pages on your site. This can lead to increased use of server resources and, consequently, slow down the site's performance. Identifying the cause is relatively simple; you need to open the file located at:

/var/www/httpd-logs/sitename.access.log

When indexed by search engines, the user will see entries of the following nature:

11.22.33.44 - - [Date and Time] "GET /your-page-path HTTP/1.1" 200 1234 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

As a first solution to reduce the load, you can use the setting of meta tags "noindex" and "nofollow" on pages that do not need to be indexed. The second solution is the .htaccess file, where entries corresponding to specific search engines need to be added, for example, to hide from Yandex and Google:

SetEnvIfNoCase User-Agent "^Yandex" search_bot SetEnvIfNoCase User-Agent "^Googlebot" search_bot Order Allow,Deny Allow from all Deny from env=search_bot

Similarly, edits need to be made for other search engines. It should be noted that the capabilities of .htaccess are not limited to just blocking indexing. We recommend getting more familiar with its main features in the article.

Using Caching Settings

Incorrect caching settings on the server can also lead to high load. To optimize this parameter, corresponding changes need to be made in the configuration files or .htaccess. In the case of Apache, the latter option is preferable, for Nginx – the former.

On an Apache server, you need to open the .htacess file and insert the following code:

<FilesMatch ".(flv|gif|jpg|jpeg|png|ico|swf|js|css|pdf|doc|docx)$"> Header set Cache-Control "max-age=2592000" </FilesMatch>

Then, enable the Expires module using the command:

sudo a2enmod expires

After which, restart the web server:

sudo service apache2 restart

And activate the module by specifying:

ExpiresActive On

On a Nginx server, it is sufficient to add the following code to the configuration file:

location ~* .(jpg|jpeg|gif|png|ico|css|swf|flv|doc|docx)$ { root /var/www/yoursite.com; }

And perform a service reload:

sudo service nginx restart

Note that with these settings, the Allow and Deny directives will be bypassed.

Using Data Compression

Enabling data compression using Gzip on Apache and Nginx web servers helps reduce the amount of data transmitted between the server and the client, which improves performance and reduces web page loading time.

To enable Gzip on Apache, you need to activate the mod_deflate module:

sudo a2enmod deflate

Then, restart the web server:

sudo service apache2 restart

And finally, add the following block to the configuration file or .htaccess:

<IfModule mod_deflate.c> # Configure compression for specified file types AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript application/javascript application/x-javascript application/json # If the browser matches the specified pattern, apply compression only to text/html files BrowserMatch ^Mozilla/4 gzip-only-text/html # If the browser matches the specified version patterns of Mozilla 4.0.6, 4.0.7, 4.0.8, disable compression BrowserMatch ^Mozilla/4\.0[678] no-gzip # If the browser is MSIE (Internet Explorer), disable compression for all files except text/html BrowserMatch \bMSIE !no-gzip !gzip-only-text/html # If the request contains the specified pattern (extensions of image files), disable compression SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip </IfModule>

This configuration enables compression for certain types of files and disables it for images.

In the case of Nginx, configuration occurs in the http block of the configuration file. The following code needs to be added:

gzip on; gzip_disable "msie6"; # Adds the Vary header, indicating that the response may change depending on the Accept-Encoding header value gzip_vary on; # Enables compression for any proxy servers gzip_proxied any; # Sets the compression level. A value of 6 provides a good balance between compression efficiency and resource use gzip_comp_level 6; # Sets the size of the buffer for compressed data (16 buffers of 8 kilobytes each) gzip_buffers 16 8k; # Specifies that data compression should be used only for HTTP version 1.1 and higher gzip_http_version 1.1; # Sets the file types that can be compressed gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

Similar to Apache, here the compression parameters for certain types of files are set. After making changes to any of the web servers, a service reload is required:

sudo service apache2 restart

Or

sudo service nginx restart

DDOS Attack on the Server

High server load can occur as a result of a DDoS attack. Identifying the presence of a DDoS attack can be done through monitoring a sudden increase in traffic, abnormal requests, and server performance drops. Reviewing logs for repeated requests from one IP address or port scanning can also indicate a possible DDoS attack. There are many protection measures, but we will only discuss the basics.

Using a CDN (Content Delivery Network). A CDN can serve as an intermediary between your web server and users, distributing traffic and caching content to mitigate the impact of a DDoS attack. CDNs can also have built-in DDoS protection mechanisms, including load distribution and traffic filtering.

Configuring firewalls and intrusion detection systems (IDS/IPS). Firewalls can be configured to filter traffic based on various criteria, such as IP addresses and ports. IDS/IPS can detect abnormal traffic behavior and block suspicious connections. These tools can be effective in tracking and blocking potentially malicious traffic.

Configuring Apache and Nginx web servers to mitigate the impact of DDoS attacks.

As a solution for Apache, we enable the mod_evasive module. To do this, uncomment or add the following line in the httpd.conf or apache2.conf configuration file:

LoadModule evasive20_module modules/mod_evasive.so

In the same file, you need to add a settings block:

<IfModule mod_evasive20.c> # Hash table size for storing request information DOSHashTableSize 3097 # Number of requests to one page before activating protection DOSPageCount 2 DOSPageInterval 1 # Number of requests to all pages before activating protection DOSSiteCount 50 DOSSiteInterval 1 # Blocking period in seconds for IP addresses DOSBlockingPeriod 10 </IfModule>

Similarly, we activate the mod_ratelimit module:

LoadModule ratelimit_module modules/mod_ratelimit.so

And add the configuration:

<IfModule mod_ratelimit.c> # Setting the output filter for rate limiting (Rate Limit) SetOutputFilter RATE_LIMIT # Beginning of the settings block for the location "/login" <Location "/login"> # Setting the environment variable rate-limit with a value of 1 SetEnv rate-limit 1 # Ending of the settings block for the location "/login" </Location> </IfModule>

The configuration for Nginx is similar to Apache. In the nginx.conf configuration file, the following directives need to be utilized:

http { ... # Defining a zone for connection limits limit_conn_zone $binary_remote_addr zone=addr:10m; # Defining a zone for request limits limit_req_zone $binary_remote_addr zone=req_zone:10m rate=1r/s; server { ... # Configuring connection limits limit_conn addr 10; # Configuring request limits limit_req zone=req_zone burst=5; ... } }

After making changes to each of the services, they need to be reloaded:

sudo systemctl restart apache2

Or:

sudo systemctl restart nginx

These examples provide only a basic configuration, which can be further adapted depending on specific requirements and the nature of attacks.

Optimizing MySQL Queries

Optimizing MySQL database queries on a web server can be achieved in various ways, and one of them is the proper configuration of the configuration file. Typically, this file is named my.cnf or my.ini and is located in the /etc/ or /etc/mysql/ directory. You need to open it and make the following changes:

[mysqld] # Location of the file for recording slow queries. Be sure to replace it with your path log-slow-queries = /var/log/mariadb/slow_queries.log # Threshold time for considering slow queries (in seconds) long_query_time = 5 # Enabling recording of queries that do not use indexes log-queries-not-using-indexes = 1 # Disabling query caching query_cache_size = 0 query_cache_type = 0 query_cache_limit = 1M # Size of temporary tables tmp_table_size = 16M max_heap_table_size = 16M # Size of the thread cache thread_cache_size = 16 # Disabling name resolving skip-name-resolve = 1 # Size of the InnoDB buffer pool. Set to 50-70% of available RAM innodb_buffer_pool_size = 800M # Size of the InnoDB log file innodb_log_file_size = 200M

Let's also consider additional recommendations that can facilitate interaction with the server database:

1. Use the EXPLAIN command before an SQL query to analyze its execution. This allows you to get an execution plan for the query and determine which indexes are used, which tables are scanned, etc.
2. Indexes speed up data search, so properly designed indexes can significantly improve query performance. Pay attention to columns that are frequently used in WHERE or JOIN conditions.
3. Avoid using SELECT *. Specify only those columns that are truly necessary for your query, instead of selecting all columns in a table.
4. Avoid using functions in WHERE conditions. Using functions (such as LOWER, UPPER, LEFT, RIGHT) in WHERE conditions can make indexes useless. Try to avoid their direct use in conditions.
5. Use INNER JOIN where possible, as it is usually more efficient. Also, ensure that the corresponding columns for joining have indexes.
6. Use LIMIT to restrict the number of returned rows if you need to get only a certain number of results.
7. Consider caching query results, especially if they rarely change, to reduce server load.

The Mail Server Creates High Load on the Server

In this section, we will explore how to determine that the mail server is experiencing high load and what steps can be taken to optimize its operation, including checking the message queue and configuring server parameters. Start with checking the message queue. The mailq utility can help with this, to activate it, enter the corresponding command in the terminal:

mailq

This will display a list of messages in the queue, if any. Each message will be displayed with its unique identifier and information about the sending status. A similar result can be obtained by reviewing the mail client logs.

In most cases, high load occurs in the event of server compromise when it starts sending spam. However, if after checking the administrator is confident that the server has not been attacked from the outside and users are not neglecting spam, it's time to move on to optimizing the mail server. Here are the steps that will help:

1. Ensure that your domain's DNS records are correctly configured, including SPF, DKIM, and DMARC records to improve mail delivery and protect against spam. The correct configuration of parameters can be found in the article on mail server diagnostics.
2. Check network settings, including firewall configuration and routing rules, to avoid blocks and speed up mail delivery.
3. Configure message queue parameters according to server load. This may include setting the maximum queue size and timeouts.
4. Consider the solutions we discussed in this article earlier. Periodically optimize the mail server database to improve performance, use caching mechanisms to speed up data search and processing, such as DNS queries.
5. If the mail server still regularly encounters high load, consider scaling options, such as using a cluster of mail servers or cloud solutions.

Conclusion

Increased server load directly affects website loading speed, ultimately impacting user experience and reputation in search engines. Thus, effectively managing this load plays a key role in ensuring continuous functionality of the resource and increasing its accessibility for visitors.