Content

The 1. page content

Introductory

Monit is a free and open source process and resource monitoring tool that can be used to monitor system processes and resources using a web browser and command line. Monit automatically stops or restarts processes or services if, for example, they use too many resources or behave abnormally. It also sends email alerts about various events.

In this write-up, we look at two scenarios:

• One in one Debian 10 (Buster) Perfect Server 1.1, we install the Monit system monitor program freshly on the version where it was not used before, and then set it up.
• In the other, we perform some fine-tuning on an existing Monit instance (installed and configured by the ISPConfig automated installer) on a perfect Debian 11 (Bullseye) server so that it works properly for us. To do this, Debian 11 (Bullseye) is the perfect server VirtualBox version Or the version installed on a live server can serve as a basis.
  So if this scenario suits us, in this case we can skip this section and continue with post-adjustments.

Installing Monit

This fresh Monit installation is therefore Debian 10 (Buster) perfect server 1.1 (VirtualBox), where there are already quite a few working services that can be monitored, and on this server - managed by ISPConfig - self-signed SSL is also available, so we can use this for Monit. And in the last section of the description, the one on the live server Let's Encrypt we will also use a valid SSL issued by Monit.

Updating packages

First update (rootas) our package repository database and packages:

apt-get update
apt-get upgrade

Installation and status query

Let's install Monit:

apt-get install monit

Status query:

systemctl status monit

Configuring Monit after fresh installation

Monit's main configuration file is the /etc/monit/monitrc. It contains many basic settings, some of which are active, and there are also commented settings that serve as samples. As with most programs, Monit also has configuration subdirectories from which individual configuration files can be read. therefore, instead of the main config file, it is recommended to place our own configurations in these directories - leaving the original config file intact - in which the default settings can also be overridden. These libraries to be scanned are a /etc/monit/nonitrc are defined at the very end of the file in the include sections.

Overview of configuration libraries

Monit has 4 configuration subdirectories, their operation and use are as follows:

• /etc/monit/conf-available/: Library of available settings. This directory contains the settings of the services and resources that Monit wants to monitor, and new ones are also created here. Monit does not read this directory directly, so what is in it will only be active after linking to the enabled directory. So it's kind of a backup settings directory like that Apache also in the case of conf-available, mods-available, and sites-available libraries.
• /etc/monit/conf-enabled/: Library of active enabled settings. The settings that Monit reads are added to this (in the include line at the end of the monitrc file). The library is used in such a way that the desired settings are enabled symbolic links link here using the conf-available/ from library.
• /etc/monit/conf.d/: Monit reads this directory as well (also in the include section at the end of the monitrc file), but there is nothing in it by default. Consequently, if we put settings here, they will also be active.
• /etc/monit/monitrc.d/: Scanning of this directory is not enabled by default, so the files in it are not active. By default, it includes a fail2Ban configuration file. If we use Fail2Ban program, then we can even turn one on conf-enabled/ by linking to a directory, or we can scan this directory at the end of the monitrc file with another include line, etc.

So the bottom line is that among the configuration libraries, the conf-available/ and the conf-enabled/ we use libraries, we do not deal with the other two.

Setting up a web interface

Monit has its own web interface where you can monitor the detailed status of the system from your browser.

To enable the web interface, create a new file, say a /etc/monit/conf.d/ in the library:

nano /etc/monit/conf.d/web-interface

And let's include the following:

set httpd port 2812 and
    SSL ENABLE
    PEMFILE /etc/monit/ssl/ispserver.pem
    allow admin:abc123

Where the rows mean the following:

• set httpd port 2812 and: Web access is done on port 2812 (Monit's default port).
• SSL ENABLE: Since the ISPConfig server already HTTPS redirection is in effect, so we can only access this interface via HTTPS
• PEMFILE: Enter the PEM file here, which contains the SSL certificate. The file specified here doesn't exist yet, so we'll have to tinker with it a little later...
• allow : : Enter the admin user and add the password for HTTP authentication here. You can also set something else here.

If you have it, save it.

SSL setup

Monit can be given SSL in several formats, the most convenient of which is probably the PEM file, since we only have to deal with one file. This is also beneficial because ISPConfig generates this for us .key and the .crt by concatenating files. Also in the case of older Perfect server installations we had to create the chaining and concatenation of these SSLs, as well as the renewal script, in newer ISPConfig installations this is already built in. The bottom line is that ISPConfig has what is needed here .pem with a format that he renews for us from time to time.

All of this would be too nice if this a /usr/local/ispconfig/interface/ssl/ispserver.pem file one by one, this could be specified in the Monit configuration. Alternatively, it can be done, but if we set it directly and then restart the monit the service with the command:

monit reload

then it throws the following error:

The SSL server PEM file '/usr/local/ispconfig/interface/ssl/ispserver.pem' permission 0750 is wrong, maximum 0700 allowed
/etc/monit/monitrc:308: SSL server PEM file permissions check failed ''

Here, for the sake of the example, I commented out the above setting and temporarily put it directly in the ISPConfig .pem file to see the error. The file currently has 0750 permissions, but Monit allows a maximum of 0700.

Because of this small detail, we will have to deal with this further, but for now all we will do is to copy this for Monit .pem file and change its permissions and start the Monit web interface with it.

Accordingly, let's now create a directory and enter it:

mkdir /etc/monit/ssl
cd /etc/monit/ssl

Copy the PEM file in question into this and set its permissions:

cp /usr/local/ispconfig/interface/ssl/ispserver.pem .
chmod 700 ispserver.pem

Then reload the Monit configurations:

monit reload

If monit did not give an error, we can also view the web interface.

Web interface overview

Enter the monit address in the browser, which is the following on my Debian 10 virtual machine:

https://debian10.linuxportal.vm:2812/

Request HTTP credentials here:

enter the user and password set in the configuration, then the Monit home page will appear:

There is only a basic CPU, memory and swap file display here. If we click on the hostname of our server in the System column (debian10 in my case), the general status page of the system will appear:

As you can see, Monit is still in a very basic state, so the next thing we need to do is to set the monitoring of different resources, limit values, and the actions to be activated when the limits are reached, notifications, etc.

Set up services and resources

In this section, we review - without claiming to be complete - some of the more important services and resources to be monitored. The Monitor conf-available library already contains some pre-made settings, which can be supplemented by the With setup examples on Monit's official site.

System resources

Here we set the limit values ​​of the system resources, which include the monitoring of load, memory, CPU and swap values.

Let's create the following file:

nano /etc/monit/conf-available/resources

And let's include the following:

check system $HOST
    if loadavg (5min) > 4 then alert
    if loadavg (15min) > 2 then alert
    if memory usage > 80% for 4 cycles then alert
    if swap usage > 60% for 6 cycles then alert
    # Test the user part of CPU usage
    if cpu usage (user) > 40% for 6 cycles then alert
    # Test the system part of CPU usage
    if cpu usage (system) > 20% for 6 cycles then alert
    # Test the i/o wait part of CPU usage
    if cpu usage (wait) > 40% for 4 cycles then alert
    # Test CPU usage including user, system and wait. Note that
    # multi-core systems can generate 100% per core
    # so total CPU usage can be more than 100%
    if cpu usage > 200% for 4 cycles then alert

Here's one if-then we can see a structured structure in which limit values ​​and tasks to be performed when the values ​​are reached can be specified in many ways.

The exact values ​​should be set individually for everyone, taking into account the resources, performance and average load of your server. These are therefore only guideline sample values. Let's set them in such a way that we first start with a lower, stricter setting, and then, monitoring the system for a short time, depending on the notifications, we "relax" these values, until we finally reach the optimal settings.

For example, it is different for a server with 4 CPU cores loadavg, Respectively cpu usage values ​​must be specified as for a server with 8 CPU cores. Or, for example, in the case of a configuration with 4 GB of RAM, different values ​​must be set than in the case of a machine with 8 GB of RAM, etc. We also monitor general load levels.

The purpose of the settings here is to adapt them to the average operation of our server, and to receive alerts only in the case of load "spikes" that differ from it, so that we can then check the causes of overheat peaks / abnormal operation, etc. We will notice these when the email notifications come, and we will see what values ​​are considered normal or extreme in the case of our own server.

If we have saved the file after our initial settings, then turn on what is pointing to this setting conf-enabled we can do it with a symbolic link placed in the library:

ln -s /etc/monit/conf-available/resources /etc/monit/conf-enabled/resources

Then reload the Monit configurations:

monit reload

If everything is fine, we will only get this output:

Reinitializing monit daemon

Then go back to the Monit interface in the browser, and then open the link of the line containing the host name under System again, as before:

Compared to the previous state, here a gray area appears at the bottom, where we can see our settings. From then on, the system sends alerts if one of the limit values ​​is reached.

About additional settings for system resources here you can find out. For example, we can extend these settings to desktop use, if we use a graphical desktop environment, etc.

Apache

To monitor the Apache web server, Monit already includes the configuration file, open it:

nano /etc/monit/conf-available/apache2

Which basically includes the following:

check process apache with pidfile /var/run/apache2/apache2.pid
    group www
    group apache
    start program = "/etc/init.d/apache2 start"
    stop program  = "/etc/init.d/apache2 stop"
    if 4 restarts within 20 cycles then timeout
    if failed host localhost port 80 with protocol http and request "/server-status" with timeout 25 seconds for 4 times within 5 cycles then restart
    depend apache_bin
    depend apache_rc

check file apache_bin with path /usr/sbin/apache2
    group apache
    include /etc/monit/templates/rootbin

check file apache_rc with path /etc/init.d/apache2
    group apache
    include /etc/monit/templates/rootbin

For comparison, in the server version installed with the automated installer of ISPConfig, the same config file contains the following:

check process apache with pidfile /var/run/apache2/apache2.pid
    group apache
    start program = "/usr/bin/systemctl start apache2" with timeout 60 seconds
    stop program  = "/usr/bin/systemctl stop apache2"
    if failed port 80 protocol http then restart
    if failed port 443 then restart
    if 5 restarts within 5 cycles then timeout
    depend apache_bin
    depend apache_rc

check file apache_bin with path /usr/sbin/apache2
    group apache
    include /etc/monit/templates/rootbin

check file apache_rc with path /etc/init.d/apache2
    group apache
    include /etc/monit/templates/rootbin

There is some difference. For example, the latter file is the one used earlier /etc/init.d/ instead of start/stop scripts, it uses systemctl and also monitors the operation of port 443 (HTTPS). Furthermore, the first file is the /server-status examines the operation by querying the server status, which was mentioned earlier it was discussed here in connection with another setting.

Here we can decide whether to keep the "factory" Monit Apache settings, or if we are using an ISPConfig-based system, we will take over the settings compiled by the ISPConfig people. I would recommend the latter, if the creators of the control panel or the installation script have already prepared it this way, so I will replace the contents of the config file with the second one.

Then turn on the setting as you already know and reload the configs:

ln -s /etc/monit/conf-available/apache2 /etc/monit/conf-enabled/apache2
monit reload

Then update it to the main page of the web interface:

The "Process" section appeared here, where we already have an "apache" row, and the "File" section also appeared with the two monitored Apache files. Here we can see Apache's aggregate data, but if we click on the "apache" link in the "Process" section, we can see more data about Apache:

Here, too, the lower gray lines contain the configured events. There are not only alerts here, but, for example, if Apache ports 5 or 80 are not reached for 443 seconds, the web server will be restarted. Also, if this repeats for 5 cycles, it will stop monitoring. In such cases, of course, it also sends a notification about every event, so if for some reason something like this happens, you have to look into the problem.

For example, on my live server, if I restart Apache for some reason, either manually or by changing something in ISPConfig that restarts it, Monit already indicates that it is not reached (with several websites and many configurations, restarting Apache often exceeds 5 seconds ). After the restart, the next email will come saying that the connection has been restored. It's useful, especially if you're not in front of your computer and the notifications are sent to your mobile phone (in this case, for example, you may be worried about the server while shopping. :D ).

MySQL / MariaDB

So we don't have anything to do with that right now, just so we can see what it's all about, I'll copy the contents of the original mysql config:

check process mysqld with pidfile /var/run/mysqld/mysqld.pid
    group database
    group mysql
    start program = "/etc/init.d/mysql start"
    stop  program = "/etc/init.d/mysql stop"
    if failed host localhost port 3306 protocol mysql with timeout 15 seconds for 3 times within 4 cycles then restart
    if failed unixsocket /var/run/mysqld/mysqld.sock protocol mysql for 3 times within 4 cycles then restart
    if 5 restarts with 5 cycles then timeout
    depend mysql_bin
    depend mysql_rc

check file mysql_bin with path /usr/sbin/mysqld
    group mysql
    include /etc/monit/templates/rootbin

check file mysql_rc with path /etc/init.d/mysql
    group mysql
    include /etc/monit/templates/rootbin

To enable the MySQL/MariaDB configuration, run the following command:

ln -s /etc/monit/conf-available/mysql /etc/monit/conf-enabled/mysql

then reload the Monit configuration:

monit reload

Then, when updated in the browser, the set will be expanded with MySQL parts here as well on the interface:

Here too, if we click on the "mysqld" process or the files, we can see the details here, and from here we can also restart the service manually, etc.

PHP

Monit's library of available configurations, a /etc/monit/conf-available/ for a fresh install, it doesn't include PHP configurations, so we have to create it. Or if we use several PHP versions at the same time, in this case a separate one for each version.

In this example, I am only creating settings for PHP 8.0, because this is the most recent version on this server, but we can create settings for all the PHP versions we use along this logic.

Let's create the following file:

nano /etc/monit/conf-available/php8.0-fpm

And let's add:

check process php8.0-fpm with pidfile /var/run/php/php8.0-fpm.pid
    group php-fpm
    start program = "/usr/bin/systemctl start php8.0-fpm" with timeout 60 seconds
    stop program  = "/usr/bin/systemctl stop php8.0-fpm"
    if failed unixsocket /var/run/php/php8.0-fpm.sock then restart
    if 5 restarts within 5 cycles then timeout

I took these settings from the settings of the automated installation server version of ISPConfig, which is a logical solution, since pidfiles and socket files are also set and controlled by ISPConfig, so Monit will definitely work well with this setting.

If you are ready, save it, then create the link here and reload the Monit configuration:

ln -s /etc/monit/conf-available/php8.0-fpm /etc/monit/conf-enabled/php8.0-fpm
monit reload

After that, the PHP 8.0 line appears on the web part:

sshd

Az SSH we can also monitor the status of our server. The freshly installed Monit version contains a openssh-server settings file too, let's look into it a little more before we turn it on:

nano /etc/monit/conf-available/openssh-server

check process sshd with pidfile /var/run/sshd.pid
	group system
	group sshd
	start program = "/etc/init.d/ssh start"
	stop  program = "/etc/init.d/ssh stop"
	if failed host localhost port 22 with proto ssh then restart
	if 5 restarts with 5 cycles then timeout
	depend on sshd_bin
	depend on sftp_bin
	depend on sshd_rc
	depend on sshd_rsa_key
	depend on sshd_dsa_key

check file sshd_bin with path /usr/sbin/sshd
	group sshd
	include /etc/monit/templates/rootbin

check file sftp_bin with path /usr/lib/openssh/sftp-server
	group sshd
	include /etc/monit/templates/rootbin

check file sshd_rsa_key with path /etc/ssh/ssh_host_rsa_key
	group sshd
	include /etc/monit/templates/rootstrict

check file sshd_dsa_key with path /etc/ssh/ssh_host_dsa_key
	group sshd
	include /etc/monit/templates/rootstrict

check file sshd_ecdsa_key with path /etc/ssh/ssh_host_ecdsa_key
	group sshd
	include /etc/monit/templates/rootstrict

check file sshd_ed25519_key with path /etc/ssh/ssh_host_ed25519_key
	group sshd
	include /etc/monit/templates/rootstrict

check file sshd_rc with path /etc/ssh/sshd_config
	group sshd
	include /etc/monit/templates/rootrc

Here, I would like to draw your attention to the "port 6" section in line 22. The default port for SSH is 22, but if we use a unique SSH port number, then the port number 22 must be changed to our unique SSH port number so that Monit can also monitor it.

Let's also enable this settings file:

ln -s /etc/monit/conf-available/openssh-server /etc/monit/conf-enabled/openssh-server
monit reload

ssh_host_dsa_key error

After setting up SSHD, when reloading the Monit configuration, you might get an error like this:

checksum: file /etc/ssh/ssh_host_dsa_key is not regular file
/etc/monit/conf-enabled/openssh-server:27: Cannot compute a checksum for file /etc/ssh/ssh_host_dsa_key 'sshd'

In this case, if we look at the web interface, it looks like this:

Entering the sshd_dsa_key line in the files:

In this case, we need to generate a DSA host key for ourselves ssh-keygen command:

ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key

Here, just press enter on the passphrases.

After that, Monit can be reloaded without error:

monit reload

Then the main page is updated:

Below, the file problem has already been resolved, you just need to enter the sshd line above and restart the service. Or you can restart it from the command line. Or we wait until the Monit update cycle elapses, which is 2 minutes by default, and then it disappears by itself:

I have experienced this error with manually installed Debian 10 (Buster) perfect servers (all 3 out of three samples). I have not experienced such a problem with perfect servers built with the automated installer. For those, these host_dsa keys are likely to be generated separately during the installation process.

Cron

To monitor Cron, enable the /etc/monit/conf-available/cron configuration file:

ln -s /etc/monit/conf-available/cron /etc/monit/conf-enabled/cron
monit reload

A crond Process, two cron_ files and a cron_spool directory below the Directory were created here. Thus, the system monitors these as well.

Filesystem

Another thing that is recommended to turn on is file system monitoring. This is not included in the fresh Monit installation, so we need to create the configuration file:

nano /etc/monit/conf-available/filesystem

Then add the following content:

check filesystem rootfs with path /
    if space usage > 90% then alert
    if inode usage > 80% then alert

Then turn it on and update the configuration:

ln -s /etc/monit/conf-available/filesystem /etc/monit/conf-enabled/filesystem
monit reload

Here, the Filesystem section appeared in the middle, where rootfs was inserted. This is a very useful thing, because, for example, it notifies you if the disk space on our storage has reached a certain percentage, etc.

Email notifications

If we want to receive email notifications about different Monit events, we need to create another configuration file:

nano /etc/monit/conf-available/alerts

Then, include the following content:

set mailserver localhost
set alert felhasznalo@domain.tld

Replace your own email address here. Then save and reload the configuration:

monit reload

In addition to these, there are even more setting options, which the Examples on Monit's website worth studying and experimenting with.

A next page let's continue with a look at the second scenario, i.e. configuring Monit after ISPConfig is automatically installed...

Navigation