How to upgrade our Matomo (formerly Piwik) web analytics system to version 4.x (page 2)

botond published 2020/12/15, k - 09:34 time

Content

  1. page: Settings before updating Matomo, update
  2. page: Post-update settings, geolocation

 

2. page content

 

Continuation

The description on the first page we made the necessary settings before updating the Matomo (Piwik) web analytics system and updated the system, and on this page we made the necessary settings after the updates.

 

 

Matomo 4 - Provide post-upgrade system requirements

After installing the Matomo (Piwik) web analytics system we met the system requirements, but after upgrading to the current major version, our server must also meet the changed system requirements. You can access a long list of these by clicking on the "View the full system check report" link in the middle "System Check" panel of the Admin main page:

Matomo - System Check Page

On this page we get a very long list with some problems to be solved. These, of course, are highly dependent on the configuration of our system, so other states may occur on a different server, which may include different errors. In this description, we will now go over those that are on this list. Other bugs are not covered in this description at this time, but if you have run your Matomo system according to the previous installation guide here, you will most likely not encounter any other issues here.

At the beginning of the list, we can see that, for example, our version of PHP is fine, newer than the minimum version of 4 required for Matomo 7.2.5. Above is a separate scrollable panel that displays information in the form of a log file. This section is not as clear as the formatted list layout below, so let’s scroll further to the section below.

File integrity

File integrity is a kind of security and technical check in which Matomo compares the files in our installed copy with the "template" for that version. For example, if you find unnecessary files in Matomo's directories, it indicates that they are not needed in the current version, so you can delete them. Or, if one of the files does not match the size of the corresponding file in the "factory" instance, it also indicates. In the same way, if something is missing, it also indicates to us. In addition, the system compiles ready-made Shell commands to run to correct errors.

For me, the system displays:

Matomo - System Check - File Integrity Errors

If a similar gray panel does not appear here, but there is a green check mark in the file integrity line, then there is nothing to do here, skip this section. But after upgrading to the main version, we almost certainly need to delete a few things.

Here we log in with our matomo user, what else we created it for him after installation (a PHP-FPM and run the commands. From the root user, you can easily switch to su command to the matomo user:

su -l matomo
Of course, we could run it as root, but if it already has a separate user, it's a good idea to do all the Matomo-related tasks with it. So, for example, if a command came up here that needed to create files or directories, the owner of the created things would be right.

However, if you did not create such a user after installing Matomo that year because you did not make the PHP-FPM modification, run the specified commands as root.

So based on that, I need to run the following commands:

rm -Rf /opt/matomo/libs/bower_components
rm \
    "/opt/matomo/bower.json" \
    "/opt/matomo/composer.json" \
    "/opt/matomo/composer.lock" \
    "/opt/matomo/js/piwik-min.js" \
    "/opt/matomo/js/piwik.js.orig" \
    "/opt/matomo/misc/gpl-3.0.txt" \
    "/opt/matomo/misc/others/ExamplePiwikTracker.php" \
    "/opt/matomo/vendor/davaxi/sparkline/.codeclimate.yml"
For the sake of clarity, I have grouped the second command into several lines so that it can be read on narrower displays.
The commands just mentioned apply only to my situation, in other systems you may need to run different commands, so let's copy our own commands from the gray panel.

After running the requested commands, refresh the list page with F5 and make sure that the gray panel you just mentioned is gone and replaced with a green check mark.

Database

The next problem that awaits resolution is for the database. And we have more to do here.

Matomo - System Check - Database Character Encoding

Database conversion

The point here is that Matomo 4 uses a different character encoding that allows you to display various special characters, such as emojik, and so on. For some reason, this is not done by their update system, for sure, so run the given command:

/opt/matomo/console core:convert-to-utf8mb4

Convert Matomo database

When you run the command, it first prints that it stops tracking web page traffic during the operation and asks if you can run the operation. Here you can go to "y".

 

 

Depending on the amount of our data, this will run at different times. The factors that determine the amount of data are the number of websites measured, website traffic, and the "age" of our Matomo copy. The latter is because the system also creates monthly summary tables from which you can read statistical statements much faster. So if you have been using Matomo for a long time, you will need to convert more tables.

Matomo database conversion is complete

He ran here in a few seconds. But on my regular server, for example, it took several minutes.

Enable LOAD DATA INFILE

LOAD DATA INFILE MySQL / MariaDB allows the database to retrieve rows from files at high speed. Based on this, the Matomo (Piwik) web analytics system will be able to read the stored data much faster when, for example, we ask it for various statistical summaries and statements.

So, the error message that remained in the database section after the previous problem was resolved:

Matomo - System Check - Database - LOAD DATA INFILE error

In this PHP environment, this PHP mysqli property is disabled by default, but it may be enabled elsewhere, so it does not report this error. In the latter case, skip this section.

To enable LOAD DATA INFILE, the PHP variable "mysqli.allow_local_infile" must be enabled. By default, that's enough, but you may need to set it to "open_basedir" so that PHP can access the files you need, including the mysqli plugin. So it is advisable to set both to work for sure.

There are two ways to set these PHP variables, depending on whether you did the installation of Matomo at the time. PHP-FPM migration.
Configuration for PHP-FPM

If you have done the PHP-FPM setup mentioned above, in this case, log in rootas Matomo to our PHP-FPM pool running our system and edit the configuration file. For example, on this Debian 9 server:

nano /etc/php/7.4/fpm/pool.d/matomo.conf

At the end of the file are our own PHP settings that we created anno, to which we add the following two lines:

[...]
php_admin_value[mysqli.allow_local_infile] = on
php_admin_value[open_basedir] = "/opt/matomo:/opt/matomo/tmp/assets"
[...]

So to make our file look like this:

Matomo - Configure PHP-FPM - Enable LOAD DATA INFILE and open_basedir

Save and restart this PHP-FPM:

systemctl restart php7.4-fpm.service

Of course, if you run Matomo with another PHP, edit the configuration file in its pool and restart its FPM service.

The advantage of this is that these settings only affect the Matomo runtime environment.

Setting for default PHP mode

If you have not migrated your Matomo system to PHP-FPM, by default mod_php mode (Apache as a module) our system is running. Find the php.ini file of the PHP that runs our Matomo web statistics and open it rootas for editing.

On this Debian 9 (Stretch) server, the default PHP is 7.0, but now we had to add another PHP when upgrading, which is at least 7.2.5. So look for the ini file accordingly. For example, here for 7.4:

nano /etc/php/7.4/apache2/php.ini

And we add the following two lines to the end:

[...]
mysqli.allow_local_infile = On
open_basedir = "/opt/matomo:/opt/matomo/tmp/assets"

Then restart Apache, as this will run PHP:

systemctl restart apache2.service

 

If we set the PHP variables, we also fixed this problem:

Matomo - System Check - Database - LOAD DATA INFILE OK

If these settings still do not resolve the issue, On the troubleshooting side of Matomo we can find more ideas on what might still prevent the LOAD DATA INFILE from working.

 

 

We are at the end of the list, there are no other problems with this installation. In this case, a green-framed panel at the top of the page tells us that our Matomo configuration is flawless:

Matomo - Flawless system configuration

 

Geolocation

Although this topic would even fit into the post-upgrade requirements chapter, Geolocation deserves a separate chapter, as on the one hand it is not a prerequisite, so it is not mandatory to set it up / use, and on the other hand setting it up requires a bit more work, which in this separate chapter fits more comfortably.

When installing Matomo (Piwik) web analytics system we set the geolocation, and it works without any problems even after updating the current main version, then we have nothing more to do, we can skip this section.

However, I had problems on my server after updating Matomo, which was due to the fact that I had long ago installed the components needed for GeoIP 2 to work, which had become obsolete over time. So in this section, I want to present a real-life problem of its own, which may be unique, but may also occur to others after updating Matomo, so the solution described here may help.

The symptom

After upgrading to the main version 4.0.x of the Matomo web analytics system, the measurement of web pages stopped. Entering the admin main page above, I received an error message in a red panel in the middle:

Matomo - Admin - Error Message

I then looked around the other menus and went to the Geolocation menu and threw such an error page:

Matomo - GeoIP - fatal error - MaxmindDB reader error - getWithPrefixLen () method does not exist

So the error message is exactly:

A fatal error occurred
The following error just broke Matomo (v4.0.5):

Call to undefined method MaxMind\Db\Reader::getWithPrefixLen()
in /opt/matomo/vendor/geoip2/geoip2/src/Database/Reader.php line 232         

Which is that the new Matomo 4.0.5 tried to invite the max php plugin (GeoIP 2) one getWithPrefixLen () method that does not exist.

At the bottom of the screen, even alternative troubleshooting is recommended, such as disabling third-party plugins and listing them as they are. Of course, it seemed to be apparent that it was not these that was causing the problem, but the GeoIP part, so we had to continue on this line.

Temporary solution

Something had to be done quickly to make the page measurement work, so I went back to the previous page where there was still a menu and on the left I went to the extensions menu and temporarily turned off the Geoip2 (core) extension:

Temporarily disable Matomo - Geoip2 plugin

If there is a problem around geolocation, this is the logical step first. After switching off, the system was restored, everything worked, except, of course, city-level geolocation. So that there was no more error and the measurement of the pages went on, it was time to go around the problem more thoroughly and find a solution to it.

Final solution

Details of the maxminddb php plugin in phpinfo:

phpinfo - Old maxminddb data

Here, the maxminddb php plugin is version 1.4.1, and the libmaxminddb library version is version 1.3.2. Compared to current versions: current version of maxminddb php plugin 1.8.0 (release October 2020, 1) and the current version of the libmaxminddb library 1.4.3 (2020 August 6 edition). So time passed over them.

Because these add-ons were compiled manually from a source after installing the Matomo web analytics system, so they were not updated automatically, the compilation and installation must be performed again.

Here we are now doing the same as earlier after installing Matomo, only with newer versions: we install the libmaxminddb C library first and then the maxminddb php extension.

Libmaxminddb Reinstall the C library

If you have not set up geolocation before, but now decide to do so, you must first install the packages that contain the translation tools. Log in as root and run the following command:

apt-get -y install gcc build-essential

However, if you have previously used the MaxMind geoip2 php plugin, these packages are already installed.

Find the latest version of the libmaxminddb C library on GitHub at the link below:

https://github.com/maxmind/libmaxminddb/releases/latest

Copy the libmaxminddb-xyztar.gz link address and download it to your / tmp directory:

cd /tmp
wget https://github.com/maxmind/libmaxminddb/releases/download/1.4.3/libmaxminddb-1.4.3.tar.gz

Of course, if there is a newer version than this, we will use it!

 

 

Unzip the downloaded archive file to tar and then go to the directory you created:

tar -xvzf libmaxminddb-1.4.3.tar.gz
cd libmaxminddb-1.4.3

Then compile the source code and make sure everything went well:

./configure
make
make check

If there was no error, we get the following output:

Compiling and checking the libmaxminddb 1.4.3 C library

Finally, install the compiled package and run it ldconfig command to rebuild the cache (rootrun as):

make install
ldconfig

Installing the libmaxminddb 1.4.3 C library

This is done by compiling and installing the libmaxminddb C library.

You can delete unnecessary things from the / tmp directory:

cd /tmp
rm -rf libmaxminddb-1.4.3*

Reinstall MaxMind-DB-Reader-php PHP plugin

You will need phpize command found in PHP development packages. A PHP development kit is available for any version of PHP, now we will need it to run our Matomo (Piwik) web analytics system.

Get the right phpize version

Download the development package for our appropriate PHP version. In this example, you will need 7.4, which is installed by running the following command:

apt-get -y install php7.4-dev
At that time, we already installed the development package, but if you had to install another PHP when upgrading Matomo now, you will need the phpize command in the development package, because each php version has its own phpize command that prepares the plugins for that PHP system. The last installed PHP system and the phpize command in its development package are always configured in the alternatives system, so the physical location of the phpize command changes to the previous command:

phpize -> / usr / bin / phpize -> / etc / alternatives / phpize -> /usr/bin/phpize7.4

Earlier versions of phpize can still be found in the / usr / bin directory. However, if you have more than one PHP system installed on your server, then phpize command link points to the last installed version. Therefore, it is a good idea to use the appropriate version of the phpize command directly to configure extensions for your appropriate php system. Although 7.4 is the new default here, for the sake of clarity I will use phpize7.4. Of course, everyone uses their own version of PHP.
Install PHP plugin

Go to the / tmp directory and download MaxMind-DB-Reader-php PHP plugin zip version of the From GitHub:

cd /tmp
wget https://github.com/maxmind/MaxMind-DB-Reader-php/archive/master.zip

Unzip and enter the ext / subdirectory under the created directory:

unzip master.zip
cd MaxMind-DB-Reader-php-master/ext

Run the appropriate version of the phpize command here (as root):

phpize7.4

The output of the command depends on the build of our PHP version:

Configuring for:
PHP Api Version:         20190902
Zend Module Api No:      20190902
Zend Extension Api No:   320190902

Then configure, compile the source, and install it (also as root):

./configure
make
make install

Compile and install the MaxMind DB php PHP plugin

Then open the appropriate php.ini file for editing.

 

 

PHP-FPM

If you run Matomo with PHP-FPM, you will:

nano /etc/php/7.4/fpm/php.ini
It is also worth noting here that the php.ini settings apply to all FPM pools. So, for example, if you run other websites with the same PHP version, but in other FPM pools that do not use MaxMind's geoip database, it is a good idea to set the plugin only in Matomo's pool so that other websites do not load. unnecessarily this PHP plugin.
This can be done in the Matomo FPM configuration instead of php.ini as follows:
nano /etc/php/7.4/fpm/pool.d/matomo.conf

Then add the following line after setting the other PHP variables:

php_admin_value[extension] = maxminddb.so

So in this case, this php module is loaded only in Matomo's pool and is not loaded unnecessarily when running other websites. With this, even if at a minimal level, we can save some resources on the server.

Apache module

And if you run it as an Apache module, you can:

nano /etc/php/7.4/apache2/php.ini

Of course, we also use our own PHP version here if we have installed a different one (for example, 7.2 or 7.3).

Then add the following line to the end of php.ini:

extension=maxminddb.so

Save and restart the appropriate service.

PHP-FPM

For PHP-FPM:

systemctl restart php7.4-fpm.service
Apache module

For default PHP, Apache:

systemctl restart apache2.service
PHP plugin check

You can check the plugin by placing a standard phpinfo.php file in the Matomo directory and running it in a browser:

nano /opt/matomo/phpinfo.php
<?php
    phpinfo();
?>
https://www.wordpress.vm/matomo/phpinfo.php

phpinfo - New maxminddb data

 

Finally, take away the unnecessary things:

cd /tmp
rm -rf MaxMind-DB-Reader-php-master/
rm master.zip

 

After the virtual machine, I did the same on my server, so now we can come back to the Geoip2 plugin, which previously caused an error.

 

Turn back the Geoip2 plugin

In the Matomo admin interface, go to the Extensions menu and then turn the Geoip2 extension back on:

Enable Matomo - Geoip2 plugin

Then go to the "Geolocation" menu and select "DBIP / GeoIP 2 (Php)" and save the form.

With this, the geolocation part is also ready.

 

Download server

In the meantime, a lot has been uploaded to the Debian 9 (Stretch) perfect server, so it's worth uploading the newer server version (1.1). The current state of the server is available between downloads: Download the perfect server: Debian 9 (Stretch) v1.1

 

 

Conclusion

You don't always have to do everything to upgrade Matomo, unless the installation is very old and several things are obsolete on the server, such as this perfect Debian9 server, on which I have now updated everything once I have picked it up again. Also, I had MaxMind stuff up a long time ago, so I had to reinstall the necessary ingredients. However, those who installed the Matomo web analytics system later than me have a good chance that new components have already been added, so I didn't have to adjust that much now when updating the main version.

 

 

Navigation

This description consists of several pages: