Upgrade Mautic from v2.16.3 to v5.0.3 via docker

[luk4s]

I’d like to share story and my experience with mautic migration to new server with upgrade to latest version.

Story:

We have Mautic v2.16.3 running by our web provider, which is not willing to support instance anymore, so I decided to migrate instance to own server.

Input data

mautic: v2.16.3
mysql: 5.5 (database dump)
dump size: 639M

First try was very naive, but it was try

Run v5.0.3 directly with database from v2.16.3. Heres are errors:

[notice] Migrating up to Mautic\Migrations\Versionzz20230929183000
[error] Migration Mautic\Migrations\Version020230615115326 failed during Execution. Error: "An exception occurred while executing a query: SQLSTATE[22032]: <<Unknown error>>: 3140 Invalid JSON text: "The document is empty." at position 0 in value for column '#sql-1_7d.headers'."

In ExceptionConverter.php line 117:

  An exception occurred while executing a query: SQLSTATE[22032]: <<Unknown error>>: 3140 Invalid JSON text: "The document is empty." at position 0 in value for column '#sql-1_7d.headers'.


In Exception.php line 28:

  SQLSTATE[22032]: <<Unknown error>>: 3140 Invalid JSON text: "The document is empty." at position 0 in value for column '#sql-1_7d.headers'.


In Connection.php line 71:

  SQLSTATE[22032]: <<Unknown error>>: 3140 Invalid JSON text: "The document is empty." at position 0 in value for column '#sql-1_7d.headers'.

=> mautic/app/migrations/Version020230615115326.php at 1bf73cce081f948c1945e4679a455d6cb508435c · mautic/mautic · GitHub

One of tables probably did not contains JSON valid value (headers column) => SO based on source of migration, I was checked data inside tables and fix was easy, so try again:

UPDATE emails SET headers = "[]";

=> but then error

An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'conditions' in 'form_fields'

This says to me that I must migrate over each major version…

Start with upgrade to v3

I migrated in sandobx, I recommend this for everybody, do not play with production, use sandbox!

Here is my docker-compose.yml

name: "v3-mautic"
version: "3"

services:
  database:
    image: percona:5.7
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: mautic
    ports:
      - "3306"
    volumes:
      - db_data:/var/lib/mysql
      - ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
      - ./percona.cnf:/etc/my.cnf.d/mautic.cnf # <- explained later
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_czech_ci --sql-mode=""

  mautic:
    image: mautic/mautic:v3-apache
    volumes:
      - mautic_data:/var/www/html
    environment:
      - MAUTIC_DB_HOST=database
      - MAUTIC_DB_USER=root
      - MAUTIC_DB_PASSWORD=root
      - MAUTIC_DB_NAME=mautic
      - PHP_INI_DATE_TIMEZONE=Europe/Prague
      - MAUTIC_TRUSTED_PROXIES=["0.0.0.0/0"]
      - MAUTIC_RUN_CRON_JOBS=false
      - MAUTIC_RUN_MIGRATIONS=false
    depends_on:
      - database
    ports:
      - "8080:80"
volumes:
  db_data:
  mautic_data:

GitHub - mautic/docker-mautic at mautic3

! docker image v3-apache includes version 3.3.4

Probably its mentioned somewhere but I missed it, docker images officialy on dockerhub ARE NOT UPTODATE!

step-by-step:

1. place dump into docker-entrypoint-initdb.d = in compose file its binded to database container, percona automatically load dump on first start
2. start database contianer and wait until data are loaded a db is ready = docker compose up -d database --wait
3. Now needs to start mautic container, update mautic inside and run migrations:

docker compose run --rm mautic bash

run it again, v3 copy mautic data into persistant layer and exit…
then attach container without entrypoint

docker compose run --rm --entrypoint "" mautic bash

php bin/console mautic:update:apply
php bin/console mautic:update:apply --finish

Now I received error

Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 66002784 bytes)

This is known limit mysql5.7 with default config = database is to heavy, so bypass it by override mysql setting.
In my example I created configuraton called “percona.cnf” and bind it into database volume (its already in example docker-compose.yml above)

[mysqld]
innodb_large_prefix=1
innodb_file_format=Barracuda
innodb_file_per_table=1

After restart databa se container, run again update script of mautic =>

Success! You are now running version 3.3.5 of Mautic.

Continue with upgrade to v4

GitHub - mautic/docker-mautic at mautic4

Thanks to topic: Update Mautic Docker from v3 to v4

Just updated docker-compose.yml, => use image mautic/mautic:v4-apache. Remove mautic volume!

!! docker image v4-apache includes version 4.4.9 !! But for success upgrade must be latest v4

Existing containers and volumes must be destroyed before start

docker compose down -v mautic

Then you can create new container and prepare mautic:

docker compose run --rm mautic bash

then attach container without entrypoint

docker compose run --rm --entrypoint "" mautic bash

and run migration command…

note: mautic:update:apply failed without error details

php bin/console mautic:update:apply
php bin/console mautic:update:apply --finish

These update command will updated mautic codebase to latest 4.4.11 => with this version manual migrations pass

php bin/console doctrine:migration:migrate

At this point, database is v4.4.11 migrated, I updated all emails record for default empty JSON array:

UPDATE emails SET headers = "[]";

Continue with upgrade to v5

This is stright forward.

1. export database (because migration between percona5.7 and 8) by docker compose exec database mysqldump -u root -p mautic > docker-entrypoint-initdb.d/v4_mautic.sql
2. use new docker-compose.yml for v5 according docker-mautic/examples/basic at mautic5 · mautic/docker-mautic · GitHub, I use one of example and just add docker-entrypoint-initdb.d into db volumes.
3. make sure DOCKER_MAUTIC_RUN_MIGRATIONS=true is enabled in .mautic_env
4. start db container for create database and load dump
5. then start mautic_web container, in my case everything went fine

config/local.php

local.php contains aside credentials to db also paths and other settings. I must manually adjusted credentials (easy) and paths to uploads, logs, etc…

For change cache_path, log_path I was change our original path from /data/websites/my.mautic/www/app to path correspond with mautic5 docker image => /var/www/html/var

sed -i 's|/data/websites/my.mautic/www/app|/var/www/html/var|g' local.php

Rest paths to media, uploads was specified reltive to previous, and mautic5 image use docroot folder

sed -i 's|/var/www/html/var/../media|/var/www/html/docroot/media|g' local.php

Do not forget to backup your local.php first !

Conclusion

Throughout the migration process, I encountered numerous errors, particularly with constraints conflict. An example of such errors is when attempting to add a foreign key constraint:

mysql> ALTER TABLE campaign_summary ADD CONSTRAINT FK_6692FA4FF639F774 FOREIGN KEY (campaign_id) REFERENCES campaigns (id);
ERROR 1215 (HY000): Cannot add foreign key constraint

Similarly, another issue arose with:

mysql> ALTER TABLE campaign_summary ADD CONSTRAINT FK_6692FA4F71F7E88B FOREIGN KEY (event_id) REFERENCES campaign_events (id) ON DELETE CASCADE;
ERROR 1215 (HY000): Cannot add foreign key constraint

After spending several hours manually executing migrations and dealing with these errors, I discovered that the Mautic image was not the latest version. Running an update with php bin/console mautic:update:apply resolved all these issues.

Using Docker (Compose) for migration greatly simplified setting up Mautic, eliminating the need to install all dependencies manually and allowing for quick switching between versions.

I hope this topic isn’t duplicity and that it will help someone in their migration journey.

[tobsowo]

Thank you.
Sharing this in the next newsletter.