Yet Another Azure Topic

[adam.w]

Your software
My Mautic version is: 5.1.1
My PHP version is: 8.2
My Database type and version is: mysql 8

Your problem
I have seen many partial tutorials on how to run Mautic in Azure. None of them seem to result in a working implementation. They are either missing crucial details for what needs to happen before pushing to Azure or the Azure functionality used is not clearly stated.

I have done a lot of investigation to figure out what works and what doesn’t, and to understand why. I have come to a point where the problem is no longer Mautic or Azure, the problem is now me missing knowledge that I need to fix the this storage issue.

Everything I’m doing is going to be made public so hopefully other people can follow and use it, it is messy right now but I can make it all available (currently has some sensitive data that needs to be pulled out).

The part that I am struggling with could possibly be solved by another post here that mentioned using rsync when updating an install.

Here is my approach to run Mautic on Azure in a way that is scalable and secure while keeping it low cost and maintainable:

• Build docker images locally
• Push to Azure Container Registry
• Use Azure Storage to persist and share files as needed
• Use a Container Apps Environment to manage scaling and security
• Use a Container App Job to replace the Cron container
• Use a Container App running php-fpm as my backend
• Use a Container App running running supervisor as my worker
• Use a Container App running a custom Nginx image to handle ingress
• Use an Azure MySql resource to host the database
  This looks like it will cost $25-$30 a month before any scaling

I am now working on mapping the storage correctly so that all the systems have access to the files they need while not creating a difficult/complex upgrade path for the future.

If anyone else is interested in implementing this, we should plan some time to have a working session to complete this.

Azure web apps will not work due to the features needed being in preview and do not have the needed functionality to work with storage containers.

I need to populate the smb file share docroot with the files in the backend container at /var/www/html/docroot. I can’t mount there because it hides the existing files.

I don’t want to have a copy operation every time the container starts because I don’t want to waste the resources.

I tried creating a job to run manually to copy the files over by running cp -R -u -p /var/www/html/docroot /datastore where /datastore is the mountpoint but, nothing is being copied over and I am not sure why because I am not seeing any errors in the logs.

I also tried creating a /data directory to copy it to during the container build. and then copy from /data to /datastore it copies to /data but I still cant seem to get the files to move to the /datastore mount.

Someone with Kubernetes experience would likely be able to fix this very quickly since that is the backend of the container environment.

If there is anything specific that I can provide to help fix this please let me know, I just didn’t want to dump all the files here and bloat this more.

[clementine]

Hi,

You have done some extensive research, It just seems far fetched to me that we would need all these services and containers you mentioned to get Mautic running in a scalable fashion, but I could be mistaken. (Meaning : it should not be this complicated, right?)
Could you share the stuff you tried/failed after redacting sensitive stuff?

We are trying to get this on Azure as well and have installed Mautic on Azure by following the guideline below (you can also use a container app instead of an App Service).
By installed we mean : the website runs, we can complete setup and show the dashboard, enter some data, segments , stages…so far so good. (NO campaigns implemented yet,no scheduled mails yet, etc => this is probably where your mentioned cron container comes in…but we havent gotten that far yet, see below)

So this seems to work for the basics. We did not test the advanced stuff yet, or consider scaling like yourself at this time…why? Well… every time the app service or container app restarts for config changes or whatever, the installer is shown again, which ofcourse dumps all the data created from before the container restarted (since it creates new tables after the “database” part of the installer I assume). => Maybe you already noticed this as part of your investigation?

For us this is a work in progress and we are trying to solve the “installer shows on restart” problem before continuing.( => might be related to your "Azure storage to persist files item?)

Step 1: Set Up Azure Database for MySQL
1. Navigate to Azure Portal: Go to Azure Portal.
2. Create a MySQL Database:
◦ Search for Azure Database for MySQL and click Create.
◦ Choose Single Server for simplicity.
◦ Configure the following:
Resource Group: Create a new resource group, e.g., MauticRG.
Server Name: Choose a unique name, e.g., mauticdb.
Location: Select your preferred location.
Version: Use 8.+ (ensure it’s compatible with Mautic).
Compute & Storage: Select a suitable configuration based on your anticipated load.
◦ Set up Authentication with a username and password (e.g., admin and a strong password).
◦ Review + Create to finish setting up the MySQL server.
3. Configure MySQL Server:
◦ Go to the Networking section of your MySQL server and allow access from Azure Services.
◦ In Connection Security, configure SSL (optional but recommended).
4. Create the Mautic Database:
◦ Connect to the MySQL server using a MySQL client like MySQL Workbench.
◦ Create a database for Mautic:
sql
Copy code
CREATE DATABASE mautic CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
Step 2: Set Up Azure App Service for Mautic
1. Create an Azure App Service:
◦ In the Azure Portal, search for App Services and click Create.
◦ Configure the following:
Resource Group: Select MauticRG.
Name: Give your app a name (e.g., mauticapp).
Publish: Choose Docker Container.
Operating System: Select Linux.
Region: Choose the same region as your MySQL server.
2. Set Up Docker Configuration:
◦ In the Docker tab, configure:
Image Source: Select Docker Hub.
Image: mautic/mautic (potenitally with version :v4 , :v5, …)
◦ Review and create the App Service/container app
3. Configure Environment Variables:
◦ Go to Configuration > Application settings and set the following variables for the Mautic application:
MAUTIC_DB_HOST: Your MySQL server’s hostname (e.g., mauticdb.mysql.database.azure.com).
MAUTIC_DB_PORT: 3306
MAUTIC_DB_NAME: mautic
MAUTIC_DB_USER: The username, e.g., admin@mauticdb
MAUTIC_DB_PASSWORD: The password you set up.
MAUTIC_RUN_CRON_JOBS: true
4. Enable Managed SSL (Optional):
◦ In TLS/SSL Settings, configure HTTPS Only to enforce secure access.
◦ You can configure a custom domain if you have one.
5. Restart the App Service:
◦ Restart the App Service from the Overview section to apply the settings.
Step 3: Access Mautic
1. Navigate to Mautic:
◦ Go to the App Service URL (e.g., https://mauticapp.azurewebsites.net).
◦ Complete the Mautic setup by filling in the database connection details using the environment variables you set.

I am able to navigate to the page

[adam.w]

I think the reason why you are having to reinstall every time is that the docker image is creating this volume: 48 VOLUME [/var/www/html/config]. In Azure you will need to either persist storage by mapping the directories like this, and I believe the media directory, to either /home or through azure storage. if the configurations are not persisted they will redeploy the template every time. That template does not have the DB driver set which I think is also the signal for launching the installer. If this is the cause you can quickly verify it by running it locally in docker. Local Docker will persist the volume and you wont have this problem. I also think I read somewhere that the mautic/mautic docker isnt meant to be a production system and that you should deploy with composer instead. Other concerns you should have with that are:

• ENV PHP_INI_DIR=/usr/local/etc/php wont be persisted
• the html folder has wide open access chmod 777
• this might be personal preference but apache is more difficult to efficiently scale, but again that could just be me.
• logs will not persist through restarts unless you set up monitoring in Azure

For all of those reasons above, I decided to build the docker image locally using the nginx/php-fpm example in the docs.

I push to an azure container registry because I can speed up deployments, they cost next to nothing and I don’t have costly ingress traffic from deployments. from what I looked at hosting on ACR ended up as a nominal cost and gives me a lot more control.

Azure file shares are what I opted for to keep persistent data like the configurations in the html directory, because they can scale independently to a container, and you dont pay for the scaling. So if local container file storage persisted in /home were to cause conditions that make the containers scale, I’m not paying for it.

I chose container apps over instances, because it is the closet thing Azure has to having all the docker compose functionality while also supporting non-local persistent storage. I went with a container environment because it is actually running K8’s on the backend and is fully managed. so you get all of the scalability and abstraction you get from kubernetes without having to manage kubernetes and it is incredibly low cost in comparison to AKS. It also has the load balancer, virtual network, and nic config out of the box and you can add your own vnet at creation if you want more control.

I also migrated to an alpine base container to reduce size and decrease resource usage. I still have optimizations to do because I decided that I was actually going to put a build stage in the dockerfile so that when this is redistributed people will not have to deal with the conflicts between node and php during build. Since it is in a container environment I can keep the backend servers completely restricted except for a single port that is only available to systems that are inside the vnet, increasing security.

I use nginx to serve static files, like apache in your build, but I also use it as a reverse proxy so that the more expensive part of the workload wont scale just because more people are connected. it will only scale if more people are connected and actually using the system.

Cron or cron like functionality is important as well. you will need them for things like mail handling and campaign automation

As you get into the storage, keep in mind that if you are using anything in azure that implements orchestration you may not be able to bind multiple volumes in the same directory without them overwriting each other. to make it work you can use the subpath feature. This is how I found out that the container environment is sitting atop k8 pods.

Once I got this all up and running I checked it out to make sure the functionality was all working.

Now I am moving all of the infrastructure setup into IaC, and properly storing my secrets. Ill move the cron jobs to azure functions, and potentially drop nginx as well, I will likely keep it though so that I can add more systems into this and use nginx to handle routing. I’m thinking about things like strapi, vtiger, suiteCRM for marketing and Epic Web, Azure AI, for web applications.

The reason I am doing all of this is because I have worked with startup after startup that need the functionality that companies charge 400 to 1500 a month for but dont have the budget to support it. Also it is not the software that companies like hubspot offer that is worth overpaying for. it is the marketing, sales, and product people that can actually make it drive revenue that they should be paying for.

[clementine]

@adam.w

me is that the docker image is creating this volume: 48 VOLUME [/var/www/html/config]. In Azure you will n

First of all, thanks for your response. I realy appreciate you taking the time and effort.

As it goes in this business I had to shift priority to something else for a bit. But that does not mean we stopped working on it. I will be trying to get this up and running again this week, using the considerations of your posts. (and maybe a little guidance from ChatGPT )
And yes we are also startup with the exact limitations you mentioned trying to get this functionality running (6 people at our company, 2 devs, who are specialized in C# and not well based in docker, php and/or linux, so the learning curve is steep)

I will post back with some results later.
Again, thanks for taking the time and effort.