# A small guide to send mails using doctrine for queue in Mautic 5 **URL:** \ **Category:** Product Support\ **Created:** [August 13, 2024, 5:41pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118 "2024-08-13T17:41:39Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 13, 2024, 5:41pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/1 "2024-08-13T17:41:39Z")
This is a small guide to configure doctrine to send the queue. Will be in the kb soon But since this was a great pain for me for 2 or 3 weeks I want to share it now for any that finds the same issues I did. I am using CPanel and no docker or any other thing but a simple Linux hosting. Another thing, this is for Mautic 5, not previous versions. * * * # Detailed Guide: Configuring Mautic for Email Queue Management with Doctrine and Cron Jobs ## 1. Introduction to the Concept **What is Mautic?** Mautic is an open-source marketing automation platform that enables you to engage with your audience through various channels such as email, SMS, social media, and more. It’s a powerful tool for managing marketing campaigns, segmenting your audience, and tracking engagement. **What is Doctrine?** Doctrine is an Object-Relational Mapper (ORM) for PHP, which is used by Mautic to manage database operations. In the context of this guide, Doctrine also serves as a transport layer for the Symfony Messenger component that Mautic uses to queue and process tasks, such as sending emails. **Why Use a Queue System for Emails?** Sending large volumes of emails directly can overwhelm your server, leading to performance issues or being flagged as spam by recipient email servers. A queue system allows you to process and send emails in smaller, controlled batches, which is more efficient and reduces the likelihood of your emails being marked as spam. **The Importance of Sending Emails in Small Batches** By sending emails in smaller batches, such as 60 emails every 10 minutes, you: - _Reduce server load_: Prevent the server from being overwhelmed by processing too many tasks at once. - _Improve deliverability_: Lower the chance of being marked as spam due to sending large volumes in a short time. - _Increase reliability_: Ensure that your email campaigns are delivered consistently and within acceptable time frames. **Introduction to Lock Files and Their Purpose** A lock file is used to ensure that only one instance of a process runs at a time. This is crucial when dealing with cron jobs that may overlap. By using a lock file, you can prevent multiple instances of the email sending process from running simultaneously, which could cause conflicts or excessive resource consumption. ## 2. Setting Up Mautic with Doctrine for Email Queues **Option 1: Configuring via `local.php` File** 1. **Step 1: Locate the `local.php` File** The `local.php` file is typically located in the `app/config` directory of your Mautic installation. This file contains the configuration settings for your Mautic instance. 2. **Step 2: Edit the Configuration File** To configure Mautic to use Doctrine as the queue transport, you need to edit the `local.php` file. Add or modify the following settings: 3. **Step 3: Save the File** After making these changes, save the `local.php` file. These settings will now be applied the next time Mautic processes a queue. **Option 2: Configuring via Mautic’s Queue Configuration Screen** 1. **Step 1: Access the Queue Configuration Screen** 2. **Step 2: Configure the Queue Settings** **Explanation:** By setting the _Queue Transport_ to _Doctrine_, you’re instructing Mautic to manage queued emails using Doctrine’s database storage. This allows Mautic to handle email processing more efficiently, especially under heavy loads. ## 3. Implementing Cron Jobs for Automated Email Sending **Creating the Shell Script** 1. **Step 1: Create the Script** Place the script in a suitable location, such as `/path/to/mautic/var/lock/consume_mautic.sh`: 2. **Step 2: Make the Script Executable** Give the script execute permissions: **Configuring the Cron Job** 1. **Step 1: Open the Crontab** Use the following command to edit your crontab: 2. **Step 2: Add the Cron Job** Add the following line to your crontab: **Setting Up Directories and Permissions** 1. **Step 1: Create Directories** Ensure that the necessary directories exist and have the correct permissions: 2. **Step 2: Check Permissions** Make sure that the user running the cron job has the necessary permissions to read/write in these directories. **Testing the Setup** 1. **Step 1: Run the Script Manually** You can run the script manually to ensure it works as expected: 2. **Step 2: Check the Logs** Review the log file to ensure that the emails are being processed: ## 4. Monitoring and Troubleshooting **How to Monitor Logs** Regularly check the log file (`mautic_consume.log`) to monitor the performance of the email queue and troubleshoot any issues that arise. **Common Issues and Solutions** - _Lock File Not Being Removed_: Ensure that the script is properly configured to remove the lock file after processing. If the lock file remains, it could prevent future executions. - _Emails Not Sending_: If emails are not being sent, check the configuration of your SMTP server and ensure that the Mautic email settings are correctly configured. ---
**Author:** ![larsmattia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/larsmattia/32/13016_2.png) [@larsmattia](https://forum.mautic.org/u/larsmattia)\ **Post date:** [August 14, 2024, 11:39am UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/2 "2024-08-14T11:39:02Z")
Screenshot of cofngiruing in the web frontend: ![grafik](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/0/068f76b413bd3d13b40f4a3af3d6ad7c66e56912.png) There is also [Queue settings](https://docs.mautic.org/en/5.x/configuration/settings.html#queue-settings) in the docs. ---
**Author:** ![larsmattia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/larsmattia/32/13016_2.png) [@larsmattia](https://forum.mautic.org/u/larsmattia)\ **Post date:** [August 14, 2024, 11:53am UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/3 "2024-08-14T11:53:36Z")
is the `bin/console mautic:messages:send` cronjob still required after enabling the queue? ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 2:47pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/4 "2024-08-14T14:47:16Z")
> [@rcarabelli](#): > > _Enable Queueing_: > > - Set the _Queue Mode_ to _Enabled_. This ensures that emails and other tasks are not executed immediately but are instead placed in a queue for later processing. I don’t see such an option in the Queue Settings ☹ ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 3:05pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/5 "2024-08-14T15:05:43Z")
Good point, that’s an error, will rewrite that part. It’s only cog \> configuration \> queue settings The fill Scheme with “doctrine” and Host with “default” ![image](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/7/76f21c09cc6da7e7659233db1b802b8113fda940.png) ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 3:07pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/6 "2024-08-14T15:07:34Z")
I have it exactly like that but it doesn’t seem to work… ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 3:08pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/7 "2024-08-14T15:08:29Z")
No, you have to remove that cron. The only cron needed is the one that will trigger this new file you are creating. BTW this is only for mautic 5 ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 3:13pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/8 "2024-08-14T15:13:36Z")
you created your file with your files location in it, chmod it and then tested running it and it don’t work? Testing it in the terminal should give you this ![image](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/a/a9b2bbe3c9ebc0303f25bd88b2afc0f3f6967037.png) ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 3:19pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/9 "2024-08-14T15:19:57Z")
I just saw that the emails are correctly being processed by `php /var/www/html/bin/console mautic:broadcasts:send`. They get marked as sent but they never actually do get sent… I see exactly what your showing here BTW. What I couldn’t find is this config option: > [@rcarabelli](#): > > In the _Queue Transport_ dropdown, select _Doctrine_ ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 3:34pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/10 "2024-08-14T15:34:30Z")
So as far as I can see, the message queue is always empty… I don’t understand why… ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 3:39pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/11 "2024-08-14T15:39:12Z")
Have to read more times after asking chatgpt to make my writing style better. The idea is just to write doctrine and default ![image](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/7/76f21c09cc6da7e7659233db1b802b8113fda940.png) > I just saw that the emails are correctly being processed by `php /var/www/html/bin/console mautic:broadcasts:send`. They get marked as sent but they never actually do get sent… This part you are talking about puzzles me. That is something that Mautic is programmed to do. Have you set up your email account in configuration \> email settings? ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 3:51pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/12 "2024-08-14T15:51:43Z")
I just realized I didn’t have `MESSENGER_TRANSPORT=doctrine://default` in my `.env`. I added that and things seem to be moving forward. I saw some movement inside the `messenger_messages` table but no emails sent yet ☹ ---
**Author:** ![mchojrin](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mchojrin/32/10159_2.png) [@mchojrin](https://forum.mautic.org/u/mchojrin)\ **Post date:** [August 14, 2024, 5:24pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/13 "2024-08-14T17:24:11Z")
Well… I think I kind of figured it out. At least I saw some emails being sent 🙂 I was going a little crazy so I updated my local dev environment to use my gmail credentials and tried to replicate the scenario and guess what… it worked! So I went ahead to see what was going on in prod. I disabled the cronjobs to run everything manually. First I noticed that when I run `php /var/www/html/bin/console mautic:broadcasts:send --limit=50 --batch=20` I got 50 new messages inside the `messenger_messages` table. But then I noticed something unusual: the messages would disappear after a couple of seconds without me running any new command. This had me puzzled for a while but then I thought, since the cronjobs are stopped, there must be another consumer somewhere around… I took a closer look at the `docker-compose.yml` definition and noticed I have two separate services: `mautic_cron` and `mautic_worker` (I got this from an example found [here](https://github.com/mautic/docker-mautic/blob/084bcee5cc963379a8c519fe2fad4df8eeb37615/examples/fpm-nginx/docker-compose.yml)) but I’m not sure I actually need both of them. That’s something I still need to figure out. Anyway, once I stopped the `mautic_worker` the messages stopped disappearing magically. Bingo! I’m still not sure why those messages consumed by the worker wouldn’t get sent but hey… things look good 😉 Thanks for your support and I hope my story helps others. ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 5:40pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/14 "2024-08-14T17:40:03Z")
Will test that to add it to the guide ---
**Author:** ![larsmattia](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/larsmattia/32/13016_2.png) [@larsmattia](https://forum.mautic.org/u/larsmattia)\ **Post date:** [August 14, 2024, 6:05pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/15 "2024-08-14T18:05:42Z")
the containerized mautic uses a container that just runs “messenger:consume” for email, hit and failed. This is the supervisor config used: [docker-mautic/common/supervisord.conf at 084bcee5cc963379a8c519fe2fad4df8eeb37615 · mautic/docker-mautic · GitHub](https://github.com/mautic/docker-mautic/blob/084bcee5cc963379a8c519fe2fad4df8eeb37615/common/supervisord.conf) The cron container creates this cronjobs if the file doesnt exist yet: [docker-mautic/common/entrypoint\_mautic\_cron.sh at 084bcee5cc963379a8c519fe2fad4df8eeb37615 · mautic/docker-mautic · GitHub](https://github.com/mautic/docker-mautic/blob/084bcee5cc963379a8c519fe2fad4df8eeb37615/common/entrypoint_mautic_cron.sh#L7-L14) It configures the doctrine queue for email and hit by env: [docker-mautic/examples/basic/.mautic\_env at 084bcee5cc963379a8c519fe2fad4df8eeb37615 · mautic/docker-mautic · GitHub](https://github.com/mautic/docker-mautic/blob/084bcee5cc963379a8c519fe2fad4df8eeb37615/examples/basic/.mautic_env#L10-L11) ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 6:21pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/16 "2024-08-14T18:21:30Z")
Can’t give an opinion on that because I don’t use that. This is for simple installations. My case is on CPANEL using Softaculous, and for that I needed a different approach, so did this guide that is working for me. It can be refined or corrected by the ones that know mautic more than me, but for now, after 3 weeks from migrating from Mautic 4 to 5, I can send emails again. ---
**Author:** ![dirk\_s](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/dirk_s/32/748_2.png) [@dirk\_s](https://forum.mautic.org/u/dirk_s)\ **Post date:** [August 14, 2024, 8:11pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/17 "2024-08-14T20:11:57Z")
We will need to further extend the script, as most mail services are rate limited. Very often AWS SES limits e.g. to 14 mails/sec and just starts to drop anything above that limit. Sending via API its very likely to hit that limit. I’m right now still testing - with my script, that records the time for sending X (=14) mails and calculates how long to wait until the next 14 mails can go. I fill the queue up to 800 mails per cronjob in this example, thats why it will take up to 60 cycles (1 sec.) by 14 mails to send them. You can define the batch for all the calls - size them as you like them. Have a look here - comments and feedback is welcome! [https://twentyzen.com/en/think/faq-items/mautic-v5-cron-job-with-rate-limit-for-mail-sending/](https://twentyzen.com/en/think/faq-items/mautic-v5-cron-job-with-rate-limit-for-mail-sending/) That script also runs the important mautic tasks in sequence, so no need to run them one by one, with a cronjob for each - one will be enough to trigger the whole script. You can run it every minute… if you have a fast enough server. But in my example, the maximum amount of mails to be send in 1 min is 840 mails. Given that you have other jobs to run as well, 1 minute may often be to close to start the next round. It has some locking, so it doesn’t start the next one until the one before is finished. ---
**Author:** ![rcarabelli](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/rcarabelli/32/6807_2.png) [@rcarabelli](https://forum.mautic.org/u/rcarabelli)\ **Post date:** [August 14, 2024, 8:36pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/18 "2024-08-14T20:36:09Z")
Well that sounds interesting. I have no big issues because is my own dedicated server and have 1 IP por domain. But your point is very valuable. Maybe we can do a symfony middleware to delay the sending of each message. I guess it can be also achieved by a plugin and apart from the delay per message (like 1 second delay per message) that can also prevent sending like too many emails form the same delay in the same period of time, like only 1 gmail recipent each 10 seconds. It sounds plausible and would run in parallel on how the messages are sent (the guide I made for example). ---
**Author:** ![dirk\_s](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/dirk_s/32/748_2.png) [@dirk\_s](https://forum.mautic.org/u/dirk_s)\ **Post date:** [February 2, 2025, 8:12pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/19 "2025-02-02T20:12:49Z")
Update - newest version is in this github project: > **[GitHub - twentyZen/mautic-cronjobs: This script and environment file helps to manage...](https://github.com/twentyZen/mautic-cronjobs)** > > This script and environment file helps to manage cron jobs for Mautic. ---
**Author:** ![buggersee](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/buggersee/32/13477_2.png) [@buggersee](https://forum.mautic.org/u/buggersee)\ **Post date:** [February 4, 2025, 2:24pm UTC](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118/20 "2025-02-04T14:24:08Z")
Hello, I just checked your script. It looks great so far. But i have a question. Wouldn’t it be better to also set the --limit arg for the broadcast:send ? Now it only sets the --batch. E.g. If we use a batch size of 1020, without the --limit arg it would default to 100, no? So it would only send 100 mails everytime the script runs. Or do I have something wrong? `COMMAND_BROADCASTS_SEND="mautic:broadcasts:send --limit=$BROADCASTS_SEND_LIMIT --batch=$BROADCASTS_SEND_LIMIT|true"` Wouldn’t this be better to set both the same? Greetings, [Next page](https://forum.mautic.org/t/a-small-guide-to-send-mails-using-doctrine-for-queue-in-mautic-5/33118.md?page=2)