Mautic Email Queue processing,why do Email Sending Time is obviously different from arriving time of receivers Inbox

[johntech]

My Mautic version is:4.0.1
My PHP version is:7.4.26
My Database type and version is:5.5.68-MariaDB

My problem is:
Mautic Email Queue processing will cause email sending time to be very different from receiving time.

My settings are below. How should email be handled? Queue

Message limit for queue processing 2

Path to the email queue directory %kernel.root_dir%/…/var/spool

Recovery timeout for resending failed messages 900

Clear timeout for deleted failed messages 1800

0 0,12 * * * certbot renew --no-self-upgrade >/dev/null 2>&1

*/5 * * * * /usr/bin/php /var/www/html/bin/console mautic:segments:update >/dev/null 2>&1 #mautic-segments-cron

*/3 * * * * /usr/bin/php /var/www/html/bin/console mautic:campaigns:rebuild >/dev/null 2>&1 #mautic-campaings-rebuild-cron

*/3 * * * * /usr/bin/php /var/www/html/bin/console mautic:campaigns:trigger >/dev/null 2>&1 #mautic-campaigns-trigger-cron

*/4 * * * * /usr/bin/php /var/www/html/bin/console mautic:emails:send >/dev/null 2>&1 #mautic-email-send-cron

*/3 * * * * /usr/bin/php /var/www/html/bin/console mautic:broadcasts:send >/dev/null 2>&1 #broadcast send

*/15 * * * * /usr/bin/php /var/www/html/bin/console mautic:webhooks:process >/dev/null 2>&1 #mautic-webhooks-cron

0 0,12 * * * /usr/bin/php /var/www/html/bin/console mautic:reports:scheduler >/dev/null 2>&1 # generating reports

*/1 * * * * /usr/bin/php /var/www/html/bin/console mautic:email:fetch >/dev/null 2>&1 #fetching email from inbox

*/2 * * * * /usr/bin/php /var/www/html/bin/console mautic:import >/dev/null 2>&1 #import contacts in background

When Campaign emails are sent to 6 different emails at 09:00, Email inbox like below

First, two of the emails have arrived into inbox Time showing 09:04 at 09:04 my local time

the other two, time showing 09:04 at 09:08 my local time

The rest two, showing 09:04 at 09:12 of my local time

All the email sending time is the same for all receivers. This will be confusing for the receivers to see the email sending time is too outdated from their receiving time of inbox.

Anyone has met this issue?

[joeyk]

Hi, this is EXACTLY what you programmed in your crons, so this works how expected.

1. You are priocessing 2 emails at the time:

Message limit for queue processing 2

2. You are sending emails every 4 minutes (once they are added to the queue)

*/4 * * * * /usr/bin/php /var/www/html/bin/console mautic:emails:send >/dev/null 2>&1 #mautic-email-send-cron

So:

When Campaign emails are sent to 6 different emails at 09:00,

This is what happens:

9.00 the emails are added to the queue by this cronjob:

*/3 * * * * /usr/bin/php /var/www/html/bin/console mautic:campaigns:trigger >/dev/null 2>&1 #mautic-campaigns-trigger-cron

At 9:00 the spool folder is still empty, so there is nothing to send. Couple of seconds after 9:00 the emails will be there as the previous cronjob requires couple of seconds to fill up the spool. But since you are running the trigger and email send symultaniously, the email send cron will run before the emails are in the spool.

At 9:04 your email send cron will notice that there are a bunch of emails waiting to be sent. Hi can only send 2 (this is your setting - limit for message proccressing), so 2 emails are sent out as you mentioned:

First, two of the emails have arrived into inbox Time showing 09:04 at 09:04 my local time

Then 4 mintes passing again (since you are running your email send cron every 4 minutes.) And 2 emails go out again:

the other two, time showing 09:04 at 09:08 my local time

And this happens once again:

The rest two, showing 09:04 at 09:12 of my local time

If you would like to learn more about how email sending works, check this video:

This Is How Email Sending Works In Mautic

[johntech]

Thanks for your explanation. I have watched your video several times before. Every 4 minutes Mautic will send 2 emails. This is no problem. The issue is that the time on the recipient is different from the actual time going into their box. For this one"The rest two, showing 09:04 at 09:12 of my local time", I want to show my sending time 09:12 at the recipient box at 09:12 of local real-time. Is my Cron Job not correct? How to fix this? I will test again.

Also, I have updated this cron job at the not same time as the Campaign trigger. But the time issue is still like before.
Old:/3 * * * * /usr/bin/php /var/www/html/bin/console mautic:broadcasts:send >/dev/null 2>&1 #broadcast send
New:/2 * * * * /usr/bin/php /var/www/html/bin/console mautic:broadcasts:send >/dev/null 2>&1 #broadcast send

I attached my Gmail original information. You can see the issues from queue sending.

[joeyk]

Hi, so your problem is, that the email is sent at 9:04 but it arrives at 9:12 ? Is the long ‘delivery time’ the issue?

[johntech]

The actual delivery time is correct according to the queue. The Email is sent by the queue at 09:12 and arrives in the Recipient box at 09:12. But the Display time on the email(created at ) is a very long time at 09:04. Do I explain well this issue? Maybe you can test this feature. It will be weird for receivers for the time issues.

[joeyk]

The email is created by the trigger cron (or broadcast schedule cron) at 9:04, but leaves the spool only at 9:12 initiated by the send cron. it spends 8 mintes in the spool collecting virtual dust

[johntech]

This will be a big problem if we send too many emails. I have tested other automation platforms which don’t have such issues. Do you know any solution?

[joeyk]

Honestly I don’t understand what you are trying to acheive with limiting your email output to 2 emails / 4 min.
With these settings you won’t be sending too many emails.
You are sending 2 / 4 min, so your hourly output is 30 emails…

Why are you limiting to 2 / execution? Why don’t you execute more often? Why don’t you batch execute?

[johntech]

With this testing, we can know if there is any problem with the queue sending. Batch execute can solve this issue? pls share me some guides to test. I want to make sending is no any future problem.

[joeyk]

Can you plz let me know how many emails you want to send per minute?

[johntech]

5 emails per minute. Currently I have 5K email list to send. I have to use the queue but met this issue

[joeyk]

Are you sending broadcast or this is a campaign trigger?

[johntech]

Firstly I Click"Campaigns", then “New” to create one Campaign to start at a certain time.

[joeyk]

Why not a broadcast? Are you continously adding contacts or you working with set lists?

[johntech]

@joeyk

broadcast

I still use the 6 different emails to test. “Firstly I Click"Campaigns”, then “New” to create one Campaign to start at a certain time." I think this way is broadcast, is it? If not how to do it? I will test the queue again

[joeyk]

Campaigns are workflows. When someone enters the campaign, then the workflow starts for this person. If you are sending mass email, then this is the wrong way to do it. You won’t benefit from the workflow’s features. It’s like shooting at a pigeon with a cannon.

You can use broadcasts on the other hand. It’s for sending a newsletter once.

1. You create an email in Channels - Emails - Segment email
2. Choose the segment, create your email and send it out.

In the background your broadcast cron will pick it up, and push the emails into the queue based n your setting. (limit it 2 / min if you want). This is beneficial for you, because the emails are created continously, and instead of stockpiling emails in your spool, you will send out the freshly produced emails.

[johntech]

Hi Joey I have tested the same result. I think the queue will always put email into the spool and create all emails at the same time.

[joeyk]

Did you restrict email creation to 2/cycle? Not sending but creation.

[johntech]

Do you mean this? I still limit for all email to send 2 email per 4 minutes

[joeyk]

Not at all. You are just editing the sending part, not the creation part. Your problem is, that emails are created at once. You need to make sure the emails are slowly created and slowly sent.

(Very few email software can do this, but Mautic can.)

This is how you do it:

1. Set broadcast cron to produce emails 2 / cron execution:
   mautic:broadcasts:send --limit=2

2. Set Email cron to send out freshly produced emails 2 / cron execution:
   mautic:emails:send --message-limit=2

3. Go to Channels > Emails > Create a segment email, PUBLISH your email

AND (Yes AND!!!)

4. Set the publishing window to a current or future date. Set unpublish window to a later date

Send out the email by clicking on send. The broadcast cron will do the actual creation of emails, and the send cron will send out the emails. My video above explains this in details, just rewatch if something is not clear.