How to process emails faster?

[fisch]

Your software
My Mautic version is: 5.0.4
My PHP version is: PHP 8.1.2-1ubuntu2.19
My Database type and version is: MariaDB 10.6.18-MariaDB-0ubuntu0.22.04.1

Please tell me how to increase sending speed! I currently send 180 emails per minute using doctrine spooling and this cronjob:

*/1 * * * * sudo -u www-data php /opt/mautic/bin/console mautic:broadcasts:send --channel=email --limit=170 --batch=170

A single systemd worker with messenger:consume processes these emails.

With this setup, all emails that are spooled into the messenger:messages table in the database are processed within one minute, before the next cronjob starts. However, this is not fast enough, I will need to at least triple the speed.

I have tried using multiple systemd workers with some success, increasing the speed up to 400 emails per minute. But using multiple workers will cause a database lock contention at some point, which leads me to assume that this job cannot be run multiple times.

My VM has 4 vCpus and 16 GB RAM. These limits are not even reached by half.

Thanks for reading

[ricfreire]

Hi @fisch

Is the email you’re sending optimized? Does it contain a lot of images?

Best regards,
Ricardo

[fisch]

Hey Ricardo,

thank you for your reply!

The email does contain a lot of HTML img tags, but the images are hosted on another server. Each email is about 140kb in size.

The emails are not optimized, at least not that I know of.

[srdjan9791]

Hi,
Try adding multiple treads. Example

• • • • • php /opt/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=1 --limit=2500 > /dev/null 2>&1
• • • • • php /opt/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=2 --limit=2500 > /dev/null 2>&1
• • • • • php /opt/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=3 --limit=2500 > /dev/null 2>&1

[serendipitytech]

@fisch

h some success, increasing the speed up to 400 emails per minute. But using multiple workers will cause a database lock contention at some point, which leads me to assume that this job cannot be run multiple times.

My VM has 4 vCpus and 16 GB RAM. These limits are not even reached by half.

Thanks for reading

@fisch I had a similar issue that I was chasing over at this thread

Messenger:consume email is very slow

Hi @srdjan9791, thanks for the suggestion! We just completed some testing over the last few days and I was meaning to come back and up date this as well - I read some other posts that indicated that multiple crons would end up sending duplicate emails, so that made me a little nervous. I tried using some of the suggested scripts out there that manage things with a lock file and flock, but those weren’t increasing our send rate. I’m not sure if the duplication issue was only when using the stan…

TLDR: I found moving to RabbitMQ instead of the default docrtrine queue and running multiple crons (I chose to do this through supervisor) increased our send rate very well.