Mautic: 5.2.2
PHP: 8.3
We’re only sending segment emails at this time, and to do this I’m using messenger:consume email cron to process my doctrine queue.
I have emails loaded in messenger_messages, but no matter what I try for --limit and --time-limit I can’t seem to get anything more than an average of .6 emails per second when calculating out how many are cleared over time.
I’m using DuoCircle as our SMTP relay, and trying to determine where the bottleneck is as I see no evidence of memory or cpu issues.
@dirk_s I’ve using your script, keeping the defaults that were set to 14 email per second, I’ve tried tweaking a few things as well, but can’t seem to get any difference in actual throughput regardless of settings.
I’m in the process of setting up a RabbitMQ option in case that will help, I have it running on a dev server but need to clear this queue first.
I guess I’m mostly just not clear on where the issue is, at this point I suspect it’s just the long handshake with smtp and it doesn’t seem to send any more than a single email in a batch at all.
Our previous system was able to send send different emails in parallel, but that doesn’t seem to be a solid option that I can see without danger of duplicate sends. The current queue of 33,000 emails is made up of 9 separate emails being sent to 9 individual segments. Is it possible to have each email use a separate worker somehow in parallel?
I do no have a dev site with rabbitmq to test out. I’ve noticed that queueing messages into rabbit only happens at 2.0/s so that’s pretty slow as well.
I ran a test to queue messages directly and got
Published 1000 messages in 0.022 seconds
Average: 45454.55 msg/sec
So, I’m still chasing bottlenecks here, wondering if my expectations are maybe not inline with what Mautic can do?
I was able to process a sample email of 1420 messages, I got the send speed using rabbit up to .9 per second, so only a slight improvement. My dev server has 12 cores and 64GB memory, so I don’t think this is a resource issue, as nothing is showing full utilization.
Any input welcome on what are fair expectations for mautic performance and where I might look to make adjustments.
Hi,
Maybe you should use mupltiple threads like
-
-
-
-
- php /var/www/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=1 --limit=2500 > /dev/null 2>&1
-
-
-
-
- php /var/www/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=2 --limit=2500 > /dev/null 2>&1
-
-
-
-
- php /var/www/mautic/bin/console mautic:broadcast:send --max-threads=15 --thread-id=3 --limit=2500 > /dev/null 2>&1
Hi @srdjan9791, thanks for the suggestion!
We just completed some testing over the last few days and I was meaning to come back and up date this as well -
I read some other posts that indicated that multiple crons would end up sending duplicate emails, so that made me a little nervous. I tried using some of the suggested scripts out there that manage things with a lock file and flock, but those weren’t increasing our send rate. I’m not sure if the duplication issue was only when using the standard queue settings?
We only send segment emails at the moment, so my understanding is that the only cron that works here is the messenger:consume email. boadcast:send doesn’t appear to read from the messenger_messages table? Or at least when I had that cron running, no emails went out.
Since I moved to rabbitMQ instead of the messenger_messages table, I started using supervisor to spin up additional workers. With this we were able to get up to 3 or 4 emails a second running 3 or 4 workers.
I would assume that I could accomplish the same things with the multiple crons you suggested with rabbitMQ and still avoid duplicates as well, but I like the additional control with supervisor, at least as we are still testing out Mautic.
1 Like