Messenger:consume email is very slow

General Discussion
,
1

Mautic: 5.2.2
PHP: 8.3
We’re only sending segment emails at this time, and to do this I’m using messenger:consume email cron to process my doctrine queue.

I have emails loaded in messenger_messages, but no matter what I try for --limit and --time-limit I can’t seem to get anything more than an average of .6 emails per second when calculating out how many are cleared over time.

I’m using DuoCircle as our SMTP relay, and trying to determine where the bottleneck is as I see no evidence of memory or cpu issues.

@dirk_s I’ve using your script, keeping the defaults that were set to 14 email per second, I’ve tried tweaking a few things as well, but can’t seem to get any difference in actual throughput regardless of settings.

I’m in the process of setting up a RabbitMQ option in case that will help, I have it running on a dev server but need to clear this queue first.

I guess I’m mostly just not clear on where the issue is, at this point I suspect it’s just the long handshake with smtp and it doesn’t seem to send any more than a single email in a batch at all.

Our previous system was able to send send different emails in parallel, but that doesn’t seem to be a solid option that I can see without danger of duplicate sends. The current queue of 33,000 emails is made up of 9 separate emails being sent to 9 individual segments. Is it possible to have each email use a separate worker somehow in parallel?

2

I do no have a dev site with rabbitmq to test out. I’ve noticed that queueing messages into rabbit only happens at 2.0/s so that’s pretty slow as well.

I ran a test to queue messages directly and got
Published 1000 messages in 0.022 seconds
Average: 45454.55 msg/sec

So, I’m still chasing bottlenecks here, wondering if my expectations are maybe not inline with what Mautic can do?

I was able to process a sample email of 1420 messages, I got the send speed using rabbit up to .9 per second, so only a slight improvement. My dev server has 12 cores and 64GB memory, so I don’t think this is a resource issue, as nothing is showing full utilization.

Any input welcome on what are fair expectations for mautic performance and where I might look to make adjustments.