Mautic sending email is too slow

Support
#1

I am using Mautic 4.2 and Amazon SES which has currently 14/sec max. send rate. However, I am not being able to send more than 7/sec of emails using queue. I have to be ready for future which might need about 300/sec.

  1. I have checked that infrastructure is not a problem as I cant see spikes on CPU or Memory or Network etc in Mautic server
  2. I am using cron jobs to send emails but to test emails/sec I directly run command in console as below
    a. Firstly I populate queue in spool using campaign rebuild and trigger
    b. Count total no. of emails in queue say 1000
    c. Run command to send email say 800 per minute
    /bin/php /var/www/html/mautic/bin/console mautic:emails:send --time-limit=60 --message-limit=800
    d. Count total no. of remaining emails in queue and it is around 500 to 550

So that is the problem there, I was expecting Mautic to complete 800 per min as in command but something is wrong there. Can you please help me with this problem?

Also I tried to run same command with diff. locks which exceeds max send rate, However, it does not reach 800 per min anyway.

Any help is appreciated, thanks

#2

Hi, check this forum post:

#3

Thanks @joeyk for reply.

I looked into the post you referred but as mentioned in Optimize part, I try to send 790 with below command but I find that Mautic is not able to send 790 per minute, it just sends 500 per minute

app/console mautic:emails:send --message-limit=790 --time-limit=60

Can you please help here?

#4

The reason:
Your server is too slow. You’d need to investigate the bottle neck. (DB, memory, CPU?)
At the same time I would be cautions with sending so fast. If you exceed the limit, you lose emails.
Joey

#5

Thanks for getting back to me

We already checked our server CPU, memory and network which is ok.
As long as we are close to SES max send rate, we don’t worry sending to fast.

Can you please suggest anything else that is delaying sending emails? Is multithread with locks only way?

Have a good day

#6

Server CPU /memory showing OK values when you run HTOP doesn’t mean that you fully utilize it. You need to tweak your DB and PHP settings in a way to speed up send.
Mautic 5 will speed up email sending by 3,5x. I’s wait for that couple of months.
Joey

#7

Thanks @joeyk for replying.

I agree with you that we need to tweak our PHP setting and DB. Can you please let me know what is setting I need to look into?

Excited for new release already.

Have a great day

#8

Hi, I can’t give you optimization advice as I don’t know your environment. I suggest you post a job in the forums.
I hope someone can solve it. Joey

#9

Hi @joeyk

I applied multi-threading to send emails and got success to achieve a faster send rate like with one thread is 10/sec and I have applied 5 threads which gives 50/sec throughput.

However, I have found that there is an issue with duplicates. Same email gets sent multiple times when multiple threads are used.

Can you please help here?

Command below:

/bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock-name=“1” & sleep=2 & /bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock-name=“2” & sleep=2 & /bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock-name=“3” & sleep=2 & /bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock-name=“4” & sleep=2

#10

Hi, try to set limits according to your cron cycle. And run this command every minute with a time expiry of 45 sec.

#11

Thanks @joeyk for your advice. I set time limit of 45 sec and message limit of 300 but there are still duplicate emails in the inbox. Can you please review below command? Thanks, appreciate your help

/bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock_mode=flock --lock-name=“1” --message-limit=300 --time-limit=45 & /bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock_mode=flock --lock-name=“2” --message-limit=300 --time-limit=45 & /bin/php /var/www/html/mautic/bin/console mautic:emails:send --lock_mode=flock --lock-name=“3” --message-limit=300 --time-limit=45

#12

Can u plz check what happens if you keep 15 sec break between the 3 threads?

sleep 15

#13

@joeyk , Thanks for your suggestion, I tried this and ended with duplicates again. Then I investigated a little bit and I found that when using multi threading, some threads collide with each other and they give some warning that there is no file *.sending and thus results in failed message.

These failed messages are then picked up again as configured in system. So, I configured not to retry failed message and wipe them out. Then there are no duplicates now. I know that this is not 100% accurate solution as there might be true failure sometimes that we want system to resend again.

Anyways, I am still surprised at why multithreading is not working as expected even though I have added lock_mode=flock.

Thanks for your help, appreciated