(I previously put this on the Mautic Slack channel but haven’t had a resolution yet)
It’s taking me about 6 minutes to send 1,400 e-mails… I have a list of about 600,000+ that I want to hit in the next month, so at 230 contacts per minute my send time will be about 43 hours. I’m on a self hosted, AWS EC2 instance… is there a way to speed up send processing? I am using SES, which supports a very high throughput, so I am pretty certain that Mautic is the bottleneck.
I am using Mautic 1.x
Mautic v 1.x?? or v2.x.x
(I previously put this on the Mautic Slack channel but haven’t had a resolution yet)
It’s taking me about 6 minutes to send 1,400 e-mails… I have a list of about 600,000+ that I want to hit in the next month, so at 230 contacts per minute my send time will be about 43 hours. I’m on a self hosted, AWS EC2 instance… is there a way to speed up send processing? I am using SES, which supports a very high throughput, so I am pretty certain that Mautic is the bottleneck.
I am using Mautic 1.x
I had this issue too, also with EC2 and SES (though with Mautic 2.1.1), and logged this bug report. Looks like there may be a known issue causing it, but I haven’t had much more information than that so far.
In my case, yes — I’m using a t2.micro instance. (Wondering whether this might be a factor, I posted more details here, asking what the recommended minimum requirements are.) Is that problematic, and if so, what size machine would you recommend?
FWIW, CPU usage is generally at around 10%, with spikes up to 40%. Here are all the CloudWatch metrics:
The problem with the sending speed on Amazon SES is the implementation using SMTP. We have exact the same problem and it is not related to the instance type. Amazon offers an HTTP API to send emails through SES, but that does not solve the problem. I have implemented the sending through that API, but the sending speed did not increase. It takes about 100-200ms for each email to be processed.
The problem is mainly the fact, that the Switchmailer spool calls the send function of your transport sequentially. This means you can’t transmit emails asynchronous and really leverage the power of the Amazon API.
To solve that problem you have to create your own logic to process the local filesystem spool. I made a quick and dirty implementation of that (just a proof) and the sending speed is incredible. But as a said, that means you have to bypass Switchmailer completely which is kind of ugly.
Interesting, thanks MxyzptlkFishStix. Sounds like you’re getting much better performance using the Vultr box, even though its basic specs look worse than the EC2 t2.micro (which has 1 vCPU and 1GB memory)!
wittwerch, that’s too bad. I think I read that an upcoming version of Mautic (3.0?) is due to have SES API support and hoped that would much improve things. (For the case where all emails are the same, i.e. no fields merged in, you would think that would be incredibly easy — you could theoretically, I believe, send a single API call.) I guess that was overoptimistic!
Sorry, I’m just now circling back on this… It doesn’t seem to matter how large my instance in AWS is, the sending speed still crawls.
Wittwerch, any chance I could get my hands on that proof you made that bypasses the local filesystem pool?
Just bumping this thread, as it is a very important issue/feature for me.
I have about 30 days before I need to start hitting my 600k+ list.
Bump
Bump
Yes would be great to have a solution to it as soon as possible, we can’t send any emails at this stage either.
@martinstrom I did get a very encouraging response on Mautic Slack that pointed me to this thread… it seems that they are very close to a final solution:
Bad news it seems only to be for the bounce and complaint management. 
@wittwerch is correct about multi-threading using Amazon API - I formerly used http://sendy.co which had a fast sending speed as a result of using Amazon API instead of SMTP.
I’m currently comparing the sendy code logic with what can be done with mautic. There is a class developed as part of sendy that uses that logic at sendy/includes/helpers/class.amazonses.php
wittwerch published his proof of concept code in this thread:
https://github.com/mautic/mautic/issues/1740
At assessment stage will seek the help of a developer once logic is reached.
It looks like it didn’t exceed 20% (averaged over a 5 minute period).
Fyi;
-
I enabled the APC module in php.ini
-
I switched from “send immediately” to “queue”
-
during my sending process I bumped things from 100 per batch to 1,000
I can now send about as fast as my little heart desires
[quote=16156:@caleb.hurd]Fyi;
-
I enabled the APC module in php.ini
-
I switched from “send immediately” to “queue”
-
during my sending process I bumped things from 100 per batch to 1,000
I can now send about as fast as my little heart desires[/quote]
Thanks I will test this
@martinstrom I also switched from MySQL to Aurora which seemed to help a lot.
@caleb.hurd my speed is little better now but still have problems that if there is a large sending in the queue the new leads that signs up do not get their emails until the spool folder is empty and that can take up to 24h when we have sent out a broadcast to the whole list.
Do you also have the same problem or is your speed good now so Mautic sends the emails in your spool folder really fast?