Mautic don`t send emails

Product Support
1

Your software
My Mautic version is: 4.4.2
My PHP version is: PHP FPM 8.1
My Database type and version is: 10.3.34-MariaDB-0ubuntu0.20.04.1 - Ubuntu 20.04

Your problem
My problem is:
My mautic has a record of 1.2 million contacts. I schedule sending to the segment that contains 400,000 people, and after sending about 250,000 emails, mautic stops sending with the following error:

In ProcessEmailQueueCommand.php line 141:
                                                                      
   rename(): supplied resource is not a valid Stream-Context resource

Screenshot by Lightshot

WhatsApp Image 2022-09-22 at 13.55.12
WhatsApp Image 2022-09-22 at 13.55.121061×121 49.3 KB

These errors are showing in the log:

[2022-09-22 15:34:05] console.CRITICAL: Error thrown while running command “mautic:emails:send --bypass-locking --lock-name=envio-mv-8 --message-limit=100000 --time-limit=10000”. Message: “rename(): supplied resource is not a valid Stream-Context resource” {“exception”:“[object] (TypeError(code: 0): rename(): supplied resource is not a valid Stream-Context resource at /var/www/html/mautic-s2g/app/bundles/EmailBundle/Command/ProcessEmailQueueCommand.php:141)”,“command”:“mautic:emails:send --bypass-locking --lock-name=envio-mv-8 --message-limit=100000 --time-limit=10000”,“message”:“rename(): supplied resource is not a valid Stream-Context resource”} {“hostname”:“mautic-cron-broad-s2g”,“pid”:37783}

How can I fix this problem?

Best Regards,
Igor Barros.

2

Hi, are you using multiple lock names?

3

Hi Joeyk, yes, I run with multiple lock names:

          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-0 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-1 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-2 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-3 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-4 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-5 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-6 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-7 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-8 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-9 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-11 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-12 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-13 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-14 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-15 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-16 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-17 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-18 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-19 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-20 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-21 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-22 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-23 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-24 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-25 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-26 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-27 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-28 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-29 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-30 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-31 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-32 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-33 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-34 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-35 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-36 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-37 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-38 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-39 --message-limit=100000 --time-limit=10000
          • root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --bypass-locking --lock-name=envio-mv-40 --message-limit=100000 --time-limit=10000
4

Hey,
Someone plz correct me if I’m wrong…
But what happens here is:

  1. you run yur cron commands every minute.
  2. you try to send 100.000 emails in 1 command (with a time limit of 10000 seconds, which is 2.7 hours)
  3. you bypass locking, which means the lock name will be ignored.
  4. you run 40 threads.

So what happens is, that after 10 minutes you have 400 threads, after 1 hour 2400 threads, and the maximum is 2.7 hours, which is 6480 threads.

If I’m correct, then how is your server handling that? And if you are bypassing the locking, then how you are not sending the same message to the same contact when 2 threads are picking up the same contact? (since your are not locking them)

Or I’m misunderstanding something here?

5

Hi Joeyk,

Reading your reply, you are correct and we are wrong.
We did this wrong cron because we need to send 650.000 emails as fast as possible.

How can we do this send as fast as possible?

I have a dedicated server only to cronjobs:
Dual Intel XEON E5-2640 v4 40x2.4Ghz
256GB DDR4
2x 2TB SSD NVMe
10Gbps unmetered link

How can I perform the best cronjob lines?

Thanks,
Igor Barros.

6

Hello Igor !
This is part of the learning process, I made the same mistake as you did. I would suggest to go like this:

Understand how much you can send every minute. This will be the basis of your sending speed.

  1. Fill up your emails in the spool folder. Once you ready count the files:
    ls | wc
    This will give you the number of files.
  2. Launch send email command in your command line:
    php /var/www/html/mautic-s2g/bin/console mautic:emails:send --time-limit=50
    This will send emails for 50 seconds
  3. Check how many emails you’ve sent by counting the files again:
    ls | wc
    Let’s say this is 200 emails. So now you know that your sending speed is 200 / min on each thread.

2. Adjust desired speed. Your SMTP might limit your speed based on your reputation. Make sure you are not trying to send faster. Also: you need proper domain/IP reputation to send fast. If we assume, that is all good you can start scaling.

  1. Let’s assume your speed is the above identified 200/50 sec. You want to leave that 10 seconds leftover from the 1 minute intevall for the script to run. Maybe it’s better to work with 45 seconds, that depends on your server.

  2. If you need to send 650.000 emails in 2 hours, then you need to have a pretty strong infrastucture, with IP rotation and superb domain reputation. Let’s assume you have it. So 650.000 / 60 / 2 = 5416 / min, which ends up in 27 threads.

  3. You can add your 27 threads with the locking mechanism (don’t bypass plz) and sent the time to:
    php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-40 --time-limit=50
    Since the script only runs for 50 sec, you can relaunch it every minute.

Caution:
Make sure your feedback loop is working and people can unsubscribe. Don’t send unsolicited email.

I have no idea if your server AND your mysql settings AND your SMTP can handle it, that’s another discussion.

It’s always smarter to start slow and scale by watching the performance.

Best of luck!

7

Hey Joeyk!

Thank you very much to this support, with your cron, I was able to sent 650.000 emails in 31 minutes.
Taking care with SMTP, IP and resources limits…

Here is the final cronjob to send emails:

* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-0 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-1 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-2 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-3 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-4 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-5 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-6 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-7 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-8 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-9 --time-limit=50
* * * * * root php /var/www/html/mautic-s2g/bin/console mautic:emails:send --lock-name=envio-mv-10 --time-limit=50

About the cronjob to create the emails, can you please sugest the cronjob too?

Here is my details:
Contacts: 1.881.421
Last contact ID: 1.969.156

Here is my actually cronjob:

0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=0 --max-contact-id=40000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=40001 --max-contact-id=80000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=80001 --max-contact-id=120000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=120001  --max-contact-id=160000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=160001  --max-contact-id=200000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=200001  --max-contact-id=240000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=240001  --max-contact-id=280000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=280001  --max-contact-id=320000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=320001  --max-contact-id=360000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=360001  --max-contact-id=400000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=400001  --max-contact-id=440000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=440001  --max-contact-id=480000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=480001  --max-contact-id=520000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=520001  --max-contact-id=560000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=560001  --max-contact-id=600000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=600001  --max-contact-id=640000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=640001  --max-contact-id=680000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=680001  --max-contact-id=720000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=720001  --max-contact-id=760000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=760001  --max-contact-id=800000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=800001  --max-contact-id=840000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=840001  --max-contact-id=880000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=880001  --max-contact-id=920000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=920001  --max-contact-id=960000 --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=960001  --max-contact-id=1000000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1000001 --max-contact-id=1040000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1040001 --max-contact-id=1080000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1080001 --max-contact-id=1120000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1120001 --max-contact-id=1160000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1160001 --max-contact-id=1200000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1200001 --max-contact-id=1240000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1240001 --max-contact-id=1280000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1280001 --max-contact-id=1320000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1320001 --max-contact-id=1360000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1360001 --max-contact-id=1400000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1400001 --max-contact-id=1440000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1440001 --max-contact-id=1480000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1480001 --max-contact-id=1520000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1520001 --max-contact-id=1560000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1560001 --max-contact-id=1600000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1600001 --max-contact-id=1640000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1640001 --max-contact-id=1680000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1680001 --max-contact-id=1720000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1720001 --max-contact-id=1760000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1760001 --max-contact-id=1800000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1800001 --max-contact-id=1840000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1840001 --max-contact-id=1880000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1880001 --max-contact-id=1920000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log
0,30 * * * * root php /var/www/html/mautic-s2g/bin/console mautic:broadcasts:send --min-contact-id=1920001 --max-contact-id=2000000  --bypass-locking >> /var/log/mautic/cron-mautic-s2g.log

How can I perform this cronjob?
Thanks again.

8

Are you using Mautic page tracking javascript on your website?
Joey

9

No, I only use Mautic to send emails to our customers.

I receive the leads from forms or via API and work the leads on segments and campaigns.

10

Okay, they your script might be okay. If you’d had tracking the anonymous visitors would delute the ID pool, and your spooling would be uneven. But this seems fine.