A small guide to send mails using doctrine for queue in Mautic 5

Product Support
21

Thanks for checking the script!

Interesting - does mautic:broadcast:send also have a --limit parameter? Need to check. If so we can easily add it…

1 Like
22

Yes - it looks like we need to define the --batch and --limit.
If I understand correctly, the batch should be:

  • 10 for SMTP
  • 50 for mailjet since API v3
  • etc.

This is the batch size - mails per call. With queuing, it will be pumped into the messenger_messages table and be no issue.

But without queue, we should set both parameters to useful numbers.
So if I run the queue each other minute, and have 14 mails / sec. and 40 loops to consume the queue… I may send up to 560 mails per cronjob run.

This means, I could set the broadcast command to 560 mails and batch them by 560 (if I wouldn’t want the queue to grow much more). But there is a reason there is the queue so we could load it a bit more and let the consumption run over time.

But true - we should also set the limit. Without queuing, the batch size needs to be adjusted to the service.

Do you agree?

This was the source, summarized by Perplexity:

Yes, the mautic:broadcasts:send command in Mautic offers both a --batch and a --limit parameter, although they are used differently in practice.

### --batch Parameter
The --batch parameter determines the number of emails sent in a single API call to the email service provider. This is particularly important because different email service providers have different limits for the number of emails per API call. For example, SparkPost allows up to 1000 emails per call, while SMTP typically only allows 10 emails per batch.

### --limit Parameter
The --limit parameter specifies how many contacts are processed per execution of the command. This means that during each execution of the command, a certain number of contacts are processed before the command is run again. The default value is 100 contacts per execution.

### Practical Use
In practice, the --batch parameter is used to control the number of emails sent in a single API call, while the --limit parameter controls the total number of contacts processed per execution. Both parameters can be used to optimize email sending and control server load.

Example of using both parameters:
bash* *php /path/to/mautic/bin/console mautic:broadcasts:send --batch=100 --limit=200* *
This command would process up to 200 contacts per execution and send 100 emails per API call.

It is important to note that the use of these parameters may vary depending on the version and configuration of Mautic. In some cases, the --limit parameter may not be explicitly supported, and instead, the --batch parameter is used to control the number of emails per API call.

23

Yes thats exactly how I understand it, too!

Last time I send a bigger segment mail I discovered and debugged this. I had a --batch defined but the broadcast still always used to send 100 mails per cronjob run. After researching a while, I found out what I wanted to achieve with the --batch arg was actually done with the --limit arg. I don’t see any default value for the --batch arg in the help command though. So I’m curious what happens when you only define the --limit. Is there a default value that is not documented or will it send it in one batch?

My current setup utilizes the queue system and your script.
My AWS SES limit is 19 mails/s.
In the .env I set EMAILS_PER_BATCH to 17 to have some buffer. (SES should support up to 50 per api call as far as I know?)
BROADCASTS_SEND_LIMIT is 1020 because my server should handle this.
So the MAX_LOOPS would be 60.

The only change I made was setting the --limit and --batch to the BROADCASTS_SEND_LIMIT value because I wanted to send in one batch. It seems to work fine, but it is not tested with a bigger segment mail yet (like 50k emails)

The script runs via supervisor, after the script finished there is a 2s delay, then it starts all over again…
With this config it should not go over the sending limit at all though. Especially since I also use ses etailors transport which probably has rate limiting in place already.
I can keep you posted if the next segment mail is sending like I expect it to.

Greetings,

1 Like
24

How does this work when using rabbitmq for the queue service?

25

how often do you set the cron job for?

26

I let it run every other minute.

27

Does this guide work on Mautic 5.2.2 and 5.2.3 ?

And will it be a standard feature of Mautic 6.x ?

28

Just a quick heads up here. Running the script with the etailors ses plugin was kinda spamming the amazon ses api, because whenever you consume the messages, the plugin will check your current account maxSendRate limit via api. It was fixed with a caching functionality (see Adding SES Api Call caching functionality. by BuggerSee · Pull Request #33 · pm-pmaas/etailors_amazon_ses · GitHub)
Also important to know is that the ses plugin will consume 1 DB queue entry every second. So you should not set the --batch higher than your maxSendRate else it will send faster than allowed.
I also put the actual consuming of messages in a separate supervisor thread for performance reasons and because the loop is not needed with the ses plugin + queue.

29

I don’t think I fully understood. Are you talking about the script in combination with symfony &
the plugin by pabloveintimilla?

What would need to be changed in the script? I’m happy to further improve it.

I didn’t run it with etailors.

30

Ah okay. No I was specifically talking about using the script in combination with the etailors ses plugin.

I dont use the plugin by pabloveintimilla, so I don’t know how it works to be honest.
But etailors’ plugin always fetched the amazon send rate everytime you call the consume command, so it was getting rate limited by the amazon api. (Because your script kinda calls it every second)

The pullrequest I linked earler fixed this issue with caching functionality though, so now your script is fully compatible with the etailors ses plugin, too!

31

Yes you can add it. Wrote this guide because my worker sometimes didn’t stopped and my server crashed regularly and is a very big server.
So the focus was specific for doctrine and to make sure the worker died when it should and no other worker could be triggered before the previous was first stopped.

Also, it should manage the broadcasts since mautic 5 using doctrine, all messages go to the temporary table before being sent and is the worker who manages the sending speed.
So the broadcast or the campaign trigger messages end in the same table, thus this script handles them the same.

32

rabbitmq as far as I know replaces doctrine, so I don’t really know, don’t have experience with that.

33

Any period you want, the lock mechanism in the file will prevent you creating new workers thus the sending speed will be always limited. I have it like at 5 minutes to fill gaps if the previous worker dies suddenly.

34

what version of mautic are you using?

35

Yes, that will send, if I remember correctly, the segment emails. Sort of. Will send them to the message_messages table to be then handled by the worker.

36

so far, yes

37

Thanks for sharing - script looks solid.

I have been facing issues lately with my set up which send duplicated emails to same contact up to 3 emails at same time and it is driving me crazy. i am tempted to use your script - i am using amazon ses and sending 14 emails per second is more than enough.

i just have one question - how to restrict sending windows to only certain days of the week and only between certain hours e.g only from Monday to Friday and only from 8 am to 6 pm

thanks

38

Hello Dirk,

Have a new better version, modified it to control the sendings per seconds on SES.

Here is the code:

/mautic_installation_dir/var/lock/consume_mautic.sh

#!/bin/bash
set -euo pipefail

#Lock file (50s)

LOCKFILE=“/mautic_installation_dir/var/lock/consume_mautic.lock”
LOCKFILE_TIMEOUT=50

PHP_BIN=“/usr/bin/php”
MAUTIC_BIN=“/mautic_installation_dir/marketautomation/bin/console”

#If lock is older than X seconds, exit.

if [ -e “$LOCKFILE” ]; then
LOCKFILE_AGE=$(($(date +%s) - $(stat -c %Y “$LOCKFILE”)))
if [ “$LOCKFILE_AGE” -ge “$LOCKFILE_TIMEOUT” ]; then
echo “Lock > ${LOCKFILE_TIMEOUT}s. Eliminando…”
rm -f “$LOCKFILE”
else
echo “Process already running. Exiting…”
exit 1
fi
fi

#Crear lock y asegurar cleanup al salir

touch “$LOCKFILE”
trap ‘rm -f “$LOCKFILE”’ EXIT INT TERM

#Cadencia y cortes

INTERVAL=2 # cada 2s (ráfagas)
PER_BURST=6 # 6 por ráfaga
HARD_LIMIT=50 # cortar a los 50s
SENT=0
START=$(date +%s)

while (( $(date +%s) - START < HARD_LIMIT )); do

#Procesa HASTA 6 por iteración; corta rápido si no hay más

“$PHP_BIN” “$MAUTIC_BIN” messenger:consume email --limit=“$PER_BURST” --time-limit=1 --memory-limit=1096M
((SENT+=PER_BURST))

#Si al dormir nos pasaríamos de 50s, cortar sin dormir

NOW=$(date +%s)
if (( NOW - START + INTERVAL >= HARD_LIMIT )); then
break
fi

sleep “$INTERVAL”
done

echo “Total enviados en ~${HARD_LIMIT}s (ráfagas de $PER_BURST c/ ${INTERVAL}s): $SENT”
exit 0

Sorry it’s in spanish (spanglish in fact)

This is my cron job:

/mautic_installation_dir/var/lock/consume_mautic.sh >> /mautic_installation_dir/var/logs/mautic_consume.log 2>&1

Don’t forget to chmod -x that file so it is executable

39

2nd answer:

Probably this version of the consume mautic file will send only Monday to Friday and only from 8 am to 6 pm
Test it, 95% sure it will work

#!/bin/bash
set -euo pipefail

########################################
# USER CONFIG – EDIT THIS SECTION
########################################

# Paths
PHP_BIN="/usr/bin/php"
MAUTIC_BIN="/mautic_installation_dir/marketautomation/bin/console"
LOCKFILE="/mautic_installation_dir/var/lock/consume_mautic.lock"

# Lock timeout (seconds)
LOCKFILE_TIMEOUT=50

# Working days (ISO: 1=Mon ... 7=Sun)
# Default: Monday–Friday
WORKING_DAYS=("1" "2" "3" "4" "5")

# Time window (24h format, server time)
# START_HOUR inclusive, END_HOUR exclusive.
# Example: 8–18 = from 08:00 up to 17:59
START_HOUR=8
END_HOUR=18

# Burst & run limits
INTERVAL=2       # seconds between bursts
PER_BURST=6      # messages per burst
HARD_LIMIT=50    # max seconds this worker is allowed to run per execution

# Messenger limits
TIME_LIMIT=1     # seconds for messenger:consume
MEMORY_LIMIT="1096M"

########################################
# DO NOT EDIT BELOW THIS LINE
########################################

# Day & time restriction
DOW=$(date +%u)   # 1=Mon ... 7=Sun
HOUR=$(date +%H)  # 00..23

# Check working day
IS_WORKING_DAY=false
for d in "${WORKING_DAYS[@]}"; do
  if [ "$d" = "$DOW" ]; then
    IS_WORKING_DAY=true
    break
  fi
done

if [ "$IS_WORKING_DAY" = false ]; then
  echo "Not a working day. Allowed days: ${WORKING_DAYS[*]}. Exiting..."
  exit 0
fi

# Check time window
if [ "$HOUR" -lt "$START_HOUR" ] || [ "$HOUR" -ge "$END_HOUR" ]; then
  echo "Outside allowed time window (${START_HOUR}:00–${END_HOUR}:00). Exiting..."
  exit 0
fi

# Lock handling
if [ -e "$LOCKFILE" ]; then
  LOCKFILE_AGE=$(( $(date +%s) - $(stat -c %Y "$LOCKFILE") ))
  if [ "$LOCKFILE_AGE" -ge "$LOCKFILE_TIMEOUT" ]; then
    echo "Lock older than ${LOCKFILE_TIMEOUT}s. Removing stale lock..."
    rm -f "$LOCKFILE"
  else
    echo "Process already running (lock younger than ${LOCKFILE_TIMEOUT}s). Exiting..."
    exit 1
  fi
fi

touch "$LOCKFILE"
trap 'rm -f "$LOCKFILE"' EXIT INT TERM

# Burst sending loop
SENT=0
START_TS=$(date +%s)

while (( $(date +%s) - START_TS < HARD_LIMIT )); do
  "$PHP_BIN" "$MAUTIC_BIN" messenger:consume email \
    --limit="$PER_BURST" \
    --time-limit="$TIME_LIMIT" \
    --memory-limit="$MEMORY_LIMIT"

  SENT=$((SENT + PER_BURST))

  NOW=$(date +%s)
  # If sleeping would push us past HARD_LIMIT, break without sleeping
  if (( NOW - START_TS + INTERVAL >= HARD_LIMIT )); then
    break
  fi

  sleep "$INTERVAL"
done

echo "Total planned in ~${HARD_LIMIT}s (bursts of $PER_BURST every ${INTERVAL}s): $SENT"
exit 0
40

Hey @rcarabelli

Thanks for your reply and your script - i will give it ago.

The reason i want to use something like this is to limit sending duplicated email to leads which happens a lot lately and it is frustrating.

Just to confirm i assume both script run on a command every minute correct ?

also are you using broadcast:send (i have seen on the forum people using it to control how many emails to be sent) - this part in Mautic is still not easy for new users.

Thanks