Segment email sent to same contacts multiple times

jgordon · December 13, 2024, 7:25pm

Your software
My Mautic version is: 5.1.1
My PHP version is: 8.2.24
My Database type and version is: 11.4.3 MariaDB

I am using Mautic to at the point exclusively send segment emails. I am using the etailors_amazon_ses plugin to send via SES. I am getting reports that recipients are receiving the same email 3-4 times on a send. Looking at the numbers for the email send in Mautic I see that 641 were sent, but in the SES dashboard I see 781 were sent.

i installed Mautic using the script from Mauteam. Sending email for our non-profit is only part of my job and so I don’t even remember why the email:send line is commented out as I set this all up a couple of months ago. While I normally would start troubleshooting and testing potential fixes, the only way to know if it is solved is to send out another email and so I would appreciate any help I can get on this one.

Here is the contents of my crontab file.

Watch out not to put nightly work during the day because of different timezones.

*/10 * * * * /var/www/html/var/lock/consume_mautic.sh >> /var/www/html/var/logs/mautic_consume.log 2>&1

Main cronjobs

0-59/3 * * * * sleep 0 && php /var/www/html/bin/console mautic:segments:update
1-59/3 * * * * sleep 0 && php /var/www/html/bin/console mautic:campaigns:update -n > /dev/null 2>> /var/log/cron/cron-error.log
2-59/5 * * * * sleep 0 && php /var/www/html/bin/console mautic:campaigns:trigger -n > /dev/null 2>> /var/log/cron/cron-error.log
#* * * * * sleep 0 && php /var/www/html/bin/console mautic:emails:send -n > /dev/null 2>> /var/log/cron/cron-error.log

   sleep 0 && php /var/www/html/bin/console mautic:webhooks:process -n > /dev/null 2>> /var/log/cron/cron-error.log

Old data cleanup

0 1 * * * sleep 0 && php /var/www/html/bin/console mautic:maintenance:cleanup -n --days-old=160 > /dev/null 2>> /var/log/cron/cron-error.log
30 1 * * * sleep 0 && php /var/www/html/bin/console mautic:unusedip:delete -n -l 1000000 > /dev/null 2>> /var/log/cron/cron-error.log

Imports and reports

0 2 * * * sleep 0 && php /var/www/html/bin/console mautic:import > /dev/null 2>> /var/log/cron/cron-error.log
0 3 * * * sleep 0 && php /var/www/html/bin/console mautic:reports:scheduler > /dev/null 2>> /var/log/cron/cron-error.log

Maxmind IP Lookup

0 4 * * * sleep 0 && php /var/www/html/bin/console mautic:iplookup:download -n > /dev/null 2>> /var/log/cron/cron-error.log
30 4 * * * sleep 0 && php /var/www/html/bin/console mautic:donotsell:download -n > /dev/null 2>> /var/log/cron/cron-error.log
0 5 * * * sleep 0 && php /var/www/html/bin/console mautic:max-mind:purge -n > /dev/null 2>> /var/log/cron/cron-error.log

joeyk · December 17, 2024, 8:32am

Hello, what does this really do?

*/10 * * * * /var/www/html/var/lock/consume_mautic.sh

Can you share the script? Is it possible that you run the consume script concurrently?

jgordon · December 17, 2024, 2:10pm

I honestly don’t know if I am running the script concurrently or not. Do you know where I should check to see if that would be happening and how I would go about correcting that?

Here is the script. I modeled it from here. Configuring Doctrine for Email Queue Management in Mautic with Cron Jobs - Mautic Knowledgebase

# Check if the lock file exists

# Path to the lock file
LOCKFILE="/var/www/html/var/lock/consume_mautic.lock"
LOGFILE="/var/www/html/var/logs/mautic_consume_detailed.log"

# Define the lock file timeout (9 minutes = 540 seconds)
LOCKFILE_TIMEOUT=540

# Function to log messages
log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOGFILE"
}

log "Script started."

# Check if the lock file exists
if [ -e $LOCKFILE ]; then
    LOCKFILE_AGE=$(($(date +%s) - $(stat -c %Y $LOCKFILE)))
    if [ $LOCKFILE_AGE -ge $LOCKFILE_TIMEOUT ]; then
        log "Lock file is older than $LOCKFILE_TIMEOUT seconds. Removing stale lock file."
        rm -f $LOCKFILE
    else
        log "Process is already running. Exiting."
        exit 1
    fi
fi

# Create the lock file
touch $LOCKFILE
log "Lock file created."

# Run the messenger:consume command for emails with increased verbosity and log the output
log "Starting to consume messages..."
/usr/bin/php /var/www/html/bin/console messenger:consume email --limit=40 --time-limit=480 --memory-limit=128M -vvv >> >

log "Finished consuming messages."

# Remove the lock file when done
rm -f $LOCKFILE
log "Lock file removed."

log "Script finished."

jgordon · December 30, 2024, 9:54pm

Here is the log file for the time frame during the last email send. Nothing stands out to me, but maybe this would help.

2024-12-13 14:00:01 - Script started.
2024-12-13 14:00:01 - Lock file created.
2024-12-13 14:00:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:08:04 - Finished consuming messages.
2024-12-13 14:08:04 - Lock file removed.
2024-12-13 14:08:04 - Script finished.
2024-12-13 14:10:01 - Script started.
2024-12-13 14:10:01 - Lock file created.
2024-12-13 14:10:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:18:03 - Finished consuming messages.
2024-12-13 14:18:03 - Lock file removed.
2024-12-13 14:18:03 - Script finished.
2024-12-13 14:20:01 - Script started.
2024-12-13 14:20:01 - Lock file created.
2024-12-13 14:20:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:28:03 - Finished consuming messages.
2024-12-13 14:28:03 - Lock file removed.
2024-12-13 14:28:03 - Script finished.
2024-12-13 14:30:01 - Script started.
2024-12-13 14:30:01 - Lock file created.
2024-12-13 14:30:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:38:03 - Finished consuming messages.
2024-12-13 14:38:03 - Lock file removed.
2024-12-13 14:38:03 - Script finished.
2024-12-13 14:40:01 - Script started.
2024-12-13 14:40:01 - Lock file created.
2024-12-13 14:40:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:48:03 - Finished consuming messages.
2024-12-13 14:48:03 - Lock file removed.
2024-12-13 14:48:03 - Script finished.
2024-12-13 14:50:01 - Script started.
2024-12-13 14:50:01 - Lock file created.
2024-12-13 14:50:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 14:58:03 - Finished consuming messages.
2024-12-13 14:58:03 - Lock file removed.
2024-12-13 14:58:03 - Script finished.
2024-12-13 15:00:01 - Script started.
2024-12-13 15:00:01 - Lock file created.
2024-12-13 15:00:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:08:03 - Finished consuming messages.
2024-12-13 15:08:03 - Lock file removed.
2024-12-13 15:08:03 - Script finished.
2024-12-13 15:10:01 - Script started.
2024-12-13 15:10:01 - Lock file created.
2024-12-13 15:10:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 60 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:18:04 - Finished consuming messages.
2024-12-13 15:18:04 - Lock file removed.
2024-12-13 15:18:04 - Script finished.
2024-12-13 15:20:01 - Script started.
2024-12-13 15:20:01 - Lock file created.
2024-12-13 15:20:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 40 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:28:03 - Finished consuming messages.
2024-12-13 15:28:03 - Lock file removed.
2024-12-13 15:28:03 - Script finished.
2024-12-13 15:30:01 - Script started.
2024-12-13 15:30:01 - Lock file created.
2024-12-13 15:30:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 40 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:38:03 - Finished consuming messages.
2024-12-13 15:38:03 - Lock file removed.
2024-12-13 15:38:03 - Script finished.
2024-12-13 15:40:01 - Script started.
2024-12-13 15:40:01 - Lock file created.
2024-12-13 15:40:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 40 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:48:03 - Finished consuming messages.
2024-12-13 15:48:03 - Lock file removed.
2024-12-13 15:48:03 - Script finished.
2024-12-13 15:50:01 - Script started.
2024-12-13 15:50:01 - Lock file created.
2024-12-13 15:50:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 40 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.                                             

2024-12-13 15:58:03 - Finished consuming messages.
2024-12-13 15:58:03 - Lock file removed.
2024-12-13 15:58:03 - Script finished.
2024-12-13 16:00:01 - Script started.
2024-12-13 16:00:01 - Lock file created.
2024-12-13 16:00:01 - Starting to consume messages...

 [OK] Consuming messages from transport "email".                                

 // The worker will automatically exit once it has processed 40 messages,       
 // exceeded 128M of memory, been running for 480s or received a stop signal via
 // the messenger:stop-workers command.                                         

 // Quit the worker with CONTROL-C.

jgordon · January 7, 2025, 8:43pm

I have it “solved” by setting the consume cronjob to run every 20 mins instead of 10 mins. I doubt this is an effective permanent solution, but does the job for now since my list is under 1k.

system · January 9, 2025, 8:43am

This topic was automatically closed 36 hours after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Troublshooting Multiple Email Sends to Same Contact ID Product Support	4	215	December 27, 2023
Emails Sending Problem - not sent to all contacts Product Support	6	25	December 4, 2024
Mail to segments sending only few Product Support	0	189	December 22, 2021
contact receives the same email several times Product Support	1	323	October 17, 2019
problem mautic email Product Support	2	226	January 13, 2018