I recently had an issue that made me question my understanding of how Mautic email delivery works, and as I can’t find any documentation that confirms or refutes my supposition, I thought I’d post here:
What happened:
I’m configured to queue email delivery. I limit deliveries per run to a few hundred, and run the cron job every 15 minutes. I’m set to permit only one contact per day.
I clicked send on an email to several thousand people, but after the first few batches has run (been taken out of the spool directory and actually send to recipients) I started getting emails about broken links.
In order to minimize the damage, I stopped my cron job for email delivery and emptied the spool/default folder.
The broken links turned out to be a problem with my database schema, which apparently hadn’t updated successfully with the recent Mautic update, and correcting that fixed my links. So I didn’t need to change the email I just sent–but since there is no way to tell Mautic to resend an email (that I know of) I cloned it, gave the clone a new name, and sent the clone.
Nothing happened… In the web gui, the “send” progress bar crepts across the screen at a glacial pace as always, so I thought the email were sending, but when Amazon SES reported no activity after a while, I checked the spool folder and it’s empty. My email says that all the several thousand individual messages are queued, but the spool folder is empty.
I then unpublished my clone message and did the same thing again. Clones the original message in just the same way and sent it in just the same way (but after setting the “contacts per day” to 11) and this time it worked as I expect, filling up the spool folder.
SO MY QUESTION IS… What exactly is the workflow for sending email? What does it really mean when messages are “queued”? I thought that meant they were in the spool folder awaiting delivery, but obviously not.
Does that mean all those thousands of message are still queued somehow and if I only knew how, I could go delete them, but because I don’t, they are going to start delivering the (now duplicate) message at some point in the future, annoying a few thousand of my leads?
I’ve actually run into this problem (and reported it before) where emails weren’t delivered but were instead marked as “queued” but were not in the spool folder, so apparently I’m just misunderstanding the processing.
@cStuartHardwick I know this is a pretty old thread but I figure if no one has answered you yet it may still be welcome if you do eventually receive an answer. And this may be useful to others in the community who are wondering.
I’ve run into similar problems and I’m still figuring out some of the Mechanics myself. But some of what I know may benefit you.
Mautic has 3 ways of sending.
Direct without queueing (which is not ideal for SES or API usage due to high volumes per API call). This could work with smaller servers and smtp mail. The mail will be queued by the mail server in reasonable batches.
With a queue which saves unsent emails to the spool folder. AFAIK once the message is queued it is also tagged in the database as queued to be sent to the lead/contact. And this is update on send to avoid re sending the mail multiple times to the same person if it’s a campaign email. So deleting the queued files will leave Mautic thinking that the mail has already gone out.
Message broker on a separate server. This creates the queue on another server that sends out the mail via API or SMTP. This is ideal if you are doing higher volumes of mail.
Depending on how the message was tagged in your Campaign (Transactional / Marketing) the message will be allowed to go out once or multiple times.
Campaign messages are retried 3x on failure (bounce) by default. But adding a message with the same content and subject to a new campaign will not allow the message to go out again AFAIK. This info is stored that restricts this is stored in the database. And it is usually a safe way to avoid annoying people by repeating your email over and over accidentally.
I’m currently attempting to send such a message by Duplicating it as a Segment email. And setting up a segment to send this email to all users who have not read this email yet. This is a recommended way of re-sending such an email. I’ll let you know once it’s executed if it was allowed with the same subject line.
The issue I bumped into was that SES silently dropped some of the messages I sent them because I either sent too fast or something else. So my Mautic install thinks that the mail was sent because it left the Mautic server via API. When in fact those mails never got sent and there was no notification of the fact.
To work around this I’m using the unopened email segment. And I’ve switched to a more practical method of sending for the volume I’m sending at.
Hope this helps you or anyone else in the community who is wondering.