In short my question is: How can I mark email as failed if it fails one time and avoid trying the second and third time?
it was brought to my attention by email expert that if sending message via api fails because of status code (status code is something other then 2xx & 429), then email should be marked as failed and mautic should not try to resend it at all (not even once) and should not mark the lead with DNC record.
I have looked into the core and I do not see a way to get the logic above into the transport class without modifying the core.
I have also looked into existing api transport implementations (like Sendgrid and Amazon) and did not see any code that would make an effort to cover that.
I’m glad you brought this up. This question has been raised multiple times in the forum.
By using Sendgrid or Amazon the error codes land by these providers, and they foward it to you.
Some of the 4xx error codes should be repeated in my opinion:
421Service not available, closing transmission channel (This may be a reply to any command if the service knows it must shut down) 432 4.7.12A password transition is needed 450Requested mail action not taken: mailbox unavailable (e.g., mailbox busy or temporarily blocked for policy reasons) 451Requested action aborted: local error in processing 451 4.4.1IMAP server unavailable 452Requested action not taken: insufficient system storage 454 4.7.0Temporary authentication failure 455Server unable to accommodate parameters
These are all temporary errors, which should ‘count’ up to 3 and once you reach 3 automatically add DNC.
I believe you misunderstood the question. The problem is how to pass the information to mautic when sending fails during submission, not after it has been submitted and SMTP responds with a specific error code.
I would also like to point out that the SMTP codes can’t be interpreted in such a simplified manner as described for multiple reasons. First of all it’s the MTA that should do the retry attempts based on correct setup. Only once the MTA determines the message as bounced (soft or hard) and passes the information to Mautic there could be an additional action like DNC. In addition this count should be applied on 3 independent send attempts and Mautic should not attempt to resend the same message - that’s the MTAs job whether that MTA is used via API or SMTP. Second problem lies with the fact that the SMTP codes reliability is quite limited - the numbers are indicative but not reliable. For example the most common case being full mailbox. Some servers will respond with 45x error to tell the sending MTA to retry delivery later (hoping that some space will free up), others respond with 55x to indicate that the sending MTA should not retry sending. To make things even worse the 45x errors with “over quota” message sometime mean that the sending MTA exhausted it’s delivery rate quota and should slow down… as I’ve been in deliverability for over 15 years I have seen all sorts of crazy SMTP codes and messages and our current MTA config has about 300 special rules on how to treat various responses.
The problem @mzagmajster is dealing with relates to API submission errors rather than SMTP responses. In this case the plugin makes an API call to send message and API responds with 401 or 404 error which needs to be interpreted as EMAIL_FAILED rather than BOUNCED as it has never been sent in a first place.
Even if we considered SMTP submission the DNC logic seems off as it would disable recipients in case the SMTP rejects the message for lack of authorization (eg. sending address not being permited to relay). The DNC should only be applied to the actual result of delivery attempt not to submission attempt.
Yes, you are right, I share your view on everything including SMTP error codes.
Yes, I misunderstood the question. Since I’m using Amazon SES and POSTAL, luckly never have to deal with failed API attempts.
Looking forward if any improvement will be done here. Could be similar like with webhooks: mautic already has a feature where an email is sent when the webhook fails.
we are trying to catch the case where user enters invalid credentials or tries to send using a domain that is not authenticated… some of these cases would definitelly would result in failed API requests no matter what service is being used.
this is not about bounces but about rejects on submission. Bounces are working just fine it’s handling certain edge cases during submission (like misconfiguration or domain abuse attempts) we are trying to solve.
very much so. it’s not a matter of complexity but user-experience:
a) if you misconfigure your sending by mistake and send your campaign to a large list and it all fails and all contacts turn DNC you will have a headache
b) if a sender sets up the sending incorrectly and the campaign is sent using his email address as “From” and that address is not authorized he will be in trouble.
c) I don’t even want to go into the details of how this affects email security and opens the doors to email abuse.
So yes - having a distinction between submission failure and delivery failure is important.
@pangea I understand your arguments but I don’t agree.
For me it’s a matter to be seen in the Mautic interface beforehand.
It’s difficult to send a campaign by mistake, you have to validate 3 times in a row that you want to send an e-mail campaign
If the person who sets up the campaign makes a mistake, he will learn and will not do it again in the future.
It remains a software that requires a hand in hand and to know what we do.
The aim of the software is to accompany the use.
The fact that the user can see the problem doesn’t mean he will understand the problem.
As an example - right now you set up your campaign and you decide you want to send it using a different From address than the one that is authorized, eg. instead of @mybrand.com you change it to @otherbrand.com. While the SMTP/API service has the @mybrand.com setup the @otherbrand.com is not and hence the emails will either fail sending or will fail to be properly authenticated (DKIM/SPF/DMARC). In the first case it will result in DNCs, in second it will result in major reputation impact and bad deliverability. I would love if we could provide a warning in the UI before the send but I’m not sure there is such an option.
For my part, with Mailjet, if the e-mail cannot be routed because the @otherbrand.com domain is not configured, then Maimjet will block the sending and that’s all.
There will be no DNC on the Mautic side.
Thanks for the input folks, we seem to be getting a bit off topic so to summarise:
I think the fundamental issue is that if there is some problem with the API connection completely unrelated to the inbox - for example the email never left the MTA because the API limit was hit - it should not be marked as bounced or have a DNC set because the email failed to send, it didn’t bounce.
If your API limit is exceeded and you sent to 2 mil contacts they would all get marked DNC as a result, whereas it should really be an error which does not impact the bounce/deliverability.
I believe the problem is that we do not differentiate between ‘the MTA could not send this email’ and ‘the mail did not make it from the MTA to the user successfully’.
I have asked some of the developers who have been pretty deep in this area of Mautic to take a look and share their thoughts.
rate limits are perfect example - if a sending domain has a rate limit and that limit is exceeded the attempt to send has to be retried but after certain number of attempts it needs to be failed but not set as DNC. I thought of a workaround yesterday that I told @mzagmajster that if the swiftmailer core interprets the status codes as SMTP code even if they come from API, then he could return a “fake” code for certain API responses, eg. 550 for API authentication failure instead of passing back 429.
disclosure: I am not an email expert by any means, but I am working with mautic code base for quite some time.
But per my last investigation of this issue (and suggestion in the earlier post). I do not think its possible to enforce respect for api status codes during the webhook response processing (processCallback method to the extent that is required in this situation) because of some code in the core.
There is an event listener that essentially says “if failed retires is less then three try again else mark as DNC”. Until we have this peace of code in mautic core no mater what you do on plugin side or swiftmailer its not gonna bring the desired results.
I think that the most accurate representation of what I think needs to be revised in order to reach the requirement is in my earlier post.