I never did figure out why bounce emails were going into their own folders, making them inaccessible to Mautic’s bounce processing–but I did get the Amazon SNS topic working instead,
For anyone coming along after me, the trouble was this:
The Amazon callback webhook wasn’t running because my mail delivery was set to “Other SMTP” instead of Amazon SES.
Mail delivery was set to “Other SMTP” because I’d never been able to get the connection to work when selecting Amazon SES as the email type.
I now understand that the reason for that is that choosing Amazon SES disables the UI controls that let you select the port and crypto, and instead forces you (for some odd reason) to use the values hard coded in the AmazonTransport.php under app/bundles/emailbundle/swiftmailer. These values are set to port 2587 and TLS, neighter of which work–at least in my Amazon region. The saving grace is, you can edit the file and change the values to 465 and SSL, the correct values for my geography. Then email will work when configured for “Amazon SES” and thus, the Amazon webhook will run and successfully confirm the SNS topic subscription.
Unfortunately, I’ve yet to see bounce handling actually work, but perhaps I have some other issue now that the topic hitch is resolved.
I was getting an error when Amazon SNS called my Mautic Webhook because my Mautic installation was configured to send mail (through Amazon SES) using the “Other SMTP” config, rather than the “Amazon SES” selection.
That’s because I was unable to get it to connect to Amazon SES using the Amazon SES email type. And THAT’S because for some reason, when you select “Amazon SES” from the dropdown, it removes the port selection from the GUI and uses port and crypto configuration hard-coded in apps/bundles/emailbundle/swiftmailer/trasports/amazontransport.php (which is hard coded as port 2587 and TLS). My Amazon SES does not support either of those settings, but supports only SSL on either ports 25, 465 or 587 (and I’ve been using 465). Changing to port 465 and SSL got the webhook to run and confirm the SNS subscription.
So it wasn’t that my hosting environment had an issue, but that Mautic was trying to use settings other than what Amazon SNS & SES require (at least from my account). This is no big deal except that I had to hunt and hunt and hunt and eventually play a hunch to figure out the solution was just to edit the transport file.
I still haven’t gotten bounce processing to work, though I do have SES set to delivery bounces and complaints via the SNS topics. I’m still trying to figure out how to effectively debug the plumbing.
Hmm, sounds like that might be something we need to look into as a bug/enhancement for the Amazon integration. Do you have an account on GitHub? I’ll take a look and see if there’s anything reported already tomorrow!
Are running into the same issue as @zhammer on a 2.16.0 instance running PHP 7.3.
The SNS subscription confirms OK but Mautic is not picking up the JSON payload from Amazon. Amazon sends a “Delivery Status Notification” as a fallback, and the bounced email address is not marked as “Email Bounced” in Mautic.
I suspect the PHP version may be the issue.
Will try installing a fresh instance downgraded to PHP 7.2 and report back on the results.
Changing the port as @fabi states in the AmazonTransport.php file did not help for us.
Commented on the open Github issue for anybody who wants to follow along
After doing more digging today, this might be related to having CloudFlare in front of the Mautic instance that is having trouble receiving bounce/complaint notifications from SNS. Downgrading to 7.2 did not help. Haven’t had a chance to test whether its related to CloudFlare yet, but @zhammer by any chance, did you also have CloudFlare in front of your instance?
Edit: Disabled CloudFlare and tested against a fresh instance running PHP 7.2.23 and Mautic 2.16.0. Retrieved the SNS log from Amazon CloudWatch - looks OK from their end. But contact is still not marked as bounced.
Edit 2: What looked like a successful notification above wasn’t actually a bounce notification. It was just Amazon’s confirmation request. No new events were showing up in SNS when sending to the bounce simulator (or other non-existent inboxes).
Switching to a different Amazon SES region fixed the problem for us!
With eu-west-1 (Ireland) instead of us-west-1 (N Virginia), the bounce notification worked right away with the same instance that was having problems earlier. So it was actually Amazon failing to send the callback notification to Mautic, not a Mautic bug. Anyone having issues - try confirming your domain in a different SES region.
finally i managed a fully working SES-integration with Mautic !
E-Mail Sending & SNS-Webhook Bounce management is working.
i forgot to choose my SNS-Subscription in SES under E-mail-adresses > mydomain.com > Notifications > edit configuration > Bounces/complaints
I had it set up for Domains-tab, but i forgot to do it for e-mail-tab too.
I tried everything before including changing ports, changing the AmazonTransport.php file and different mautic settings, BUT i just forgot to setup this one thing in SES. Right now i’m using all the default mautic & server settings, so mautic should work out of the box with SNS-Webhook. I’m using 2.16.0 hosted on Digital-Ocean 5$ droplet managed with runcloud panel.
Under configurations in mautic i choosed “amazon ses”.
I tried AWS location Nord-Virginia & Oregon. Both work fine now.
Somehow i had problems at the beginning to get my webhook confirmed by Amazon (Nord Virginia). It was still in “pending verification” state. Changing the AmazonTransport.php file as stated above worked for me. But now i changed it back to default and it works fine…i don’t know why.
Oregon worked at first try, but at the moment i’m using Nord virginia again.
@fabi interesting! I didn’t realize there were different notification settings for emails directly. I had been setting notifications at the domain level. It’ll be interesting to see if this was my issue as well!
What’s the current mechanism for reporting a bug? This is definitely a bug, having Mautic assume the port and crypto settings are all the same when they clearly aren’t. It’s not a huge issue once you know the solution, but to have to go edit the files after every upgrade is less than sub-optimal.
Thank you so much for this. As someone new to the Amazon landscape I had no idea why bounces were not working when sent to the mailbox simulator. The default setting for the email address under SES overrides the domain setting. I thought it was odd that I was still receiving bounce email notifications in my inbox despite turning it off at the domain level.
I was still getting access log hits from Amazon SNS due to my domain setting - which apparently was a big red herring as you say.
I am not sure if this is obvious to those who used Amazon for a while, but I think the Mautic documentation should be updated to include this since it is probably how a lot of people will have it set up (verifying both the domain and sender email address).
For those running into bounces in separate folders (initially hidden) and Mautic not reading from these, take note of https://blog.cpanel.com/plus-addressing-in-cpanel/ that does this since cPanel & WHM v58
You’ll need to go into the cPanel mailbox setup to change this new default behaviour by mailbox (at least for the mailbox receiving the bounce emails) then all bounces got into one folder and that is then used in Mautic mail config (don’t forget the cron jobs to read mail and process segements)
There is another way of showing bounce in email graph.
Go to SES->domains->select your domain->Notifications->Edit Configurations->
check Include original headers -> save config.
Bounces will show up in email graphs