New help with bounce management

Product Support
1

My Mautic version is: 2.16.0

My problem is: I cannot get bounce processing to work, now matter how I configure it.

If Mautic email configuration screen is set to send bounces to bounce@mydomain.com, then indeed they appear in my bounce email box, but they each appear in a unique folder (presumably named after the reply path created on the outgoing email) and so are ignored by bound management, which can only monitor a set folder.

If I remove this setting, I get no bounce notices at all.

However, if I set the bounce address under email config in Mautic, send a test email to a bogus address, and wait for the bounce to hit my bounce mailbox–then go to email config in Mautic and select the unique folder into which that bounce notice email appears, bounce processing works and Mautic will flag the test account with the bogus email “DO NOT CONTACT” just as it should. So bounce processing works–except there is no way to tell Mautic to monitor all possible inboxes, so it never sees bounce notices, and instead, they just build up in the mailbox.

I use Amazon SES to send mail and have an Amazon SNS topic set up to forward bounce notices via email to the same bounce mailbox (and it’s confirmed), but it has no impact on whether bounces are received and processed. This notification doesn’t appear ever do anything. Clearly, the bounce notices are being delivered via email according to the reply path header set in the outgoing email, not by the Amazon topic–so why does the documentation even tell us to set up the bounce and complain topics?

I tried configuring to use a webhook topic subscription instead, but as many, many, many people have reported over the last couple of years, this doesn’t work either. Amazon sends the confirmation request, but Mautic doesn’t reply, instead logging, a SwiftMail “you have requested an unknown service”.

This is very frustrating. ALL documentation I’ve found for bounce handling is very simple and straightforward–but none mentions the weird “stick each bounce in an inaccessible folder” behavior, and none seems to match the overall behaviour I am seeing in my very simple, straight out of the box installation.

Any light anyone can shed would be greatly appreciated. Thanks.

2

I never did figure out why bounce emails were going into their own folders, making them inaccessible to Mautic’s bounce processing–but I did get the Amazon SNS topic working instead,

For anyone coming along after me, the trouble was this:
The Amazon callback webhook wasn’t running because my mail delivery was set to “Other SMTP” instead of Amazon SES.

Mail delivery was set to “Other SMTP” because I’d never been able to get the connection to work when selecting Amazon SES as the email type.

I now understand that the reason for that is that choosing Amazon SES disables the UI controls that let you select the port and crypto, and instead forces you (for some odd reason) to use the values hard coded in the AmazonTransport.php under app/bundles/emailbundle/swiftmailer. These values are set to port 2587 and TLS, neighter of which work–at least in my Amazon region. The saving grace is, you can edit the file and change the values to 465 and SSL, the correct values for my geography. Then email will work when configured for “Amazon SES” and thus, the Amazon webhook will run and successfully confirm the SNS topic subscription.

Unfortunately, I’ve yet to see bounce handling actually work, but perhaps I have some other issue now that the topic hitch is resolved.

3

Hi @cStuartHardwick, thanks for coming back to us with an update and apologies for the lack of response to your original thread.

I think from reading your reply it sounds like perhaps the ports are not working on your server (maybe they are being blocked at your server level? Worth checking with your admin/hosting provider!).

Are you able to look into that and see if it is indeed a port blocking issue?

Also, what regions are you using? It would be helpful to know so we can figure out if this is a factor for others who are struggling to get the SNS setup working!

Thanks again for coming back with an update!

4

That’s close, but not quite.

I was getting an error when Amazon SNS called my Mautic Webhook because my Mautic installation was configured to send mail (through Amazon SES) using the “Other SMTP” config, rather than the “Amazon SES” selection.

That’s because I was unable to get it to connect to Amazon SES using the Amazon SES email type. And THAT’S because for some reason, when you select “Amazon SES” from the dropdown, it removes the port selection from the GUI and uses port and crypto configuration hard-coded in apps/bundles/emailbundle/swiftmailer/trasports/amazontransport.php (which is hard coded as port 2587 and TLS). My Amazon SES does not support either of those settings, but supports only SSL on either ports 25, 465 or 587 (and I’ve been using 465). Changing to port 465 and SSL got the webhook to run and confirm the SNS subscription.

So it wasn’t that my hosting environment had an issue, but that Mautic was trying to use settings other than what Amazon SNS & SES require (at least from my account). This is no big deal except that I had to hunt and hunt and hunt and eventually play a hunch to figure out the solution was just to edit the transport file.

I still haven’t gotten bounce processing to work, though I do have SES set to delivery bounces and complaints via the SNS topics. I’m still trying to figure out how to effectively debug the plumbing.

5

Hmm, sounds like that might be something we need to look into as a bug/enhancement for the Amazon integration. Do you have an account on GitHub? I’ll take a look and see if there’s anything reported already tomorrow!

6

Thanks so much CstuartGardwick !

I had the probem that my Amazon SNS Webhook to mautic was still pending. Changing the AmazonTransport.php file from “2587” and “tls” to “465” and “ssl” worked for me right away. Great.

I am using Amazon East, Nord Virginia.

7

I’m experiencing similar issues, but not sure how to track down any solutions as well.

Running 2.16.0 on a VPS - Setup Amazon SES using this info: https://www.mautic.org/docs/en/emails/bounce_management.html and the SNS topics are setup and such.

Right now I have the SNS topics set to send to both email and the Webhook so I can at least monitor the bounces happening.

Any progress on this would be appreciated here as well :slight_smile:

8

Are running into the same issue as @zhammer on a 2.16.0 instance running PHP 7.3.
The SNS subscription confirms OK but Mautic is not picking up the JSON payload from Amazon. Amazon sends a “Delivery Status Notification” as a fallback, and the bounced email address is not marked as “Email Bounced” in Mautic.

I suspect the PHP version may be the issue.
Will try installing a fresh instance downgraded to PHP 7.2 and report back on the results.
Changing the port as @fabi states in the AmazonTransport.php file did not help for us.

Commented on the open Github issue for anybody who wants to follow along

9

After doing more digging today, this might be related to having CloudFlare in front of the Mautic instance that is having trouble receiving bounce/complaint notifications from SNS. Downgrading to 7.2 did not help. Haven’t had a chance to test whether its related to CloudFlare yet, but @zhammer by any chance, did you also have CloudFlare in front of your instance?

Edit: Disabled CloudFlare and tested against a fresh instance running PHP 7.2.23 and Mautic 2.16.0. Retrieved the SNS log from Amazon CloudWatch - looks OK from their end. But contact is still not marked as bounced.

{
    "notification": {
        "messageId": "58460587-c4ba-4b19-9cb9-9fdee4ac3aaa",
        "topicArn": "arn:aws:sns:us-east-1:369433415525:Mautic_Example",
        "timestamp": "2020-03-08 05:28:23.158"
    },
    "delivery": {
        "deliveryId": "a6f86804-6da3-5e8a-bb83-3edae914cbcb",
        "destination": "https://mauticdemo.example.com/mailer/amazon/callback",
        "providerResponse": "OK",
        "dwellTimeMs": 898,
        "attempts": 1,
        "statusCode": 200
    },
    "status": "SUCCESS"
}

Edit 2: What looked like a successful notification above wasn’t actually a bounce notification. It was just Amazon’s confirmation request. No new events were showing up in SNS when sending to the bounce simulator (or other non-existent inboxes).

Switching to a different Amazon SES region fixed the problem for us!

With eu-west-1 (Ireland) instead of us-west-1 (N Virginia), the bounce notification worked right away with the same instance that was having problems earlier. So it was actually Amazon failing to send the callback notification to Mautic, not a Mautic bug. Anyone having issues - try confirming your domain in a different SES region.

image
image1182×363 32.2 KB

10

@autoize to answer you question, I do indeed have cloudflare in front of the domain.

I will try switching the SES location as well and will report back!

11

Hey guys,

finally i managed a fully working SES-integration with Mautic !
E-Mail Sending & SNS-Webhook Bounce management is working.

The problem:
i forgot to choose my SNS-Subscription in SES under E-mail-adresses > mydomain.com > Notifications > edit configuration > Bounces/complaints

I had it set up for Domains-tab, but i forgot to do it for e-mail-tab too.

I tried everything before including changing ports, changing the AmazonTransport.php file and different mautic settings, BUT i just forgot to setup this one thing in SES.
Right now i’m using all the default mautic & server settings, so mautic should work out of the box with SNS-Webhook. I’m using 2.16.0 hosted on Digital-Ocean 5$ droplet managed with runcloud panel.
Under configurations in mautic i choosed “amazon ses”.

I tried AWS location Nord-Virginia & Oregon. Both work fine now.

Somehow i had problems at the beginning to get my webhook confirmed by Amazon (Nord Virginia). It was still in “pending verification” state. Changing the AmazonTransport.php file as stated above worked for me. But now i changed it back to default and it works fine…i don’t know why.
Oregon worked at first try, but at the moment i’m using Nord virginia again.

AWS is not easy to setup for beginners. I highly recommend Kevin Rundles YT-Video: https://www.youtube.com/watch?v=9WJ3NPCjGsE&t=1008s
Thanks for your investment in the community Kevin !

12

@fabi interesting! I didn’t realize there were different notification settings for emails directly. I had been setting notifications at the domain level. It’ll be interesting to see if this was my issue as well!

13

@zhammer yes, exactly. It won’t work without the notification setting for emails directly !

14

What’s the current mechanism for reporting a bug? This is definitely a bug, having Mautic assume the port and crypto settings are all the same when they clearly aren’t. It’s not a huge issue once you know the solution, but to have to go edit the files after every upgrade is less than sub-optimal.

15

Thank you so much for this. As someone new to the Amazon landscape I had no idea why bounces were not working when sent to the mailbox simulator. The default setting for the email address under SES overrides the domain setting. I thought it was odd that I was still receiving bounce email notifications in my inbox despite turning it off at the domain level.

I was still getting access log hits from Amazon SNS due to my domain setting - which apparently was a big red herring as you say.

I am not sure if this is obvious to those who used Amazon for a while, but I think the Mautic documentation should be updated to include this since it is probably how a lot of people will have it set up (verifying both the domain and sender email address).

16

I love this forum and you guys :heart_eyes:
Just saved me a lot of time. Also in this case, email setting in Amazon SES was overriding domain setting…

17

For those running into bounces in separate folders (initially hidden) and Mautic not reading from these, take note of https://blog.cpanel.com/plus-addressing-in-cpanel/ that does this since cPanel & WHM v58
You’ll need to go into the cPanel mailbox setup to change this new default behaviour by mailbox (at least for the mailbox receiving the bounce emails) then all bounces got into one folder and that is then used in Mautic mail config (don’t forget the cron jobs to read mail and process segements)

18

There is another way of showing bounce in email graph.
Go to SES->domains->select your domain->Notifications->Edit Configurations->
check Include original headers -> save config.
Bounces will show up in email graphs

image
image816×318 33.9 KB
image

19

Can you confirm the region you are using?
I’m using exactly that configuration, with SNS validated but NO bounce and NO COMPLAINTS are managed in any way

20

Complaints are bounces in Mautic with a specific reason ‘complaint’.
There is no separate complaint designation, it’s just a hard bounce.