Filter out spam protection bot traffic

Hi

We are using Mautic and it is a very good fit for us. We send some emails to our potential customers and we want to give them points based on clicks they do on the email, so that we can focus on the potential customers that is really interested in our service.

Now there is a problem. Many of our prospects have e-mail systems that will automatically click links in e-mails. Probably in order to screen the pages and look for fraudulant behaviour etc.

Looking at the log on these contacts I can usually see that the mail is read and multiple page visits on our web site within 1-5 seconds. To get accurate read reports I believe it would be good to be able to filter out these kind of automatic bot reads / visits. I dont know if it is the e-mail client that does this when the email is first read, or if it is done in advance.

Anyone have similar experience? Anyone know more about how these e-mail systems that automatically read e-mails, and clicks on links work?

2 Likes

Hi,
There are different providers causing different issues for marketer.
Apple is pre-popping your tracking images leading to fake opens. Clicks are being clicked as well, which is a problem.
You can make a report in Mautic, where you see the open delay afetr email is sent. That might help. Or check if someone had multiple opens, that can be also a solution.
This is a new territory, everyone tries to figure out the best way to mitigate the problem.
We have a client, who sends 1000 emails / day ONLY to bank employees. We enjoy a 100% open rate every single time


2 Likes

Very interesting indeed!
Hope we over time can fingerprint these.
Maybe they come from certain ip ranges in some cases. Or we can use certain headers from the http clients they use. Is it possible to see the raw heqders of a request when someone opens an email?
Maybe we could add honeypots to the e-mail?

Would tracking visits to the page rather clicks on the link be a solution? Or would whatever is checking the link also record a page hit?

Assumes one can add the mautic tracking script to the target page
Also means tracking other than that coming via the email link would be reported on.

We tried to change the outcome if we think a bot is opening an email, but it lead to lower inboxing rate.
I think Mautic will come up with a solution sooner or later, but you need a coder for that. If you have someone who can code in symfony, I can help you how to detect those proxy opens.

Activecampaign does it:
image

With 10% out of 26% sounds bad, but remember, that around 20% of that 10% would open probably as well. So that brings down the total to 18-20% open rate, which is actually your industry average.

I wouldn’t base my marketing on opens, and luckily Mautic offers a bunch of tools to track people on the website.

Yes we also get a page hit for these. So apart from having a large amount of different ip addresses at the same time and visiting multiple pages within a few seconds. I have not found any other way to distinguish this bot traffic.

Interesting @joeyk

I wonder though if I am the only one who sees these “bots” to also click on links in the e-mails?
We are already just using web page visits to track interest because of these bot opens, but it does not work for us either since we are getting robot clicks as well.

Kind regards
Jens

Hi @robm it won’t work either. As tracking the visit on the page requires opt-in. The opt-in rate is maybe 40-60% - as the base for your tracking the visits.

A client of mine has a tracking rate of about 20% if he doesn’t cloak Matomo as he has quite a few corporate clients that block all this with firewalls.

The rate of blocked tracking (either consent, ad block or firewalls) depends heavily on the target groups imho.

I agree.
Banks and financial institutions have different scanning methods.
For couple of client I’ve set up apache level blocking of any visitor who doesn’t have an ‘agent’ specified.
That removed bots by 90%

2 Likes

Hi guys,

We sorted this out for a large corporate client who’s emails targeted other large companies.

These large companies tend to run their own email servers.
They protect their email servers and their internal email clients with email screening software.
These aren’t bots, but anti-malware filters, however the effect is similar, they follow (click on) every link in an email to scan whatever is at the end of a link. Depending on how advanced the anti-malware filter is, it might click on the links just once for any number of emails, or it might click on every link for every recipient (no cache) (most observed behavior).

The scan tools of the second kind can completely ruin your email statistics and drive crazy any marketer, so this client considered it was important to get them removed from statistics and we built our plugin to do just that.

If anyone wants to know more about our plugin, just send me a DM or contact me via email at yosu.cadilla@gmail.com

Cheers!

In the meantime, is there a reasonably reliable way to get the problem under control?

Today’ speach will be covering it incl. code.
“How to Prevent duplicate submission of a Mautic Form - Case Study and Technical Discussion”
from Leo Schuler.

2 Likes

Does anybody know when the conference videos will be uploaded to YouTube?
Or can anybody share the knowledge from the conference?

Ruth is on leave currently but would be the right person to answer the question of when / if the mauticon 2023 sessions will make it to youtube.

1 Like

I have the same problem; we send thousands of emails every week and always receive some clicks from Microsoft and Google IPs, all at the same time.

I don’t know if it will work, but I’ve just added the Microsoft IPs (40.94.27.0/24) to the “Miscellaneous Settings/List of IPs not to track contacts”. I haven’t heard anyone mention this field, but I think it’s the solution. I’m trying it out, so when I get some results, I’ll come back to share them with you guys.

1 Like