Possible work around for reporting open and clicks without Bot data

Hi Guys

I guess that someone has posted this before but I couldn’t find it so thought I would share.

As you will be aware at the moment all reporting of opens and clicks currently delivers a number of false positives due to systems like Barracuda and Proofpoint opening them and clicking on the emails. while we can block IPs that we know are those used by these types of tool this will also remove any legitimate opens by the user.

having looked at the data I noticed that in most cases these opens/clicks happen almost instantly once the email has been delivered (within 1 minute if not seconds) of the sent time.

This got me thinking. through the report function there is an option to report on delay to open. If you add this option to the report and then add a filter that points to this fields and filter out any opens that happen within (say) 1 minute of the send I successfully managed to remove almost all of those bots from the report but still received data for those individuals who opened the email after it had passed through the receivers email filters.

It’s by no means perfect but thought I would share as it may help others having the same issue. Obviously, I am aware that this would also remove legitimate openers within 1 minute but in my experience, this is few and far between so providing me with better results.

If there are any other / better solutions out there I would be interested to learn how you guys are doing it.

I guess the ideal solution would be to add a setting in config that would allow you to filter out email bots by filtering out the 1st minute of known bot IP’s

thoughts?

Don’t worry about the crickets sound here, people are excited about your suggestion.
Thank you for sharing, it is a really great idea!
Joey

1 Like

Thanks, @harry5.

What about the effect of the false positives on campaigns? In my campaign, the trigger is the non-opened action, with the false-positive, all emails are considered as open, so the campaign is useless. Any idea to solve this?

I wish I did. The same solution could apply to campaigns if there was a option to apply the same logic in configuration that would work across the whole of the platform.

I’m fortunately I’m not a dev otherwise I would have looked to build a Plug-in to do it.

2 Likes

iOS15 is right around the corner. This will increase this situation…
Besides wrong statistics, I wonder what will happen to system load. Will Apple right away preloads thousands of images?

Thanks @harry5 and @dirk_s for sharing that.

The reporting of opens has always been an unreliable measurement. So I would not rely on that.

However, I rely heavily on click tracking. Luckily I did not see too many false positives there. What are your experiences?

1 Like

Those opens will be like the Dark Material in the Universe. You know its there, but can’t see it.

I think we need new metrics:

Sent | Opened | Masked | Clicked | Bounced | Complaint

Masked or Bot opens could be the ones, which open right away after sending.

Something similar was happening already in a smaller scale:
We have a client who is a bank. We have to send a daily report in email every day to 300 decision makers as PDF. The open rate is 99%. The spam filter picks up each email and fires the tracking.

J

2 Likes

You can see the opens by time:

select email_address, time_format(timediff(date_sent,date_read), ‘%H hrs, %i min.’) as timedifference from email_stats where date_read !=“NULL” order by timedifference

Would be cool to set a treshhold saying “okay, anything under 2 seconds is a masked open, which means might or might not have happened”.

2 Likes

Do you have experience with secure link clicking of those solutions?
Did you get false optins out of that? Usually we track clicks in mails as confirmation - as tracking a landingpage visit is unreliable, if people decide to “do not track” on the target landing page.

Sure. It now becomes even more obvious.

I’m more concerned about low powered Mautic instances, which will now be more likely be flodded with requests for images right after having sent the mail to many recipients. Wasn’t a problem before, when opening stretched across hours.

Definitely a situation to recheck some installations, if they can handle it.

1 Like

The click based double opt-in is anyway better as some people will open the confirmation email on the phone, and that disrupts the continuity on tracking.

The caveat here is that you can only do that with campaign emails… Would be great if we can have “clicked email” as a segment filter.

I also found this issue with relevant information on the topic: Email Protection Systems Generate Invalid Traffic · Issue #9798 · mautic/mautic · GitHub