Spambots abuse Mautic Forms

Product Support
1

This post is more a warning to the community and also to gather ideas how to avoid spam abuse of forms.

Your software
My Mautic version is: 4.1.1
My PHP version is: 8.0
My Database type and version is: MariaDB 10.4

Your problem
My problem is:
One of our client had over 400.000 signups over night by a spambot and they used all of his email quota to send spam messages.

There seems to be a spam trend at the moment!
Spammers are looking for the placeholders in optin forms and then fill in their message into i.e. firstname and lastname. Thanks to GDPR everybody sends then an automated confirmation message.
And the personalisation within this message is abused to show a CTA and a link instead of the name of the recipient (which didn’t even signup).

Steps I have tried to fix the problem:
We inserted a honeypot into the forms.

I made a video to describe the problem and how to build a honeypot (as described in the mautic documentation): This is how spambots can sign into your double optin & how to stop them - mautic tutorial - YouTube

2 Likes
2

Hello Alex, Maybe use Capcha?

1 Like
3

We just discussed the topic in length in the DACH (DE/AT/CH) Meetup, recording is in #germany on Slack.
Bottom lines:

  • It’s a real problem (for Mautic as well as the rest of the world)

  • It’s a tough problem

  • “personalisation within this message” → needs to be sanitized!

  • For that reason we should offer a solution in Mautic (e.g. a new form field type “Text without Link”, or at least a regex-based validation for “Text” fields). Ideally this would be a regular JS-based validation PLUS a server side removal of malicious things (Idea: This could be a form action! And in any case, it would have to run first, prior to e.g. “send email to contact”)

  • If a submission fails due to validation, or if the entry is thrown entirely away server-side: fine!

  • Else, Form Spam would still be bad, because it not only gives a bad impression for the recipients, but some will flag the Spam and that will destroy your rating at the email providers

  • Thus the Captchas etc.:

  • Honeypots or simple math tasks don’t help with serious attack tools (e.g. XRumer)

  • Google ReCaptcha works pretty well (v3 is even user friendly) BUT it is not an option where GDPR has to be honored

  • FriendlyCaptcha can cost money and is not 100% as good, but may well be good enough. And is GDPR compliant

  • In Mautic, we should offer an easy way to use either solution with a form

  • Other means include low-level thiings like IP-based denial of entire countries, fail2ban, 
 (only where applicable, of course)

3 Likes
4

@ekke , @joeyk Super interesting thoughts. I saw that it was the topic of the last DACH Meeting. Didn’t had time to attend :frowning:
Anyway. I think I am going to look into one solution after the other.
I thought already that it won’t be too difficult for a serious spammer to figure out the honeypot


5

Non-EU person here, so forgive me, but how is Google ReCaptcha NOT GDPR compliant?

Would you care to explain? THanks,

6

In my opinion pretty much nothing Google is GDPR compliant. But I really wanna @ekke diplomatic answer.

7

Well, you (responsible or the website) are giving Google personal data (IP address, if nothing else) PLUS they intensively track your users behavior. All that outside of GDPR-land and with no other accepted legal foundation. Even worth: Under the US cloud act.

Larger organization have been forced to get rid of such things and other (e.g. cloud-hosted web fonts) for quite a while already.

I agree with @joeyk , we’re currently seeing a drift towards GDPR taken more seriously by courts (Schrems II, and just a few days ago some new ruling on Cookiebot and Google Analytics that may turn into a nasty problem for Google).

In many cases website providers try to heal this by asking the user for consent. But even that will change to the worse going forward.

And really: If a user does NOT give consent (which is the majority), what does that mean?

  • You don’t allow them to fill in a form?
  • Or you skip the reCaptcha? (I doubt that the Bots typically klick the consent button :wink: )
8 Likes
8

Hi Alex / hey guys

You are right we also experience some serious problems with form spam. We managed to find some solutions for this problem which i like to share with you

  1. We added a little proxy in front of the /form/submit Endpoint and check if the form submit sends any kind of cookies. This works really well because most of the spammers don’t send any cookies at all

We log all submits and review them regularly. This also helps to find problems with the mautic setup on the pages (missing cookies could also be a sign of cross domain problems)

You could advance this technic by force setting cookies and check for there existence.

  1. We generally block TOR traffic from all of our servers which prevents a lot of spam traffic. Most of the spammers try to use tor as a cheap solution to hide there tracks.

There are other services which provides ip lists to block with known spammers. They could also be usefull to prevent a lot of form spam.

  1. The mautic honeypot form ist nor working well. What works well is the following approach: we inject javascript in the form which added a timestamp value in a hidden form field.

We check then if the value is present. If not we don’t send the email (we did this with a mautic plugin but you could do similiar things with a campaign)

You could also do variants with that by using mautic it self. For example set a mautic tag on each page visit and check if the form sender got this tag. If not it might be a bot.

By checking the lead history with a script you could quit often check if the user is a real user and if you should allow the submit.

  1. Joey is right: the google captcha spam also works well in most cases but is a dsgvo problem for us. So we dont use ist. But it might be helpfull to check other captcha providers as well

For me this one looks promising: Friendly Captcha - Spam vermeiden. Datenschutz gewÀhrleisten.

It is a captcha solution which is also invisible and where the user solves a crypto puzzle.

  1. You could add some server side rate limiting by using nginx rate limit function. This prevents submitting the same form with the same ip

  2. We use a technique to check for the message length in the form submit and prevent the submit if it is not longer than x words (less than 4-5 words works really good for most spammers)

Most form spammers don’t use long messages to prevent anti spam email filters.

  1. Mail Server
    Of course you could use the anti spam mail filter to prevent that a lot of doi mails are send. The mailserver would check the outgoing mail and triggers an spam warning if any suspicous content is found.

Greetings
Sebastian

4 Likes
9

One thing i forgot to say:

For everybody with a typo3 background: in my experience alex kellner did a quit good anti spam detector in the powermail extension:

This could also be a good starting point for a code solution.

Greetings
Sebastian

10

Thank you for the insights, this is really useful.

I’m also working on a solution and so far what I noticed 2 things, which I’m sure is not valid for 100% of the cases. But after checking 30 of our clients who had similar issues there were 2 things in common:

  1. None of the spam entries have page visits.
  2. None of the spam entries have user agent.

Based on these 2 you can already create campaigns in Mautic, which cleans up this code.
Right now I’m running an experimental script every minte which cleans up these fake entries. I was also thinking to block “no user agent” submissions from the Apache, so they see a 301 when they try to access the website. (Similar to no cookie solution). I don’T use Nginx, so I can’t say how it’s done.

1 Like
11

Hey Joey

In my experience it depends on the kind of form spammer you have. I would say 70-80 % are normal script form spammer which just fetch the form once and then send it in a loop.

This kind of spammers don’t have recorded page visits and they don’t have javascript running (because it is a simple script). They just fetch the form fields once and then fire and forget.

The once I’ve seen got user agents but got no page visits.

The worst case of form spammers you could have are the ones which are using headless browsers.
They got cookies, the execute javascript and are not different to any normal user out there.

The last ones you could only fight with the anti spam content techniques. Since headless browsers are slower and require more ressources on spammer side, these are not so common.

In the end they all seem to send really “spammy” messages and these are quit often easy to detect with heuristics.

1 Like
12

Hi together,

after our last DACH (DE/AT/CH) Meetup ekke already talked about, i implemented a kind of “form-submissions-spam-detector” for our mautic customers. Perhaps you find this way also usefull for your installation(s), until mautic has a built-in solution:

I created a small bash-script (because i do not have php-knowledge), which does a mysql-query every 10 seconds, to check how many form_submissions were created in the database during the last 60 minutes:

select count(*) as number from form_submissions where date_submitted >= DATE_SUB(NOW(),INTERVAL 60 minute);

The bash-script checks, if the number is greater than for example 200. If yes, then the bash-script does this mysql-query:

UPDATE forms SET is_published = ‘0’;

After this query, all mautic forms are unpublished, so a bot is not able to send in more mautic form spam.

In the same minute i get a push message form the script to my martphone, about the affected mautic installation. So i can directly have a look if this is really spam or only a very successful marketing campaign :slight_smile:

With a slow spam-testscript using curl i created for testing, i was able to send more than 600 mautic-form-submission within 10 seconds, befor my “spam-detection-script” unpublished all mautic forms. Not perfect, but much better than if my mautic installations send out some 100k mails during night, like Alex reported.

In my opinion it would be very cool, to have something similar direct in mautic. For example an option to set a “maximum form_submissions during the last X seconds / minutes” before mautic unpublishes a form for security and sends out a mail to the mautic admin. The form_submissions count could be made directly after each form submissions.

Perhaps it would be also nice, if mautic creates a textfile with the ip-adresses, which send in unnormal many form_submissions. This textfile could be used for example to block more submissions from these ips using fail2ban or a firewall at server level.

Matthias

2 Likes
13

How did you do these two things? Especially the first one?

14

Moin Matthias

Aus meiner Sicht gliedert sich das Problem in 2 Bereiche:

  1. Verhindern das ein Spam Bot die Reputation des Mailservers ruiniert, in dem immer wieder DOI E-Mails verschickt werden, allerdings unter weiterem Betrieb des Formulares fĂŒr normale Kunden

  2. Die Eintragung von Spam Leads in Mautic und das Problem der Bereinigung

FĂŒr uns ist 1) das grĂ¶ĂŸte Problem und dringendste Problem. Die Bereinigung kann im Hintergrund erfolgen, wenn man den Spam Submit entsprechend kennzeichnet.

Wenn man einzelne IP Adressen hindern will das Formular immer wieder abzuschicken, dann kann man das mit einem Rate Limit und Nginx konfigurieren:

DafĂŒr legt man das Limit auf den Endpunkt /form/submit

Ähnliche Sachen kann man auch mit fail2ban machen. FĂŒr Apache gibt es hier anscheinend mod_ratelimit - Apache HTTP Server Version 2.4
(womit ich keine Erfahrung habe)

Die Lösung hilft natĂŒrlich erst mal nur primĂ€r, wenn man eine IP hat die Probleme macht.

Bei Nginx kann man aber auch den Parameter selber bestimmen, den er Rate Limiten soll. Daher IP, Cookie, URL 
 Ggf. fÀllt einem hier was spannendes ein. Z.b: Rate Limit pro Formular Id, pro IP, 


15

HI Peter

We use it in our software project but I extract the key points for you and wrote a little script. Here we go:

You could do it like this:

1) Config a rewrite for the form submit endpoint

In nginx you would do it like this:

location = /form/submit {
    try_files $uri /submit.php$is_args$args;
}

In apache it would be something with rewrite in the htaccess.

2) Configure a cron job which downloads the known tor exit nodes

( Please be carefull and do not download the list to frequently. They will block you quit quickly. )

I store the list in the /var/cache dir in the mautic document root. Feel free to use every other location

0 */6 * * * curl -o /var/www/beispiel.com/public_html/var/cache/torbulkexitlist https://check.torproject.org/torbulkexitlist >/dev/null 2>&1

3) Custom submit.php

You could use this little script to do your checks. For the script I assume that you got a ‘logs’ dir in the document root.

We log all requests (spam and non spam) for later analytics.

I just check for any cookies and if the user ip is associated with a tor exit node.

Please note: I just hacked this script together for you. Feel free to improve it. Maybe I will do a better script later. I got some more advanced integrations running, but don’t have the time at the moment to publish them.

But this little script is working fine.

Some further ideas:

Of course you could insert more advanced checks here. For example look for a specific cookie, look for honey pot fields in the post array 
 and so on.

You could also change the script that it will just change form fields if you think the user is a spam bot (for example fill a form field with the value bot or something and check that later)

You could also do some simple checks and look for the message length in form fields. Most spam bots try to insert very short spam because of the heuristic spam detection systems.

4) Review logs and clean dir

From time to time you should take a look into the logs dir and look for false positives.

You should also setup some sort of cronjobs which cleans the directory from time to time (older files) and access protect the directory maybe.

Greetings
Sebastian

4 Likes
16

Wow, thanks. I’ll dig into this post.

17

Hi Sebastian,

setzte fĂŒr unsere Mautic Installationen (auch) nur Nginx ein - deshalb Danke fĂŒr den guten Tip mit “Rate Limiting”. Soweit ich in der Doku aus deinem Link nichts ĂŒberlesen habe, kann man mit dieser Möglichkeit zwar ein Limit pro Zeit und IP-Adresse festlegen, aber die IP-Adressen mit der Überschreitung nicht gleich fĂŒr einige Zeit blocken, also an weiteren Requests hindern.

Habe dafĂŒr aber gerade folgenden Link gefunden, der beschreibt wie man die IPs aus /var/log/nginx/*error.log" mit “limiting requests” gleich an fail2ban weiter gibt, damit die IPs dann geblockt werden können: ​https://easyengine.io/tutorials/nginx/fail2ban/

Damit könnte man schon mal Spamer die nicht 1000nde verschiedene IP-Adressen nutzen schon mal weiter bremsen.

Auch deine Variante mit Weiterleitung an deine submit.php gefĂ€llt mir. Ich kann leider kein PHP, aber du könntest evtl. dein Script noch um meinem ursprĂŒnglichen Ansatz ergĂ€nzen. Also das du einen Request nicht nur auf Herkunft aus dem Tor-Netzwerk ĂŒberprĂŒfst, sondern eben zusĂ€tzlich, ob von der jeweiligen IP innerhalb der letzten X Minuten schon mehr als Y Submits vorhanden sind. Falls ja, könntest du diese dann wie Requests aus dem Tor-Netzwerk behandeln. Somit wĂŒrde fĂŒr normale Nutzer das Formular auch weiter erreichbar sein. UnĂŒblich viele Submits wĂŒrden dagegen erst mal nur im Logfile und nicht mehr im Mautic landen.

1 Like
18

Hi everyone! Thanks for this amazing input!
I try to digest all of these approaches and make a video on each of them asap. :slight_smile:

2 Likes
19

Just want to contribute to this thread by making people aware of hCaptcha, which works incredibly, protects privacy, and actually pays you money. Cloudflare switched over to them from Google which is now used in their challenge pages.

It would be great if Mautic added this as an option, it’s not a hard integration if you’re using Google’s system.

1 Like
20

HI MrFromGermany

Thanks for your feedback! I will try to integrate the ip based rate limiting into my approach but I think i will move it into a generic “anti spam mautic” plugin

I already take a look at the places in the mautic source code and thinking about how to implement this.
I think you could use the existing rate limiting module in mautic and re-use it in the plugin.

But in the moment i’m really busy so maybe I hack a proof of concept first in the existing approach


BTW: For a lot of sites you could also use the ip location as a criteria for spam filtering.

Regarding the rate limit:
I think you could not only use it to ban ip’s but also to protect mautic against traffic spikes
For example you could rate limit traffic which is not that important (page hits for example) to make sure that your form submit endpoint is working.

Another interesting approach would be caching specific post requests with nginx.

Like in this article

The idea would be to prevent people hammering with always the same data (for example page hit trackings)

greetings
Sebastian