Mautic Community Forums

7,000,000 Spam Traps

I have a list of 7 million+ known spam traps that I’m scratching my head about putting to use in mautic.
Id like to discuss the best way to implement an email quality control mechanism within mautic or as a part of data hygiene before the data is uploaded.

One of my business interests is buying distressed companies or the intellectual property of distressed companies and as part of due diligence there is not a way currently to vet their email lists and practices. I want to protect the integrity of our mission to return these companies to profitability and thus far the expense of correcting bad list hygiene and list building best practices is astronomical.

Any Thoughts?

You are not clear on your objectives. You are sure all 7 million addresses are SPAM traps? Why do you want to email to SPAM traps then? You want to clean a list from SPAM traps? I guess you are aware that there are services to do that. So what is your exact question?

Good afternoon. I am positive that all the addresses are confirmed spam traps. My objective is to implement a process of using my own data from these lists to clean existing lists.

I dont know why anyone would intentionally email spam traps as you mentioned so I wont comment on that.
I am aware there are services that already do this but I have a couple of goals. Eliminate the expense of using 3rd party services, and integrate this as a function of Mautic.

OK, I understand now. You want to offer this as a service to check spam trap checking for email addresses. I dont know if it would be feasible to ship a list of 7 million email addresses with a mautic installation. Also that list of spam traps may change over time, not sure if you are planning to update it.

Speaking about list hygiene, there are probably additional tests that should be made to ensure that an email address is really valid.

Not quite sure yet on where you see your part in the picture. Maybe you could set up an email validating service with an API that allows sites to check whether an email is valid or not. Then there would be a need for a Mautic plugin that can connect to the API and query the validity of emails on the list, or even check email validity right on the signup form.

It would be surely possible to write a plugin that has a local list as well. The question is just how the maintenance of the list would be managed. And surely proper validating of an email involves much more than just spam traps. Not sure what would be involved to do this all within a plugin.

I was already thinking it would be nice to have a plugin to connect to an existing Email verification service. For example https://www.ipqualityscore.com/ . They are giving 5000 API requests per month for free. That could be good enough for most sites. If you have more than 5000 signups per month, you can probably afford to pay for their services.

The simpliest way to do this is:

  • create a small php script, that checks an email address like
    https://spamtrapdefuser.com?email=emailaddress&id=1234
  • create a campaign, that pings this address via webhook with the proper info
  • once you have the result, the script could simply tap into your Mautic via API and set a do not contact or a tag of your choice.
    I have something similar set up with one of the verification services API works pretty well. (Not cheap :smiley: )

@joeyk nice thoughts there.

We have a small spinoff business we created specifically due to the reason of high cost of email hygiene. I am playing with the idea of integrating this into Mautic and offering the community some extra special rates. Will keep you posted.

Offtopic: Back in my mailing days I used to pay $3.5 / month to use the https://haveibeenpwned.com/ api. If an email was a subject to bunch of hacks, I just automatically removed them from the list.

Here’s an idea and this is for manually cleaning large lists.

Step 1: Have a clean instance of Mautic with no contacts in it. Then you will upload all of your clients emails to this list - be sure to mark the lead source as the client name

Step 2: Now take your 7+ million list of emails. Create a custom field in Mautic and call it “Spam Trap” - In your excel file make a column with spam tram and set them all too YES.

Step 3: Upload your Spam trap list too Mautic. This will either MERGE the data with the existing emails from step 1 or it will create a new contact. Be sure to allocate lots of memory too your mautic instance and upload in the background with a cron. Be patient because sometimes it will time out but don’t worry because when the cron runs again, it will upload where it left off

Step 4: Create a segment using the filter lead source = client name from step 1.

Step 5: Download the segment of all your clients leads. You will now see that the leads that have spam trap marked as yes were merged with your client data.

Uploading 7 million emails will slow down segment creation BIG time.
But I would be interested to see how it goes.

Haha agreed. I have never tried an instance of Mautic with 7 million emails… However, if your server is set up properly it could probably handle it… I was under the impression you needed this solution just for “one off” cleans - hence my suggestion. If it is dynamic, then another option would be needed.