# How did happened? Duplicate contact

**URL:** <https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575>\
**Category:** Product Support\
**Created:** [June 24, 2022, 2:47am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575 "2022-06-24T02:47:43Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 4, 2022, 7:49am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/22 "2022-12-04T07:49:15Z")

</div>

Thanks @joeyk, I am not seeing any JS (what did you mean by ja errors ? was it mean to be javascript errors ? Not seeing anything in the logs either. And CORS enabled and seems to be working as there are hundreds of new users being created every day.

I went and did a test to try and add a new “quick user” manually through the system. I used the same email as an existing one and get the following:  
 ![image](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/3/367b2fb1c260940c6c263ca2824dbc7139cdffc2.png)

When I went and saved the contact it actually made a duplicate - is this the correct behaviour?

![image](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/7/7f3a5de17d2172598bf9d4bb2873119855fb737d.png)

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 5, 2022, 6:15pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/23 "2022-12-05T18:15:27Z")

</div>

@silavapi do you have any ideas on my last post here ? Is this expected behavior?

---

<div class="post-metadata">

**Author:** ![silavapi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/silavapi/32/7424_2.png) [@silavapi](https://forum.mautic.org/u/silavapi)\
**Post date:** [December 5, 2022, 6:24pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/24 "2022-12-05T18:24:25Z")

</div>

I don’t think that it should be duplicating the contact if you are using a + suffix - does it happen without the suffix?

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 5, 2022, 6:53pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/25 "2022-12-05T18:53:29Z")

</div>

Thanks Ruth.

I just tried without the + and got same behaviour.

 ![image0.jpeg](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/e/e039bf6be3423669fb27f411905cff0f59446881.jpeg)

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 5, 2022, 9:17pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/26 "2022-12-05T21:17:00Z")

</div>

I just tested the same case with 4.3.1, 4.4.2 and in both cases I was able to create a new user with quick add with the same email address…

This does not seem correct - or maybe I am missing something here on what a unique identifier is.

---

<div class="post-metadata">

**Author:** ![techbill](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/techbill/32/8325_2.png) [@techbill](https://forum.mautic.org/u/techbill)\
**Post date:** [December 6, 2022, 2:24am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/27 "2022-12-06T02:24:54Z")

</div>

Is custom field → email set as Unique Identifier?

When I had the duplicate email issue, it was just only one contact that had duplicate contacts with same email. It not happening anymore and I don’t know what causes it in the first place.

I just toggled the email Unique Identifier No then save and then toggle it back Yes and save again.

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 6, 2022, 7:04am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/28 "2022-12-06T07:04:50Z")

</div>

Hey @techbill - yes email is set to Unique Identifier.

I tried to toggle it off, save and on again, however still same behaviour.

I have now tested this on 4.4.3 and getting same behaviour.

I know and see when you create a contact that it gives a warning that the contact email already exists, however when I save it actually creates the duplicate.

Anyone seeing same behaviour or is it not allowing you to save the dup contact ?

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 6, 2022, 8:26am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/29 "2022-12-06T08:26:27Z")

</div>

Hi @escopecz - is it possible for you to give some direction on how to go about debugging this issue ?

I checked in the local.php config file for a specific setting and could not find anything relating to this.

Any thoughts are appreciated

---

<div class="post-metadata">

**Author:** ![escopecz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/escopecz/32/370_2.png) [@escopecz](https://forum.mautic.org/u/escopecz)\
**Post date:** [December 6, 2022, 8:54am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/30 "2022-12-06T08:54:16Z")

</div>

Mautic automatically merges contact by unique identifier field(s) when the data are coming from outside (CSV import, API, form submissions) but it will allow you to create a duplicate via UI. It should display a warning that a contact with the same email address exists as you type it in to the new contact form. But it won’t stop you nor merge. I don’t recall why it was designed this way but here we go.

There is a handy command to deduplicate all contacts based on the unique identifier fields. It will merge the field values as well as the history of both contacts.

```auto
bin/console mautic:contacts:deduplicate

```

This command is not very fast. If you have millions of contacts it can take days and even weeks. I created a pull request making it way faster and it’s waiting for one more tester to approve it and merge it:

> <https://github.com/mautic/mautic/pull/11697>
>
> \<!-- ## Which branch should I use for my PR?
> 
> Assuming that:
> 
> a = current ma…jor release
> b = current minor release
> c = future major release
> 
> \* a.x for any features and enhancements (e.g. 4.x)
> \* a.b for any bug fixes (e.g. 4.0, 4.1, 4.2)
> \* c.x for any features, enhancements or bug fixes with backward compatibility breaking changes (e.g. 5.x) --\>
> 
> | Q | A
> | -------------------------------------- | ---
> | Bug fix? (use the a.b branch) | \[n\]
> | New feature/enhancement? (use the a.x branch) | \[enhancement\]
> | Deprecations? | \[y\]
> | BC breaks? (use the c.x branch) | \[\]
> | Automated tests included? | \[y\] 
> | Issue(s) addressed | /
> 
> \<!--
> Additionally (see https://contribute.mautic.org/contributing-to-mautic/developer/code/pull-requests#work-on-your-pull-request):
> - Always add tests and ensure they pass.
> - Bug fixes must be submitted against the lowest maintained branch where they apply
> (lowest branches are regularly merged to upper ones so they get the fixes too.)
> - Features and deprecations must be submitted against the "4.x" branch.
> \--\>
> 
> \#### Description:
> 
> The deduplicate command used to iterate through all contacts and checking if a duplicate exists for it. This was very inefficient and can take weeks to run it on several million contacts.
> 
> This PR optimizes the command so that:
> \- it finds which fields are unique identifiers
> \- use those identifiers to search for contacts that have some duplicates
> \- follows the previous logic of deduplicating those duplicates
> \- splits the \`ContactDeduper\` to multiple methods so it can be used elsewhere, not only in a command
> \- adds new \`--batch\` and \`--processes\` params to support parallel execution.
> \- adds new \`mautic:contacts:deduplicate:ids\` command that is used internally to run the parallel processes for specific contact IDs.
> 
> Example usage:
> \`\`\`
> bin/console mautic:contacts:deduplicate --batch=100 --processes=10
> \`\`\`
> This will split the deduplication into multiple processes. Each process will process maximally 100 concrete duplicated contacts. And there will be maximally 10 concurrent processes running at the same time.
> 
> The command will show a progress bar of how many processes are finished and when all of them are finished then it will display outputs for all of them.
> 
> \#### Steps to test this PR:
> 
> \<!--
> This part is really important. If you want your PR to be merged, take the time to write very clear, annotated and step by step test instructions. Do not assume any previous knowledge - testers may not be developers.
> \--\>
> 1. Open this PR on Gitpod or pull down for testing locally (see docs on testing PRs \[here\](https://contribute.mautic.org/contributing-to-mautic/tester))
> 2. Create multiple contacts with duplicated email address. The contact you can generate the better as this is mainly speed optimization. We still assume that there are few duplicates within many unique contacts.
> 3. Run \`bin/console mautic:contacts:deduplicate\`. Feel free to play with the command parameters.
> 4. Check that the command output makes sense
> 5. Check that the duplicates were merged together the way you expect.
> 
> Here is a handy SQL query that will create 12 duplicated contacts:
> \`\`\`sql
> INSERT INTO \`mautic\_leads\` (\`email\`, is\_published, points, date\_identified) VALUES
> ('asfd1@sdf.dd', 1, 0, NOW()),
> ('asfd1@sdf.dd', 1, 0, NOW()),
> ('asfd2@sdf.dd', 1, 0, NOW()),
> ('asfd2@sdf.dd', 1, 0, NOW()),
> ('asfd3@sdf.dd', 1, 0, NOW()),
> ('asfd3@sdf.dd', 1, 0, NOW()),
> ('asfd4@sdf.dd', 1, 0, NOW()),
> ('asfd4@sdf.dd', 1, 0, NOW()),
> ('asfd5@sdf.dd', 1, 0, NOW()),
> ('asfd5@sdf.dd', 1, 0, NOW()),
> ('asfd6@sdf.dd', 1, 0, NOW()),
> ('asfd6@sdf.dd', 1, 0, NOW()),
> ('asfd7@sdf.dd', 1, 0, NOW()),
> ('asfd7@sdf.dd', 1, 0, NOW()),
> ('asfd8@sdf.dd', 1, 0, NOW()),
> ('asfd8@sdf.dd', 1, 0, NOW()),
> ('asfd9@sdf.dd', 1, 0, NOW()),
> ('asfd9@sdf.dd', 1, 0, NOW()),
> ('asfd10@sdf.dd', 1, 0, NOW()),
> ('asfd10@sdf.dd', 1, 0, NOW()),
> ('asfd11@sdf.dd', 1, 0, NOW()),
> ('asfd11@sdf.dd', 1, 0, NOW()),
> ('asfd12@sdf.dd', 1, 0, NOW()),
> ('asfd12@sdf.dd', 1, 0, NOW());
> \`\`\`
> 
> \#### Other areas of Mautic that may be affected by the change:
> 1. Just the deduplicate command
> 2. The dedup command was removed as I think it was created by accident in a merge or something. It's funny that the deduplicate command had a duplicate. The community has only the deduplicate command.
> 
> \#### List deprecations along with the new alternative:
> 1. \`ContactDeduper::deduplicate()\` Use the other methods in this service to compose what you need. See DeduplicateCommand for an example.
> 
> \[//\]: # ( As applicable: )
> \#### List of areas covered by the unit and/or functional tests:
> 1. The command tests that it will remove all the duplicated commands.

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 6, 2022, 9:19am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/31 "2022-12-06T09:19:43Z")

</div>

Thank you for the explanation @escopecz . I will go and pull this patch and test it myself now.

What do you think of the fact that the tracking script has the possibility of making duplicate contacts as shown above in this thread ?

In the scenario, the tracking script is creating users however our thinking is maybe our client has not implemented it correctly and it is being called multiple times and hence forming a race condition of sorts ?

I did start running the dedup a while ago and saw that it was very very slow and yes we have millions of contacts ☹

---

<div class="post-metadata">

**Author:** ![escopecz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/escopecz/32/370_2.png) [@escopecz](https://forum.mautic.org/u/escopecz)\
**Post date:** [December 6, 2022, 9:38am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/32 "2022-12-06T09:38:38Z")

</div>

The tracking script should also deduplicate contacts automatically.

But indeed. I saw it too that some users implement the tracking script multiple times on their website and so multiple same requests are going to Mautic at the same time and a race conditions causes the contacts to be created twice. It’s not hard to spot that when you load the website tracked by Mautic that there are multiple POST requests to the tracking endpoint.

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [December 18, 2022, 5:46pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/33 "2022-12-18T17:46:03Z")

</div>

will this fix work with Mautic 4.x ?

I have tried manually installing patch but did landed up breaking my install, so had to put everything back

---

<div class="post-metadata">

**Author:** ![escopecz](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/escopecz/32/370_2.png) [@escopecz](https://forum.mautic.org/u/escopecz)\
**Post date:** [December 19, 2022, 9:06am UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/34 "2022-12-19T09:06:37Z")

</div>

No, it won’t… Mautic 4 won’t have any feature release and the command performance improvement is an enhancement. Mautic 4 will get only some bug fixes.

---

<div class="post-metadata">

**Author:** ![techbill](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/techbill/32/8325_2.png) [@techbill](https://forum.mautic.org/u/techbill)\
**Post date:** [January 23, 2023, 4:32pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/35 "2023-01-23T16:32:10Z")

</div>

It happened again …

This contact was created via one of the Mautic form and It was submitted one time but duplicate was created. Look at the time frame created and database ID.

 ![duplicate](https://us1.discourse-cdn.com/flex020/uploads/mautic/original/2X/8/89f4be7849fd31932ed964367c899b5405d76181.jpeg)

---

<div class="post-metadata">

**Author:** ![joeyk](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/joeyk/32/11164_2.png) [@joeyk](https://forum.mautic.org/u/joeyk)\
**Post date:** [January 23, 2023, 5:41pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/36 "2023-01-23T17:41:16Z")

</div>

Hey!  
Is this a Mautic form? Or post from another form?

---

<div class="post-metadata">

**Author:** ![techbill](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/techbill/32/8325_2.png) [@techbill](https://forum.mautic.org/u/techbill)\
**Post date:** [January 23, 2023, 5:42pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/37 "2023-01-23T17:42:33Z")

</div>

Mautic form using automatic javascript on an another website

---

<div class="post-metadata">

**Author:** ![joeyk](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/joeyk/32/11164_2.png) [@joeyk](https://forum.mautic.org/u/joeyk)\
**Post date:** [January 23, 2023, 5:43pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/38 "2023-01-23T17:43:37Z")

</div>

Original js embed from the form?

---

<div class="post-metadata">

**Author:** ![techbill](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/techbill/32/8325_2.png) [@techbill](https://forum.mautic.org/u/techbill)\
**Post date:** [January 23, 2023, 5:45pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/39 "2023-01-23T17:45:08Z")

</div>

This

```auto
<script type="text/javascript" src="//mas.agwm.org/form/generate.js?id=17"></script>

```

---

<div class="post-metadata">

**Author:** ![miamiman](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/miamiman/32/1118_2.png) [@miamiman](https://forum.mautic.org/u/miamiman)\
**Post date:** [November 6, 2023, 10:29pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/40 "2023-11-06T22:29:26Z")

</div>

Hello there. I too am running into this - although very infrequently.

Was the root cause ever discovered?

We are pulling in information from various sources (ie forms both native and non) and will try running the dedupe cron per above.

Thanks for any feedback.

---

<div class="post-metadata">

**Author:** ![techbill](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/techbill/32/8325_2.png) [@techbill](https://forum.mautic.org/u/techbill)\
**Post date:** [November 8, 2023, 7:06pm UTC](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575/41 "2023-11-08T19:06:13Z")

</div>

This still happens to us from time to time. We haven’t found the root cause of duplications so we just delete the duplication when we spot one.

It doesn’t happen often enough that is not on top of our priority list to troubleshoot and find the cause.

[Previous page](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575.md?page=1)

[Next page](https://forum.mautic.org/t/how-did-happened-duplicate-contact/24575.md?page=3)