# Best way to import 4+ million emails into mautic

**URL:** <https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095>\
**Category:** General Discussion\
**Created:** [September 30, 2021, 9:57pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095 "2021-09-30T21:57:32Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![nextafter-dan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/nextafter-dan/32/5176_2.png) [@nextafter-dan](https://forum.mautic.org/u/nextafter-dan)\
**Post date:** [September 30, 2021, 9:57pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/1 "2021-09-30T21:57:32Z")

</div>

I have a list of over 4 million contacts I’d like to get into Mautic. I’m thinking the best way is to just connect directly to the database and import them into the leads table. Would this be the best approach or is there a better way?

Thanks

---

<div class="post-metadata">

**Author:** ![mzagmajster](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mzagmajster/32/687_2.png) [@mzagmajster](https://forum.mautic.org/u/mzagmajster)\
**Post date:** [September 30, 2021, 10:48pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/2 "2021-09-30T22:48:00Z")

</div>

Hi,

I would try it with Mautic API: \< [GitHub - mautic/api-library: Mautic API Library](https://github.com/mautic/api-library) \>

---

<div class="post-metadata">

**Author:** ![mikew](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mikew/32/5156_2.png) [@mikew](https://forum.mautic.org/u/mikew)\
**Post date:** [October 1, 2021, 7:50am UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/3 "2021-10-01T07:50:28Z")

</div>

There are two options here.

1. Is to break it down into smaller files and use the custom import (there is a plugin for this that will do everything in the background)
2. Like you said to do an insert directly into the leads table. I am however unsure if there are any other indexes that need to get updated when a lead is created - but I am very interested to hear your experience once you have done it, as we often have this issue of onboarding huge amounts of data.  
This usually requires changing php.ini to allow for greater file size uploads, a pretty strong server with enough CPU as mysql ultimately dies and there is a need to monitor the Job, and to keep re-running the cron.

---

<div class="post-metadata">

**Author:** ![joeyk](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/joeyk/32/11164_2.png) [@joeyk](https://forum.mautic.org/u/joeyk)\
**Post date:** [October 1, 2021, 10:14am UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/4 "2021-10-01T10:14:24Z")

</div>

Hi,  
In my experience regular import is faster then the API.  
I would go with regular import by csv.  
The question is: how long it will take to build a segment from 4 million contacts and what Hardware will support this size of DB.  
J

---

<div class="post-metadata">

**Author:** ![nextafter-dan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/nextafter-dan/32/5176_2.png) [@nextafter-dan](https://forum.mautic.org/u/nextafter-dan)\
**Post date:** [October 1, 2021, 1:28pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/5 "2021-10-01T13:28:21Z")

</div>

What plugin are you referring to?

---

<div class="post-metadata">

**Author:** ![nextafter-dan](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/nextafter-dan/32/5176_2.png) [@nextafter-dan](https://forum.mautic.org/u/nextafter-dan)\
**Post date:** [October 1, 2021, 1:35pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/6 "2021-10-01T13:35:49Z")

</div>

You guys have some interesting points.

@joeyk I wasn’t thinking about the segment building. Is there a way to monitor that other than hitting the segments page and refreshing?

---

<div class="post-metadata">

**Author:** ![EJL](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/ejl/32/12155_2.png) [@EJL](https://forum.mautic.org/u/EJL)\
**Post date:** [October 1, 2021, 5:24pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/7 "2021-10-01T17:24:05Z")

</div>

Custom import is the plug-in I use as mentioned by @mikew . I have used dozens of times to import 80-100 million contacts in a single instance with 60 datapoints per line. I run parallel imports on files split into 500 lines. In my experience smaller files are faster to process than larger.

Real world is 5 parallel imports gives me 15-28 rows per second according to Mautic generated import history performance numbers.

What kind of hardware is your Mautic instance running? Database server been tuned?

---

<div class="post-metadata">

**Author:** ![joeyk](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/joeyk/32/11164_2.png) [@joeyk](https://forum.mautic.org/u/joeyk)\
**Post date:** [October 1, 2021, 7:08pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/8 "2021-10-01T19:08:37Z")

</div>

Run the segment building command and see how it runs: mautic:segments:update  
You need a very well tuned db server to serve that amount.

---

<div class="post-metadata">

**Author:** ![EJL](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/ejl/32/12155_2.png) [@EJL](https://forum.mautic.org/u/EJL)\
**Post date:** [October 1, 2021, 8:28pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/9 "2021-10-01T20:28:32Z")

</div>

You can run **segments:check-builders** to see query times on segments without actually running them

Description:  
Compare output of query builders for given segments

Usage:  
mautic:segments:check-builders [options]

Options:  
-i, --segment-id[=SEGMENT-ID] Set the ID of segment to process  
–skip-old Skip old query builder  
–bypass-locking Bypass locking.  
-t, --timeout=TIMEOUT If getmypid() is disabled on this system, lock files will be used. This option will assume the process is dead after the specified number of seconds and will execute anyway. This is disabled by default.  
-x, --lock\_mode=LOCK\_MODE Allowed value are “pid” or “flock”. By default, lock will try with pid, if not available will use file system [default: “pid”]  
-f, --force Deprecated; use --bypass-locking instead.  
-h, --help Display this help message  
-q, --quiet Do not output any message  
-V, --version Display this application version  
–ansi Force ANSI output  
–no-ansi Disable ANSI output  
-n, --no-interaction Do not ask any interactive question  
-e, --env=ENV The Environment name. [default: “prod”]  
–no-debug Switches off debug mode.  
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

---

<div class="post-metadata">

**Author:** ![mzagmajster](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/mzagmajster/32/687_2.png) [@mzagmajster](https://forum.mautic.org/u/mzagmajster)\
**Post date:** [October 1, 2021, 9:53pm UTC](https://forum.mautic.org/t/best-way-to-import-4-million-emails-into-mautic/21095/10 "2021-10-01T21:53:03Z")

</div>

Thanks for the info, will keep it in mind for the future.
