Mautic Community Forums

Do you think it would be useful to be able to save field mapping when uploading contacts?

TLDR: I made a feature request, but want to gather feedback on this feature.

So there you are sitting at your desk.
beachdesk

After an all nighter, Mautic is running smoothly on a dedicated server with all 12 cores humming quietly at 8% utilization, 128 Gb of memory hovering nil … terabytes of SSD space is eagerly awaiting some I/O.
server
You are ready to upload some contacts. You have set monster limits in your php.ini so you can really cram the contacts in there.

Mariadb has been tuned for performance by the top Serbian Sys Admin on Fiverr .

Your experience devil tells you “ lets see what it will do if we try to upload a million contacts at once.”
devil

Your experience angel says “ take it easy… try 20,000 contacts. You have 156 fields to map.”
angel

You sip some coffee, brush off the experience devil and wink at your experience angel. 20,000 contacts it is.
coffee

Click. Upload, Click. Wait. Ok time to map the fields. You were smart. When you created the custom fields you named as many as you could the same as the header of your CSV so mautic matches those automatically.

You clever B@stard.
1536778589_JeremySmugGif

You scan over the mapped fields one more time. Less than half matched automatically, the rest you had to select manually. Click…
frown
Contacts are uploading…

When its done… you click the upload history and on the file name and some tasty stats appear. Dang… 9.48 Contacts per second. Sweet…


Let’s do another. Click. Click… 5 minutes of clicking. About 40 minutes later another one is done… Do 10 more uploads… Lets check data…do some averages.

Oh…crrrrrrrraaaaaapppp.
crap

Lets do some math. You have 61,000,000 global contacts
That’s 3050 CSV files @20,000
Lets say 5 minutes per file to upload and map the fields.
40 minutes processing.
Oh… 137,250 minutes
2287.5 hours
95 days of 24/7 Uploading
228 days if you do nothing but this 10 hours a day
desk-sitting

Now, you are no dummy.
dummy
You don’t sit and watch uploads…
stare
You upload file, map fields, then upload in the background so you can upload the next file. Let the cron jobs trigger the next upload.
cron
Can’t speed up the server much.
Zapier has its limits too…
What would help this process?

Having the ability to save the field mapping of course. SuiteCRM (booo)
does it.


Upload file, map fields, name the mapping, click upload. The next time you upload. It’s literally a 3 click operation. Go crons go…

rocket

Do you think this would be a great mautic feature for your own uses?
help

Lets discuss

I’m a huge fan of comic books. Thx for the visual entertainment.
Like I said by your feature request. This feature exists. Just use your alias as table header.

image

@joeyk do you mean to use the “Custom Field” alias’s for the table headers ? This is what I do,
however I do agree with @EJL that this would be a nice feature to have, not a must.

@EJL are you aware of the custome import plugin ?

Hey @mikew, I just installed it yesterday and let it run overnight. So far I’m psyched!
I’m having an issue with uploads hanging or locking up until I cycle the upload off and on but that doesn’t appear to be related to the plug-in. I owe you a coffee for the suggestion

1 Like