PERFORMACE leadlists:update

Product Support
1

Hello,

I see that mautic have bad performance when I do LISTS update (execute php /var/www/mautic/app/console mautic:leadlists:update --env=prodphp /var/www/mautic/app/console mautic:leadlists:update --env=prod)

DB is MYSQL

When I see current sessions in MYSQL I see that the mautic do the following SELECT:

SELECT distinct(l.id)

FROM leads l

LEFT JOIN lead_lists_leads ll ON l.id = ll.lead_id

WHERE (l.id NOT IN (SELECT new_check.lead_id

FROM lead_lists_leads new_check

WHERE (new_check.leadlist_id = 9)

AND (new_check.lead_id = l.id)))

AND ((l.realmid = ‘1’)

AND ((ll.lead_id IS NULL) OR (ll.date_added <= ‘2015-05-22 08:01:01’))

AND (l.id <= 400004))

ORDER BY l.id ASC LIMIT 300 OFFSET 0;



so I see than the mautic works batch of 300 rows.

How can I improve my long process update of leadlists ?

How can I increase the batch from 300 to 10000 ?


2

Hello,
I see that mautic have bad performance when I do LISTS update (execute php /var/www/mautic/app/console mautic:leadlists:update --env=prodphp /var/www/mautic/app/console mautic:leadlists:update --env=prod)
DB is MYSQL
When I see current sessions in MYSQL I see that the mautic do the following SELECT:
SELECT distinct(l.id)
FROM leads l
LEFT JOIN lead_lists_leads ll ON l.id = ll.lead_id
WHERE (l.id NOT IN (SELECT new_check.lead_id
FROM lead_lists_leads new_check
WHERE (new_check.leadlist_id = 9)
AND (new_check.lead_id = l.id)))
AND ((l.realmid = ‘1’)
AND ((ll.lead_id IS NULL) OR (ll.date_added <= ‘2015-05-22 08:01:01’))
AND (l.id <= 400004))
ORDER BY l.id ASC LIMIT 300 OFFSET 0;

so I see than the mautic works batch of 300 rows.
How can I improve my long process update of leadlists ?
How can I increase the batch from 300 to 10000 ?

3

Hi Squd,

Mautic’s command line tools accept parameters for changing the batch limit as needed. So if you want to change this simply pass this param:

php /var/www/mautic/app/console mautic:leadlists:update --env=prod --batch-limit=10000

There’s some more documentation on the crons here:
https://www.mautic.org/docs/setup/index.html

You should probably test this and watch top on the CLI and see exactly how much CPU or ram is getting consumed when this executes. I have a feeling 10,000 might be a bit much unless you’ve got some pretty heavy duty hardware. If you do run well I’d like to know the specs just so we can understand what the code / hardware is capable of.

4

Hi @squd !

You must add:
–batch-limit=X
as it’s described in the really useful documentation that Mautic team is developing in https://www.mautic.org/docs/setup/index.html

5

Thanks,
Why WEB UI does not have the possibility to update on button ?

6

Hi Squd

We’ve been experimenting with different ways to trigger this. The Web UI is unable to deal with huge numbers of records. It can timeout and be unreliable, so the cron was a far more effective solution.