Mautic Community Forums

Loading 500k records freezes db

@escopecz We upgraded to 1.2.2 and have been unable to load 500K records and are now reverting back to 1.2.1. It appears that when loading the data that a series of SQL queries for building a smart list are executed that throw death lock in DB and that data is being processed on PHP and not in the DB. Is there a logging function that would reveal what is going on here?

@escopecz We upgraded to 1.2.2 and have been unable to load 500K records and are now reverting back to 1.2.1. It appears that when loading the data that a series of SQL queries for building a smart list are executed that throw death lock in DB and that data is being processed on PHP and not in the DB. Is there a logging function that would reveal what is going on here?

Check the Mautic logs in directory app/logs and the server logs.

@escopecz

undHttpException: “Requested URL not found: /testproxy.php” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /testproxy.php at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []
[2015-11-02 10:32:03] mautic.ERROR: Uncaught PHP Exception SymfonyComponentHttpKernelExceptionNotFoundHttpException: “Requested URL not found: /jmx-console/HtmlAdaptor?action=inspectMBean&name=jboss.system:type=ServerInfo” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /jmx-console/HtmlAdaptor?action=inspectMBean&name=jboss.system:type=ServerInfo at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []
[2015-11-02 11:53:43] mautic.ERROR: Uncaught PHP Exception SymfonyComponentHttpKernelExceptionNotFoundHttpException: “Requested URL not found: /testproxy.php” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /testproxy.php at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []

[2015-11-01 05:17:16] mautic.ERROR: Uncaught PHP Exception SymfonyComponentHttpKernelExceptionNotFoundHttpException: “Requested URL not found: /testproxy.php” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /testproxy.php at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []
[2015-11-01 15:17:04] mautic.ERROR: Uncaught PHP Exception SymfonyComponentHttpKernelExceptionNotFoundHttpException: “Requested URL not found: /testproxy.php” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /testproxy.php at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []
[2015-11-01 18:18:32] mautic.ERROR: Uncaught PHP Exception SymfonyComponentHttpKernelExceptionNotFoundHttpException: “Requested URL not found: /XMII/Illuminator” at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php line 352 {“exception”:"[object] (Symfony\Component\HttpKernel\Exception\NotFoundHttpException(code: 0): Requested URL not found: /XMII/Illuminator at /var/www/html/app/bundles/CoreBundle/Controller/CommonController.php:352)"} []

That’s not related to loading the leads. Someone is triggering the URL [your mautic URL]/testproxy.php and [your mautic URL]/XMII/Illuminator which is not know to Mautic. It might be an attempt to hack your Mautic or some forgotten robot. Or something completely different.

Could you describe more the problem with loading? Do you have this issue on the dashboard or when you access the /s/leads URL? We chose the way to process time related statistics in PHP instead of the database because in time of writing that parts of Mautic, Mautic supported multiple types of databases than now. And each database type had different queries for selecting data by time. Now it supports only MySql and Postgres. Both of them has different SQL queries for dealing with time (if I remember correctly), but now it could be solved by different query for every database type since there is only 2 of them.

We created a new server in EC2 with new IP and have 4 smart lists that have been constructed for test purposes with a small number of leads. We are now loading actual lead data into the system for production tests and we cannot load the list because the database hangs. It appears to be so when these leads are being loaded and the smart lists are being constructed. We tried 1.2.2 with PostGres but had difficulty so are revereting MySQL and 1.2.1.

@mihail.shumilov @escopecz we will load db logs

I have a similar problem, you were able to solve it somehow?
Our DB table has: mtc_page_hits Pass Pass Structure Structure Search Search Insert Insert Empty Empty Remove Remove ~ 785,667 InnoDB utf8_unicode_ci 780.5 MiB.
Mautic will require a login, but then freezes on the white