# Mautic stopped running this morning; maybe hacked? **URL:** \ **Category:** Product Support\ **Created:** [November 7, 2023, 5:40pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866 "2023-11-07T17:40:21Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 7, 2023, 5:40pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/1 "2023-11-07T17:40:21Z")
Hi Mauticians, Today morning, our Mautic server crashed and I see the following logs in my Apache log right before the crash: ```sh 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /s/login HTTP/1.1" 200 2815 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /text/css HTTP/1.1" 301 370 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /media/dashboards/ HTTP/1.1" 301 426 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /image/jpeg HTTP/1.1" 301 378 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /s/dashboard HTTP/1.1" 302 366 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /upgrade/upgrade.php?task=moveBundles&updateState= HTTP/1.1" 200 228 185.220.205.246 - - [07/Nov/2023:10:31:16 +0000] "GET /media/js/upgrade/upgrade.php?task=clearCache&updateState= HTTP/1.1" 500 1124 ``` It looks as if someone tried to login and/or perform some upgrade. All further requests down the line throw a 500, so that’s the last time points when requests succeeded. In the Mautic logs, I see this: ```sh [2023-11-07 00:35:06] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 00:44:04] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 00:44:05] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 00:44:05] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 01:02:04] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 02:53:04] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 06:44:06] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 08:14:39] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 08:49:01] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 09:36:00] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 10:01:25] mautic.WARNING: Received SES webhook of type 'Notification' but couldn't understand payload [] [] [2023-11-07 10:31:16] mautic.NOTICE: PHP Notice - Undefined index: path - in file /opt/bitnami/apps/mautic/htdocs/app/bundles/CoreBundle/Controller/ExceptionController.php - at line 120 {"request":"[object] (Symfony\\Component\\HttpFoundation\\Request: GET //addons HTTP/1.1\r\nAccept: */*\r\nAccept-Encoding: gzip\r\nHost: mautic.marblear.com\r\nReferer: https://52.207.144.195/addons/\r\nUser-Agent: Opera/9.80 (Windows NT 6.1; WOW64; U; en) Presto/2.2.15 Version/10.00\r\nX-Php-Ob-Level: 1\r\n\r\n)","exception":"[object] (Symfony\\Component\\Debug\\Exception\\FlattenException: {})","logger":null,"class":"Symfony\\Component\\HttpKernel\\Exception\\NotFoundHttpException","currentContent":"","layout":"Error","code":404,"anonymous":true,"baseTemplate":"MauticCoreBundle:Default:slim.html.php","templatePage":false,"template":"MauticCoreBundle:Error:404.html.php","templating":"[object] (Symfony\\Bundle\\FrameworkBundle\\Templating\\DelegatingEngine: {})","statusText":"Not Found","url":"//addons","urlParts":{"host":"addons"}} [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548, InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.ERROR: InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. - in file /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php - at line 548 [] [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548, InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.ERROR: InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. - in file /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php - at line 548 [] [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548, InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.ERROR: InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. - in file /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php - at line 548 [] [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Uncaught PHP Exception InvalidArgumentException: "The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist." at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548 {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548, InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.CRITICAL: Exception thrown when handling an exception (InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php line 548) {"exception":"[object] (InvalidArgumentException(code: 0): The \"/opt/bitnami/apps/mautic/htdocs/themes\" directory does not exist. at /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php:548)"} [] [2023-11-07 10:31:17] mautic.ERROR: InvalidArgumentException: The "/opt/bitnami/apps/mautic/htdocs/themes" directory does not exist. - in file /opt/bitnami/apps/mautic/htdocs/vendor/symfony/finder/Finder.php - at line 548 [] [] ``` I restarted the server (runs on Amazon EC2 using the outdated Bitnami package), but the server is still down. If I try to open Mautic in a web browser, no new log messages are being created in Mautic’s log. It stopped at the 10:31:17 time mark. I cleared Mautic cache already, as recommended on the troubleshooting page, but still no change. Apache itself is still running, but reports only 500. Any idea what might be going on here? Edit: If I call the URL that broke the installation: `/media/js/upgrade/upgrade.php?task=clearCache&updateState=` I get this response: ```json { "complete": true, "error": false, "updateState": "eyJwbHVnaW5Db21wbGV0ZSI6ZmFsc2UsImJ1bmRsZUNvbXBsZXRlIjpmYWxzZSwiY2FjaGVDb21wbGV0ZSI6ZmFsc2UsImNvcmVDb21wbGV0ZSI6dHJ1ZSwidmVuZG9yQ29tcGxldGUiOmZhbHNlfQ==", "stepStatus": "In Progress" } ``` ---
**Author:** ![silavapi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/silavapi/32/7424_2.png) [@silavapi](https://forum.mautic.org/u/silavapi)\ **Post date:** [November 7, 2023, 5:51pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/2 "2023-11-07T17:51:36Z")
Hi there, is it possible a user did the upgrade via the UI and it crashed? It’s easily done by the curious! If not then indeed you should look into server security etc. If it is as appears a failed upgrade, you will most likely need to either continue the upgrade at the command line, or you’ll need to roll back to your backup. > **[Mautic update failed - how to recover](https://docs.mautic.org/en/troubleshooting/update-failed)** > > . Sometimes when updating Mautic, the process might stall or fail part way through. This can cause a problem, because it can cause Mautic to be inbetween two versions and often this can make the system unusable.. . . . . . Generally speaking, updates... This might be helpful for you? ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 7, 2023, 5:55pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/3 "2023-11-07T17:55:47Z")
Hi @silavapi, Thanks for your fast response. No one of our team was using Mautic at that time. I already tried to use the Mautic console, but it is just not here on this strange Bitnami instance, although it’s mentioned in the docs. Is there a way to install this console command somehow? Unfortunately, I have no backup of the application server itself. ---
**Author:** ![silavapi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/silavapi/32/7424_2.png) [@silavapi](https://forum.mautic.org/u/silavapi)\ **Post date:** [November 7, 2023, 6:43pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/4 "2023-11-07T18:43:36Z")
I don’t know how it works on Bitnami and as far as I know they don’t support Mautic any more, so it’s going to be tough getting support on this. Does your server maybe have backups? Does this help maybe? 1. Change directory to the mautic folder : `cd /opt/bitnami/mautic` 2. From here you can run commands in the form: `php bin/console ` Source: [How to Run Mautic CLI Commands on Google Bitnami Installation | Discourse Digital](https://discoursedigital.com/blog/digital-discourse-2/how-to-run-mautic-cli-commands-on-google-bitnami-installation-71) ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 7, 2023, 6:44pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/5 "2023-11-07T18:44:07Z")
I’m not trying to recover the installation by copying the files over the installation as recommended in the document. Made a snapshot before this task. Let’s see what happens… ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 7, 2023, 7:04pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/6 "2023-11-07T19:04:36Z")
Turns out I actually had a pretty recent backup of the whole server. I restored that image from the EC2 snapshot, but now I am getting this error message on Mautic’s log: ```sh [2023-11-07 18:56:47] mautic.ERROR: [MAIL ERROR] Expected response code 250 but got an empty response (send); <<>> {"exception":"[object] (Swift_TransportException(code: 0): Expected response code 250 but got an empty response at /opt/bitnami/apps/mautic/htdocs/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php:445)"} [] ``` This is strange, as the image had been taken of a perfectly running server. It looks like the instance is trying to send out emails to people. Woah. What might cause this?! ---
**Author:** ![silavapi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/silavapi/32/7424_2.png) [@silavapi](https://forum.mautic.org/u/silavapi)\ **Post date:** [November 7, 2023, 7:37pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/7 "2023-11-07T19:37:52Z")
1. Has your server got enough space? Sometimes it can throw weird wobbly errors. 2. Have you cleared the Mautic cache? 3. If you don’t want to send emails right now, turn off all your campaigns (you can do it via the db if needed) - maybe even turn off the crons. Just be aware when you resume they’ll kick off again. When you restore the sever you’ll restore it to that state, so any campaigns in-flight will just get running again. I don’t know how Bitnami do the cron, but I’m assuming it’ll be the same directory path so it will just keep functioning as Mautic had before. ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 7, 2023, 8:11pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/8 "2023-11-07T20:11:27Z")
I stopped the server and restored the database state before the server went down. Everything seems to run normal now. Thanks for your support! I am curious: from the logs I listed, do you think this was a hacker attack? I am wondering how they actually access to the database upgrade feature. We’re using strong passwords here, but will change them now, just to be sure. ---
**Author:** ![silavapi](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/silavapi/32/7424_2.png) [@silavapi](https://forum.mautic.org/u/silavapi)\ **Post date:** [November 7, 2023, 8:37pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/9 "2023-11-07T20:37:59Z")
I don’t know enough from the server side of things to pass comment (someone else more knowledgeable may hop in later!), but it does seem a little strange for an attack vector unless your upgrade.php file has somehow been compromised. I would however strongly suggest you ensure you’re on the most up to date version of Mautic on your current major/minor, ideally that should be 4.4.10. If you’re on an outdated version, there might well be security vulnerabilities that you’re not patched against. If you’re concerned and have some budget, you can also post on the Commercial forum if you want someone to have a look at your setup for you. ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 8, 2023, 8:59am UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/10 "2023-11-08T08:59:37Z")
Thanks for the info. Actually, it happened again overnight. It seems as if someone is specifically targeting this upgrade script. I had all security patches for the Linux system installed, so I don’t think that the base system itself is the attack vector. Interestingly, after the attack, the `console` terminal command is missing in the Mautic installation. It was there before and after I restored the installation. I did not dare to update Mautic itself, since I did not want to provoke the same crash again. But I will do it now, afterr restoring the system once again. Hope that it works. ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 8, 2023, 9:17am UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/11 "2023-11-08T09:17:36Z")
Ok. I now restored the system once again, but found out that it already _was_ on the latest Mautic version: ```sh /opt/bitnami/apps/mautic/htdocs$ php bin/console mautic:update:find Great! You are running the current version of Mautic. ``` Still, the upgrade request they are calling seems to kill the whole installation, removing the `themes` folder and the `bin` folder. Any ideas what I could do to prevent this? I temporarily disabled the `upgrade.php` file. Will this also stop the command line update from working? ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 9, 2023, 6:46pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/12 "2023-11-09T18:46:09Z")
Update: Since I disabled the upgrade script, the system did not crash anymore. Still wondering why the update script can be activated from the outside in the first place. Any ideas why this is the case? ---
**Author:** ![johnwick](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/johnwick/32/3279_2.png) [@johnwick](https://forum.mautic.org/u/johnwick)\ **Post date:** [November 9, 2023, 10:00pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/13 "2023-11-09T22:00:49Z")
> [@waldgeist](#): > > ```auto > > ``` I highly recommend that you use Cloudflare Access to lock down your Mautic Instance. [yourdomain.com](http://yourdomain.com/s/login) as one of the URLs for sure. Also, check your firewall rules, etc. ---
**Author:** ![waldgeist](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/waldgeist/32/5384_2.png) [@waldgeist](https://forum.mautic.org/u/waldgeist)\ **Post date:** [November 10, 2023, 12:20pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/14 "2023-11-10T12:20:19Z")
Thanks for the hint. Will look into that. My firewall is super restrictive, only 80, 443 and 22 allowed. The problem here is that they’re calling this upgrade.php script, and this kills the Mautic instance. Since I disabled it, everything runs smoothly. To my current knowledge, they don’t get access to the server directly. It seems as if this upgrade.php script can be called without being logged in _and_ it deletes files it shouldn’t delete. But I might be wrong. ---
**Author:** ![johnwick](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/johnwick/32/3279_2.png) [@johnwick](https://forum.mautic.org/u/johnwick)\ **Post date:** [November 10, 2023, 7:55pm UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/15 "2023-11-10T19:55:02Z")
Can an put any URL into CF Access and block it so it’s only accessible to authorized individuals. ---
**Author:** ![werockmautic](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/werockmautic/32/433_2.png) [@werockmautic](https://forum.mautic.org/u/werockmautic)\ **Post date:** [February 10, 2024, 10:10am UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/16 "2024-02-10T10:10:26Z")
I’m picking up one this again, as this happened to our installation as well. We’re running the latest version of the v4 branch. ```auto 2024-02-09 15:08:41 Error 172.70.85.18 404 GET /media/js/upgrade/upgrade.php?task=clearCache&updateState= HTTP/1.0 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36 65.8 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:41 Access 172.69.43.166 200 GET /upgrade/upgrade.php?task=moveBundles&updateState= HTTP/1.0 Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:48.0) Gecko/20100101 Firefox/48.0 1.01 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:41 Error 172.70.85.18 500 GET /media/js/upgrade/upgrade.php?task=moveBundles&updateState= HTTP/1.0 Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36 3.31 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Access 172.69.194.2 200 GET /upgrade/upgrade.php?task=moveVendors&updateState= HTTP/1.0 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10; rv:50.0) Gecko/20100101 Firefox/50.0 1.07 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Error 172.69.194.84 500 GET /media/js/upgrade/upgrade.php?task=moveCore&updateState= HTTP/1.0 Opera/9.80 (Windows NT 6.1; WOW64; U; en) Presto/2.8.131 Version/11.11 1.50 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Access 172.69.195.34 200 GET /upgrade/upgrade.php?task=moveCore&updateState= HTTP/1.0 Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20100101 Firefox/6.0 1.00 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Error 172.70.85.246 500 GET /media/js/upgrade/upgrade.php?task=moveVendors&updateState= HTTP/1.0 Mozilla/5.0 (Windows NT 10.0; rv:57.0) Gecko/20100101 Firefox/57.0 2.92 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Access 172.71.242.62 302 GET /upgrade/upgrade.php?task=clearCache&updateState= HTTP/1.0 Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0 1.04 K SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Error 172.71.242.62 500 GET /upgrade/upgrade.php?task=buildCache&standalone=0&updateState=eyJwbHVnaW5Db21wbGV0ZSI6ZmFsc2UsImJ1bmRsZUNvbXBsZXRlIjpmYWxzZSwiY2FjaGVDb21wbGV0ZSI6ZmFsc2UsImNvcmVDb21wbGV0ZSI6ZmFsc2UsInZlbmRvckNvbXBsZXRlIjpmYWxzZX0= HTTP/1.0 https://OURSITE.com/upgrade/upgrade.php?task=clearCache&updateState= Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0 835 SSL/TLS-Zugriff für Apache 2024-02-09 15:08:42 Error 172.69.43.166 500 GET /index_dev.php HTTP/1.0 Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36 1.82 K SSL/TLS-Zugriff für Apache ``` After that we get errors like ```auto PHP Warning: include(/var/www/vhosts/xxxx/vendor/composer/../../app/bundles/NotificationBundle/EventListener/ConfigSubscriber.php): failed to open stream: No such file or directory in /var/www/vhosts/xxxx/vendor/composer/ClassLoader.php on line 576 PHP Warning: include(): Failed opening '/var/www/vhosts/xxxx/vendor/composer/../../app/bundles/NotificationBundle/EventListener/ConfigSubscriber.php' for inclusion (include_path='.:/opt/plesk/php/7.4/share/pear') in /var/www/vhosts/xxxx/vendor/composer/ClassLoader.php on line 576 In RegisterListenersPass.php line 111: Class "Mautic\NotificationBundle\EventListener\ConfigSubscriber" used for s ervice "mautic.notification.configbundle.subscriber" cannot be found. ``` And Mautic becomes unusable. ---
**Author:** ![werockmautic](https://sea2.discourse-cdn.com/flex020/user_avatar/forum.mautic.org/werockmautic/32/433_2.png) [@werockmautic](https://forum.mautic.org/u/werockmautic)\ **Post date:** [February 10, 2024, 10:41am UTC](https://forum.mautic.org/t/mautic-stopped-running-this-morning-maybe-hacked/29866/17 "2024-02-10T10:41:42Z")
Adding to previous post, for us: - nobody accidentally clicked on upgrade - There was no login seeable in the logs - Traffic seemingly not coming from a human - instance is behind Cloudflare I have now removed the upgrade.php manually, and hope this does not break other things.