Random contacts being added to my Mautic, but I can't find traces of those contacts. Am I being hacked?

This is exactly right. Scripts are used to scan directories on servers looking for common vulnerabilities, or access to files that can be edited to give escalated privileges. The script gets a 404 error if its web based or access denied if its otherwise. Keeping your server up to date and using something like Fail2ban is an easy way to secure. Here is a snap of my Fail2Ban log with 4500+ blocked IP address for exactly this issue.

We briefly discussed this here as well: