Migration error

Your software
My PHP version is : 7.4, 8.0
My MySQL/MariaDB version is (delete as applicable): MySQL/MariaDB version 10.5

Updating/Installing Errors
I am (delete as applicable): Updating
Upgrading/installing via (delete as applicable) : Command Line

These errors are showing in the installer :
The problem seems to be with foreign keys need to be dropped

These errors are showing in the Mautic log :

Your problem
After completing migrations successfully I get the following error:

In AbstractMySQLDriver.php line 128:
                                                                                                                         
  An exception occurred while executing 'DROP INDEX IDX_1AE3441319EB6921 ON oauth2_user_client_xref':                    
  SQLSTATE[42000]: Syntax error or access violation: 1091 Can't DROP INDEX `IDX_1AE3441319EB6921`; check that it exists  
In Exception.php line 18:
  SQLSTATE[42000]: Syntax error or access violation: 1091 Can't DROP INDEX `IDX_1AE3441319EB6921`; check that it exists  
In PDOConnection.php line 141:
  SQLSTATE[42000]: Syntax error or access violation: 1091 Can't DROP INDEX `IDX_1AE3441319EB6921`; check that it exists

There is no way I can move pass this.
The indexes don’t exist, and this prohibits me to do proper upgrades.

The culprit seems to be this PR maybe?

So if you go in this file
app/bundles/CoreBundle/EventListener/DoctrineEventsSubscriber.php
and you make this change on line 138

All the migrations will be fine.
Here is the question: is this because the bug is fixed, or it’s a workaround, that should not be applied?
If this doesn’t fix it, then what would be?

PS: this fix is for sure wrong, but maybe inspires someone to point me in the right direction.

Thanks.
Joey

@mollux can you help with this so that @joeyk still has some hair left next week? :rofl:

Some addition to this issue:

  1. No migrations left.

  1. Running the following command gives me the follwing result:

bin/console doctrine:schema:update --dump-sql

 * ALTER TABLE oauth2_accesstokens CHANGE client_id client_id INT UNSIGNED NOT NULL, CHANGE user_id user_id INT UNSIGNED DEFAULT NULL, CHANGE token token VARCHAR(191) NOT NULL, CHANGE scope scope VARCHAR(191) DEFAULT NULL
 * ALTER TABLE oauth2_refreshtokens CHANGE client_id client_id INT UNSIGNED NOT NULL, CHANGE user_id user_id INT UNSIGNED NOT NULL, CHANGE token token VARCHAR(191) NOT NULL, CHANGE scope scope VARCHAR(191) DEFAULT NULL
 * ALTER TABLE oauth2_clients CHANGE id id INT UNSIGNED AUTO_INCREMENT NOT NULL, CHANGE name name VARCHAR(191) NOT NULL, CHANGE random_id random_id VARCHAR(191) NOT NULL, CHANGE secret secret VARCHAR(191) NOT NULL
 * ALTER TABLE oauth2_clients ADD CONSTRAINT FK_F9D02AE6D60322AC FOREIGN KEY (role_id) REFERENCES roles (id)

– AND FURTHER 100 lines…

But running

Either this error, or “No key found” error.

Thx for checking!

I guess you could make a snapshot, remove the offending keys, do the migration, check consistency, add the keys back (if needed).

Sounds very familiar, similar errors were found when migrating from early M3 versions to late M3 versions for Mautic instances previously migrated from M2 to M3 due to a problem in the M2 to M3 migration code that was dormant until you needed to upgrade to a newer M3 version.

Could this still be related?

Similar issues:

1 Like

I’ve been working with lot’s of ‘retro’ Mautics lately and helped a bunch of people updating from M2.
Yes, hacking the DB and manually pushing this update forward is an option, but I wish there would be a fix for this so the migration can be completed with just one command.

3 Likes

Isn’t this the same issue that I and others have reported about last year? Involving oauth2_accesstokens, foreign keys, indexes, etc.

Yes, another important issue thrown under the carpet, and if you pull the commit thread long enough (m2 to m3 migration), you will find the same person as always…

I think that some returning issues are getting in focus from time to time.

If there is enough attention it might be fixed, otherwise the workaround becomes common knowledge, get’s a documentation and life just goes on.

This bug however will cause problems to anyone who wants to update an old instance (from 2, 3).
Not just once, but even if you managed to push the migrations further, it might come back again, like an untreated sicknes.

It directly affects (effects? :slight_smile: ) Mautic’s oldest and most loyal users with every upgrade. Not just Mautic upgrade, but I ran into this when I tried to install a plugin and needed to run migrations.

Can someone PLEASE take a look and post here an answer - is this a complicated bug, or really just few small changes. I think there is a logical issue in the code, that makes the migration stuck.

Thank you in advance.

2 Likes

Exactly, it’s like the “coup de grace” for undecided long term users, they’re putting up with the usual issues, they decide to upgrade, and this thing attacks them, which is out of the scope of knowledge for most Mautic users…
I wonder how many long-term-Mautic-using companies might have been lost to this issue…

What would be the end game here @joeyk? Getting one more release of M4 with this fix? A retroactive patch for M2/M3 migrations?

2 Likes

I think the migration mentioned above has a faulty condition, that could be fixed.

1 Like

Indeed @joeyk , but how do you think this fix should be delivered to the Mautic users?

Examples:

  • A standalone PR? (but most people will have no idea what to do with a PR)
  • New migration script(s)? If so for which versions? M2 → M3 or M3–>M4?
  • A new release, like M4.4.7?
1 Like

Great question @Yosu_Cadilla , let’s see what the fix is… If the mighration is not correct, then it should be the migration file, that is fixed. I’ve seen other migration being fixed later, after we discussed it in the forums.

1 Like

That would be a great problem for a proper DBA to solve, and seeing how many unresolved issues seem to be DB related, it would make sense to ask for such to some of the Mautic contributing companies, which might already have such a talent. Or even hire some talent for a few hours as a last resort.

1 Like

Guys for so many years no resolution to this matter? Its a bit too much imho.