Proposal: Rethinking Mautic’s email delivery to accomodate new and old requests from developers, devops and marketers.
Let’s take a step back and look at email delivery and how it could be architected in a way everyone involved is satisfied with the result.
Ideally, implementing it in a way that the same “component” can be used by any version of Mautic M4, M5 M6 and beyond.
One architectural paradigm that would support such requirements would be a microservice, maybe as an “email delivery API” or mabe gRPC? or straight as an independent APP.
For example:
Add extra metadata to each email to “denote” how it has to be handled.
Send each email to a folder or a queue like rabbit for storage.
Create an “external” email processing microservice or headless function that can consume the emails from the queue or directory and send them taking into account the values in the metadata.
Possible values to consider as metadata for the processor:
From address
Delivery date/time
Delivery route (Local, SES. Sendgrid, etc.)
Other required values…
So basically, most (if not all) email properties, could be specified “per email”, probably in the email section of Mautic, instead of in the Config/email section. Maybe with a few “email delivery templates” that can be configured at the general level.
And then, email delivery is fully configurable and controllable per “email processing microservice” Effectively uncoupling Email delivery from Symfony and giving us the freedom to implement the email processor in any way/language/paradigm is better for the purpose of sending email. (Since Symfony seems to clearly NOT be optimal for that).
The following is the reply by @escopecz, who inspired me to start this proposal. I have added spaces and points to make it more visually appealing, the original can be found here:
What are the advantages of having another app handling email sending over a Mautic command?
Let’s find out if there is more weight on the pros or on the cons side…
Does it make sense to build something for M4 which will not be maintained next year?
Is the extra maintenance of extra app/microservice worth it?
Who will make releases for it when we have a hard time finding release leaders for Mautic?
What technologies do you propose to use for it?
What will it share with Mautic and what will be only on the microservice side?
The following should be considered:
It will need to read the configuration of email transports from Mautic.
It will have to be connected to the same queuing engine.
It will have to be connected to the same database to fill in the email send events.
That will make it a coupled microservice that heavily depend on Mautic’s resources.
From my experience, such microservices are PITA to manage and eventually are merged back to the main app.
We plan to use queues for M5 with help of Symfony Messenger
So you’ll be able to choose which queuing engine you wish to use and you can connect to it with anything you like and consume the messages. So it will be possible to implement what you suggest. But I think that Mautic users don’t want the extra hustle of installing, configuring and maintaining an extra app just to send emails. There should be a default option to do it within Mautic.
You repeated several times that Mautic is for Marketers, not developers nor sys admins. A microservice would be another boulder on the road to install Mautic.
And this is the the original OP from @rcheesley (fond in the same URL as mentioned above).
With the current implementation, there are problems with sending transactional/operational emails such as order confirmations, shipping notifications, password reset emails, or 1-2-1 emails through Mautic from the contact profile.
Emails which are sent on an individual basis to a single person, and not in bulk, should be treated differently to those which are sent in bulk.
The emails still have the headers for a bulk email
A user can unsubscribe and stop receiving these emails - there is no selectivity to exclude these important communications from the DNC setting
Other providers are already addressing this issue:
Mailchimp Subscribe to list: no precedence, no unsubscribe
Amazon SES verification successful: no precedence, no unsubscribe
Apple store delivery notification: no precedence, no unsubscribe
1-2-1 emails should not have any unsubscribe or other indication that this email comes from a list. It should be the same as if the person sends it from their personal email client.
Transactional/operational emails should be separated from marketing emails so that users are still able to receive them if they set DNC (eg to ensure they still receive shipping emails, password resets etc)
Marketing emails should remain the same as they are now (bulk and unsubscribe). It should not be possible to remove the unsubscribe header. Since this is just best practice and will only hurt the client’s domain reputation in the long run.
Transactional emails should have a global setting that enables the administrator to turn bulk and unsubscribe headers on or off (with big warnings about misusing)
A permission level so that only specific roles can create and send transactional/operational/121 emails
Ideas for how this might be approached
Note that these have been suggested and are not necessarily to be considered the right way to approach the problem. They are provided for ideas/inspiration.
Add a flag to template & segment emails for Operational (yes/no). When this is set to yes, the {unsubscribe_url} token would not be forced in to the email and the email would still be sent even if a contact has unsubscribed.
Add an option for Operational along with Transactional & Marketing in the campaign event editor. Selecting that option would do the same as the flag on the emails.
Set manual emails sent from a contact profile to automatically use the correct headers and not use any unsubscribe links
I will get started by saying that fixing the current issues and limitations of email delivery in Mautic in whatever way or fashion serves that purpose is, in my opinion, the real issue.
Whether or not to use microservices or not is just tangential.
I do think we need to start moving towards a microservices architecture, and I do believe the sooner we get started the better, because microservices are not easy, the sooner we start thinking in microservices terms the better!
However the real objective is to make an email delivery system that everyone can be happy with.