mikew
November 15, 2021, 2:26pm
1
I think @joeyk has already made a video about this in the past, however I was not able to find it.
If a user is marked as DNC will they still receive transactional emails ?
joeyk
November 15, 2021, 3:58pm
2
Hi,
Here is the full video:
1 Like
mikew
November 15, 2021, 4:55pm
3
Thank for reposting this @joeyk
I wonder if this is something that should be considered for a feature or an option to enable.
Inherently transactional emails are or could be considered ‘system crytical’ like password reset and if someone has flagged themselves as DNC, and then does such a request, they will never get that email.
I can work around this with campaigns by removing the DNC, sending out the email, then reading to DNC.
What do you think
joeyk
November 16, 2021, 8:52am
4
Hi,
It is an importatn feature, that would create a balance in Force.
There is a PR to be merged in 4.1, that might address this issue:
mautic:4.x ← tomekkowalczyk:campaign-send-email-dnc-contacts
opened 10:03AM - 13 Apr 21 UTC
| Q | A
| --------------------------------… ------ | ---
| Branch? | "features" for all features, enhancements and bug fixes (until 3.3.0 is released)
| Bug fix? | no
| New feature? | yes
| Deprecations? | no
| BC breaks? | no
| Automated tests included? | no
| Related user documentation PR URL |
| Related developer documentation PR URL |
| Issue(s) addressed |
<!--
Additionally (see https://contribute.mautic.org/contributing-to-mautic/developer/code/pull-requests#step-5-work-on-your-pull-request):
- Always add tests and ensure they pass.
- Bug fixes must be submitted against the lowest maintained branch where they apply
(lowest branches are regularly merged to upper ones so they get the fixes too.)
- Features and deprecations must be submitted against the "features" branch.
-->
<!--
Please write a short README for your feature/bugfix. This will help people understand your PR and what it aims to do.
-->
#### Description:
The functionality allows you to send email messages in the action of the campaign "Send email to contacts" to people with the DNC flag. By default, this option is disabled
<!--
If you are fixing a bug and if there is no linked issue already, please provide steps to reproduce the issue here.
-->
#### Steps to test this PR:
1. Load up [this PR](https://mautibox.com)
2. Check the "Send to SYNC contacts" option to yes
3. Run the campaign action on the user who has the DNC flag, an email should be sent
<!--
If you have any deprecations, list them here along with the new alternative.
If you have any backwards compatibility breaks, list them here.
-->
1 Like
joeyk
November 17, 2021, 6:00am
6
Thank you for this link @rcheesley . Does it mean, that this PR won’t be merged until the project is not closed? Is the next step to start the discussion?
I think we need to decide if we want to apply a sticking plaster, or fix the underlying issue.
My preference would be to fix the underlying problem, personally, but if folks wanted to use the PR above as a workaround if it’s an urgent problem for them and can’t wait for the work next year, they could do that.
The discussion should happen in the PR really, and a product decision made as to which direction we want to take and whether this solution would be required if we go down separating out the transactional/operational and marketing emails.
mikew
December 19, 2022, 5:14pm
8
I just tested this which worked well: Transactional emails in campaigns ignore the DNC setting by volha-pivavarchyk · Pull Request #11786 · mautic/mautic · GitHub
However for transactional emails that are coming from the API with /email/EMAIL_ID/contact/[ID]/send these will still not be sent
adiux
January 27, 2023, 12:20pm
9
@mikew Thanks for testing! This is a good addition. We look into it.
Also, as reference, this was found on Github:
🤔 Problem outline With the current implementation, there are problems with sending transactional/operational emails such as order confirmations, shipping notifications, password reset emails, or 1-...
And this is on the forums:
Testing is in order here, but based on quick scan of the source code, should be ok since plugins extend different parts of Mautic.
Your plugin deals with tracking, while mine are dealing with EmailTransport layer
Only tangentially related, but could be added to the development:
I think we need to decide if we want to apply a sticking plaster, or fix the underlying issue.
My preference would be to fix the underlying problem, personally, but if folks wanted to use the PR above as a workaround if it’s an urgent problem for them and can’t wait for the work next year, they could do that.
The discussion should happen in the PR really, and a product decision made as to which direction we want to take and whether this solution would be required if we go down separating out t…
And this could be the solution?
Or maybe it is the perfect time for the entire mailing system to be reconsidered?
opened 01:56PM - 20 Mar 23 UTC
closed 07:14AM - 18 Sep 23 UTC
stale
As the SwiftMailer library is dead and we must replace it with Symfony Mailer th… en this is a great opportunity to refactor the email sending in Mautic. This crucial part of Mautic has evolved in the years and each new feature added new `if` in the existing implementation. So the maintenance of this code is very costly and every change in the code can break some unrelated transport for example. The transport that support toketinzation (sending more than 1 email at once via API) use different code than SMTP-based transports. So debugging is complicated.
Symfony Mailer can send only one email per request and this won't change in the future. See https://github.com/symfony/symfony/issues/32991
There is WIP PR which removes SwiftMailer with Symfony Mailer: https://github.com/mautic/mautic/pull/11613
I suggest to improve this PR with more maintainable solution when we are forced to refactor it anyway.
What the system must support:
- [ ] Sending emails in batches by default. This is not considered in Symfony mailer at all. Each mailer sends one email at a time even if the API-based mailers can send 10k emails in 1 request. This is crucial to support for fast email sending that Mautic users are used to. Imagine you can send 1 email/second or 10k emails/second difference. These mailers will have to be created by our community.
- [ ] Ideally support the [pre-made Symfony mailers](https://symfony.com/doc/6.3/mailer.html#using-a-3rd-party-transport).
- [ ] At least prepare the possibility to have different mailer for marketing and another for transactional emails.
- [ ] Support bounce callbacks.
An iterable collection of recipients that can have 1 or several thousands or records.
```php
interface RecipientsInterface extends Iterable
{
public function add(Recipient $recipient): void;
}
```
DTO class that will hold the info we need to send a personalized email.
```php
class Recipient
{
public function getRecipietTokens(): array;
public function getEmailAddress(): Address;
}
```
Wrapper DTO that will be used to send a batch email. Can be easily extended.
```php
class BatchEmail
{
public function getRecipients(): RecipientsInterface;
public function getEmail(): Email;
public function getGlobalTokens(): array;
}
```
A factory that can check whether it supports a DSN and if so, create the transport with it.
```php
interface BatchTransportFactoryInterface
{
public function supports(Dsn $dsn): bool;
public function create(Dsn $dsn): BatchTransportInterface;
}
```
The batch transports will have to implement this interface to send batch (and/or single) emails
```php
interface BatchTransportInterface
{
public function send(BatchEmail $batchEmail): void;
}
```
Optional interface to handle bounces. It's not used anywhere in this example but will be implemented similarly like BatchTransportInterface for bounce requests.
```php
interface TransportBounceInterface
{
public function processCallbackRequest(RequestInterface $request): void;
}
```
Example of how we can use the classic Symfony transports that can use only 1 email per request.
```php
class BasicTransport implements BatchTransportInterface
{
private AnyExistingSymfonyTransport $transport
public function __construct(AnyExistingSymfonyTransport $transport)
{
$this->transport = $transport;
}
public function send(BatchEmail $batchEmail): void
{
foreach ($batchEmail->getRecipients() as $recipient) {
$batchEmail->getEmail()->to($recipient->getEmail();
$this->replaceTokens($recipient->getRecipietTokens() + $batchEmail->getGlobalTokens(), $batchEmail->getEmail()->getHtml());
$this->transport->send($email);
}
}
}
```
Example of how we can implement our own batch transports that can send thousands of emails per request.
```php
class SparkpostTransport implements BatchTransportInterface
{
private SparkpostClient $sparkPostClient
public function __construct(SparkpostClient $sparkPostClient)
{
$this->sparkPostClient = $sparkPostClient;
}
public function send(BatchEmail $batchEmail): void
{
$sparkPostMessage = $this->makeSparkPostMessage($batchEmail);
$sparkPostClient->transmissions->post($sparkPostMessage);
}
}
```
A collector that will be filled on cache build with transports that are ready to send emails. There are examples of 4 methods of how to get some specific transport. We may have 4 fields for primary, backup, marketing, transport DSNs in the configuration. If some is not configured then the primary will be used.
```php
class BatchTransportCollector
{
private CoreParametersHelper $coreParametersHelper;
private ContainerInterface $serviceLocator;
private array $transports;
public function __construct(CoreParametersHelper $coreParametersHelper, ContainerInterface $serviceLocator)
{
$this->coreParametersHelper = $coreParametersHelper;
$this->serviceLocator = $serviceLocator;
}
// There can be various transports for various use cases:
public function getPrimaryTransport(): BatchTransportInterface;
public function getBackupTransport(): BatchTransportInterface;
public function getMarketingTransport(): BatchTransportInterface;
public function getTransactionalTransport(): BatchTransportInterface;
private function init()
{
if (isset($this->transport)) {
return;
}
$dsns = $this->coreParametersHelper->get('dsns'); // an array of DSN strings
foreach ($this->serviceLocator->getProvidedServices() as $serviceId) {
$transportFactory = $this->serviceLocator->get($serviceId);
foreach ($dsns as $dsn) {
if ($transportFactory->supports($dsn)) {
$this->transports[] = $transportFactory->create($dsn);
}
}
}
}
}
```
Autowiring the batch transports. Any new class implementing the BatchTransportInterface will be automatically tagged in `app/bundles/CoreBundles/Config/services.php`:
```php
$services->instanceof(\Mautic\EmailBundle\Mailer\BatchTransportFactoryInterface::class)->tag('batch.transport.factory');
```
Then in a new BatchTransportCompilerPass.php we'll do something like
```php
class BatchTransportCompilerPass implements CompilerPassInterface
{
public function process(ContainerBuilder $containerBuilder)
{
if (!$containerBuilder->hasDefinition(BatchTransportCollector::class)) {
return;
}
$configuratorDef = $containerBuilder->findDefinition(BatchTransportCollector::class);
$locateableServices = [];
foreach ($containerBuilder->findTaggedServiceIds('batch.transport.factory') as $id => $tags) {
$locateableServices[$id] = new Reference($id);
}
$configuratorDef->addArgument(ServiceLocatorTagPass::register($containerBuilder, $locateableServices));
}
}
```
Example usage of how to send a batch/segment email:
```php
$symfonyEmail = (new Email())
->from($this->getFrom()) // Similar for cc, bcc. To will be empty here.
->subject($mauticEmail->getSubject())
->html($mauticEmail->getCustomHtml());
$recipients = new Recipients();
foreach ($contacts as $contact) {
$recipients->add(new Recipient($contact->getFields(true), $contact->getEmail()));
}
$bathEmail = new BatchEmail($symfonyEmail, $recipients, $this->getGlobalTokens());
$batchTransportCollector->getPrimaryTransport()->send($batchEmail);
```
<bountysource-plugin>
---
Want to back this issue? **[Post a bounty on it!](https://app.bountysource.com/issues/121828590-new-email-service-based-on-symfony-mailer?utm_campaign=plugin&utm_content=tracker%2F5355074&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://app.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F5355074&utm_medium=issues&utm_source=github).
</bountysource-plugin>
mikew
May 24, 2023, 7:12am
11
As is written in the last link by @escopecz it would be great to have an overall remaking of emails and how they should work in a proper marketing system, taking into account bulk email, transactional email, direct email & api email to both transact & marketing.
For the moment the second last link to the plugin mautic-custom-email-settings has solved a major problem for us and we are using it in production. It is dangerous especially when users go and clone mailers carrying over the original mailer headers… but still a good solution!
1 Like
I think now could be a good time for considering a complete refactoring of Mautic’s email sending.
Here’s what I am roposing on Github:
Right about now, some changes need to ocur in the Mautic delivery system because of deprecations and M5 requirements changing.
I understand would be harder and would take extra time, but it is not like we have a deadline for M5… or do we?
Please check the conversation on Github:
opened 01:56PM - 20 Mar 23 UTC
As the SwiftMailer library is dead and we must replace it with Symfony Mailer th… en this is a great opportunity to refactor the email sending in Mautic. This crucial part of Mautic has evolved in the years and each new feature added new `if` in the existing implementation. So the maintenance of this code is very costly and every change in the code can break some unrelated transport for example. The transport that support toketinzation (sending more than 1 email at once via API) use different code than SMTP-based transports. So debugging is complicated.
Symfony Mailer can send only one email per request and this won't change in the future. See https://github.com/symfony/symfony/issues/32991
There is WIP PR which removes SwiftMailer with Symfony Mailer: https://github.com/mautic/mautic/pull/11613
I suggest to improve this PR with more maintainable solution when we are forced to refactor it anyway.
What the system must support:
- [ ] Sending emails in batches by default. This is not considered in Symfony mailer at all. Each mailer sends one email at a time even if the API-based mailers can send 10k emails in 1 request. This is crucial to support for fast email sending that Mautic users are used to. Imagine you can send 1 email/second or 10k emails/second difference. These mailers will have to be created by our community.
- [ ] Ideally support the [pre-made Symfony mailers](https://symfony.com/doc/6.3/mailer.html#using-a-3rd-party-transport).
- [ ] At least prepare the possibility to have different mailer for marketing and another for transactional emails.
- [ ] Support bounce callbacks.
An iterable collection of recipients that can have 1 or several thousands or records.
```php
interface RecipientsInterface extends Iterable
{
public function add(Recipient $recipient): void;
}
```
DTO class that will hold the info we need to send a personalized email.
```php
class Recipient
{
public function getRecipietTokens(): array;
public function getEmailAddress(): Address;
}
```
Wrapper DTO that will be used to send a batch email. Can be easily extended.
```php
class BatchEmail
{
public function getRecipients(): RecipientsInterface;
public function getEmail(): Email;
public function getGlobalTokens(): array;
}
```
A factory that can check whether it supports a DSN and if so, create the transport with it.
```php
interface BatchTransportFactoryInterface
{
public function supports(Dsn $dsn): bool;
public function create(Dsn $dsn): BatchTransportInterface;
}
```
The batch transports will have to implement this interface to send batch (and/or single) emails
```php
interface BatchTransportInterface
{
public function send(BatchEmail $batchEmail): void;
}
```
Optional interface to handle bounces. It's not used anywhere in this example but will be implemented similarly like BatchTransportInterface for bounce requests.
```php
interface TransportBounceInterface
{
public function processCallbackRequest(RequestInterface $request): void;
}
```
Example of how we can use the classic Symfony transports that can use only 1 email per request.
```php
class BasicTransport implements BatchTransportInterface
{
private AnyExistingSymfonyTransport $transport
public function __construct(AnyExistingSymfonyTransport $transport)
{
$this->transport = $transport;
}
public function send(BatchEmail $batchEmail): void
{
foreach ($batchEmail->getRecipients() as $recipient) {
$batchEmail->getEmail()->to($recipient->getEmail();
$this->replaceTokens($recipient->getRecipietTokens() + $batchEmail->getGlobalTokens(), $batchEmail->getEmail()->getHtml());
$this->transport->send($email);
}
}
}
```
Example of how we can implement our own batch transports that can send thousands of emails per request.
```php
class SparkpostTransport implements BatchTransportInterface
{
private SparkpostClient $sparkPostClient
public function __construct(SparkpostClient $sparkPostClient)
{
$this->sparkPostClient = $sparkPostClient;
}
public function send(BatchEmail $batchEmail): void
{
$sparkPostMessage = $this->makeSparkPostMessage($batchEmail);
$sparkPostClient->transmissions->post($sparkPostMessage);
}
}
```
A collector that will be filled on cache build with transports that are ready to send emails. There are examples of 4 methods of how to get some specific transport. We may have 4 fields for primary, backup, marketing, transport DSNs in the configuration. If some is not configured then the primary will be used.
```php
class BatchTransportCollector
{
private CoreParametersHelper $coreParametersHelper;
private ContainerInterface $serviceLocator;
private array $transports;
public function __construct(CoreParametersHelper $coreParametersHelper, ContainerInterface $serviceLocator)
{
$this->coreParametersHelper = $coreParametersHelper;
$this->serviceLocator = $serviceLocator;
}
// There can be various transports for various use cases:
public function getPrimaryTransport(): BatchTransportInterface;
public function getBackupTransport(): BatchTransportInterface;
public function getMarketingTransport(): BatchTransportInterface;
public function getTransactionalTransport(): BatchTransportInterface;
private function init()
{
if (isset($this->transport)) {
return;
}
$dsns = $this->coreParametersHelper->get('dsns'); // an array of DSN strings
foreach ($this->serviceLocator->getProvidedServices() as $serviceId) {
$transportFactory = $this->serviceLocator->get($serviceId);
foreach ($dsns as $dsn) {
if ($transportFactory->supports($dsn)) {
$this->transports[] = $transportFactory->create($dsn);
}
}
}
}
}
```
Autowiring the batch transports. Any new class implementing the BatchTransportInterface will be automatically tagged in `app/bundles/CoreBundles/Config/services.php`:
```php
$services->instanceof(\Mautic\EmailBundle\Mailer\BatchTransportFactoryInterface::class)->tag('batch.transport.factory');
```
Then in a new BatchTransportCompilerPass.php we'll do something like
```php
class BatchTransportCompilerPass implements CompilerPassInterface
{
public function process(ContainerBuilder $containerBuilder)
{
if (!$containerBuilder->hasDefinition(BatchTransportCollector::class)) {
return;
}
$configuratorDef = $containerBuilder->findDefinition(BatchTransportCollector::class);
$locateableServices = [];
foreach ($containerBuilder->findTaggedServiceIds('batch.transport.factory') as $id => $tags) {
$locateableServices[$id] = new Reference($id);
}
$configuratorDef->addArgument(ServiceLocatorTagPass::register($containerBuilder, $locateableServices));
}
}
```
Example usage of how to send a batch/segment email:
```php
$symfonyEmail = (new Email())
->from($this->getFrom()) // Similar for cc, bcc. To will be empty here.
->subject($mauticEmail->getSubject())
->html($mauticEmail->getCustomHtml());
$recipients = new Recipients();
foreach ($contacts as $contact) {
$recipients->add(new Recipient($contact->getFields(true), $contact->getEmail()));
}
$bathEmail = new BatchEmail($symfonyEmail, $recipients, $this->getGlobalTokens());
$batchTransportCollector->getPrimaryTransport()->send($batchEmail);
```
<bountysource-plugin>
---
Want to back this issue? **[Post a bounty on it!](https://app.bountysource.com/issues/121828590-new-email-service-based-on-symfony-mailer?utm_campaign=plugin&utm_content=tracker%2F5355074&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://app.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F5355074&utm_medium=issues&utm_source=github).
</bountysource-plugin>
