readCount discrepancy on segment emails

[simcen]

Your software
My Mautic version is: v3.2.3
My PHP version is: 7.4.13
My Database type and version is: MySQL 5.7

Your problem
My problem is:
After sending a segment email, I’m noticing a discrepancy between the “readCount” property on the email entity, which is shown in list, and the actual email stats (which email is sent to which contact and is known as read).

What I’m observing is that readCount is very often higher than number of entries in email_stats belonging to the affected email where “date_read” is not null (so the email is treated as “read” for a particular contact). This also expresses when clicking on “xx Read” colored badge, only contacts appear that have read the email - which is the expected count of reads of that email.

Sample:

These errors are showing in the log:
No errors

Steps I have tried to fix the problem:
I’ve traced down the tracking of reads of emails down to “hitEmail()” (mautic/EmailModel.php at 6cf4b343af87fa8fde29dc5a7695262d184a0a1d · mautic/mautic · GitHub) method. According to that method, “upCount()” is only executed when the email isn’t read yet, so only unique reads should be counted on “readCount”. So the “readCount” counter shouldn’t be higher than actually number of entries in email_stats where date_read is not null…

Additionally, I’ve been analysing my access logs and I can exactly reproduce the email opens that lead to email_stats entries set to “is_read” = 1 and “date_read” with the first open of the email.

Does anybody have an idea what could cause readCount being higher than actual reads recorded in email_stats?

Any help is much appreciated!

Thanks,
Simon

[simcen]

I did some more testing and was able to reproduce the problem on my local setup.

The issue appears when multiple email tracking gifs for the same email of the same recipient are opened/requested at the same time.
This can happen in real life when recipients are using modern email clients on multiple devices that prefetch content before opening the email. With push-enabled clients, the chances that one recipient prefetches content of the same email on multiple devices gets even higher.

What happens is that the check if the email was read by an individual recipient was changed by another request meanwhile, the up count on the “readCount” counter on the email entity still happens.

Does anybody have similar experiences? Any ideas on how to approach that?

Thanks
Simon

[patryk.gruszka]

This looks exactly like the problem I’ve described in Multiple counting email reads · Issue #10056 · mautic/mautic · GitHub. I’ve prepared a fix for this concurrency issue, which you can review here: [DPMMA-2218] Fix email read concurrency issues by patrykgruszka · Pull Request #12673 · mautic/mautic · GitHub.