On the 3rd November we shared in the Product Team Meeting an outline of the future direction for Mautic for discussion (meeting minutes here and link to Google Doc)
In the interest of transparency and due to a proliferation of threads that spun up outside of the product team meeting (and hence will not be featured in the meeting notes) I have brought the discussions together here so that we can continue this discussion in public.
Please ensure that discussions remain respectful and in accordance with the Code of Conduct. Refresh your memory by reading it before you post
Discussions from Slack
Thread 1
| Yosu Cadilla | Wow, this is huge! Glad you decided to open the discussion to the general public. |
| Yosu Cadilla | (About the direction of future Mautic) |
| Yosu Cadilla | However I have to point out that the document is biased |
| Yosu Cadilla | What are the specific cracks in the M3 foundations? |
| Yosu Cadilla | Why do you picture M3 like a little old house? (edited) |
| Yosu Cadilla | Both options 1 and 2 would represent a bigger house |
| Yosu Cadilla | Just a different style of bigger house… (edited) |
| Yosu Cadilla | I have to say that, after just revamping Mautic’s foundations (and fixing all the cracks in the process), it is a bit odd that anyone (except Acquia) would have any interest in going that route just again, when we’re falling behing so badly in features. (edited) |
| Yosu Cadilla | When DB envisioned Mautic, it was filled with cutting edge features, on par and in some cases even beyond the capabilities of the top MA softwere out in the market at the time, |
| Yosu Cadilla | Today we are barely on par with the lesser (and I mean with less quantity of features) MA applications out there, and that is being very generous. |
| Yosu Cadilla | This reduces Mautic’s main competitive advantage, to the fact that it is free. |
| Yosu Cadilla | Not a good prospect for anyone using or selling Mautic, unless you have your own suite of products that are non-FOSS to complement Mautic’s shortcommings. |
| Yosu Cadilla | So, in my personal opinion and in defense of the average Mautic user and Mautic Agency out there, rebuilding Mautic 2 times in a row, would increase the already huge gap Mautic has with the competing tools. Bad idea!!! |
| Norman Pracht | Symfony version has to be updated again and the way Mautic is coded today is a huge issue to insure good maintenant and adding extra feature. Here is the reason. |
| Red | When I evaluate Mautic and plan to build some plugins, I would vote to upgrade the Symfony to latest. The current approach to build new feature or new user interface is too hard. |
| Yosu Cadilla | @Norman Pracht Then the problem is much bigger than we though and we need drastic measures. |
| Norman Pracht | This is what we talk about, the problem is clearly identified and the measures we are taking together are facing those issues with the right strength i believe. |
Thread 2
| Yosu Cadilla | What about we keep the future of Mautic going in here? |
| Dennis Ameling (he/him) | Here’s the big problem - if we continue with Mautic’s current codebase, it’s gonna be VERY painful to guarantee stability while doing bug fixes or introducing new features. This has been hurting end users lately (with every new release some random bugs popped up) and will continue to be a problem for the following reasons: Frontend is legacy and the framework supporting it will soon be at end of life. This might mean that there e.g. will be browser compatibility issues for end users in the near future;The backend controllers and services are full of technical debt, not covered in tests, and even circular in some cases. Changing one method has the potential to break 10 features as code is inappropriately reused for multiple purposes.The data layer is dependent on Doctrine and restricted to MySQL with a schema architecture that struggles to scale (this especially hurts users with very large instances)Backend test coverage is currently only 31% and frontend at 0% (though Acquia is working on Cypress tests for the frontend: WIP Add initial batch of Cypress E2E tests (from Acquia's QE team) by RCheesley · Pull Request #9338 · mautic/mautic · GitHub)If we want to increase test coverage and write meaningful tests, some backend parts of Mautic will have to be reworked first, as they’re not suited to write automated tests in their current stateAs mentioned in the document, it would take 2-3x as long to rework Mautic’s existing codebase (which is a prerequisite for being able to add some new features in the new feature) than to build it from the ground up with a much more modern foundation.I don’t know if you ever worked in Mautic’s codebase, but some aspects of it are a NIGHTMARE to work with and to debug, due to the reasons mentioned above. It’s one of the reasons we are still struggling to fight through the PR/issue backlog, as with every release new issues pop up in areas that aren’t covered by automated testing.From my personal perspective, it’s in the best interest of both developers and end users if Mautic was built from the ground up again. It would also allow us to spend MUCH more time on adding new features once the new foundation is there, which will benefit end users even more. Valuable time which is currently being lost to patching so many of the “cracks”. (edited) |
| Yosu Cadilla | Wow, Thank you @Dennis Ameling (he/him) Straight facts, clearly stated, and your professional opinion to round it up, this really was a useful and valuable contribution to this discussion.I am sorry to now twist everything you said in a different direction, but that’s what I am supposed to do…Here are my takeaways:A great deal of the Mautic code wasn’t written professionally, as it didn’t include ways to maintain health/stability/future development across the core and that’s what is now causing problems.For too many reasons to count, there has not been any direction at all product-wise and development-wise, as a consequence we are facing legacy issues, compatibility issues and technical debt.Maybe Symfony 3 was not the best way to go and some of the parts that were rewritten maybe were not the ones that needed re-writing and/or important things that obviously needed rewriting where left out.The Symfony migration from 2 to 3 didn’t solve the real problems lurking on the code, and whoever directed/managed the process didn’t do a great job because he/she failed to understand the real issues at hand. Proof is that just after rewriting half of the Mautic core, we find ourselves needing to re-write it entirely.And now I am going to try to give you a solution:I remember we had most of these discussions we are now having, a long time ago, first time when DB proposed to rebuild Mautic to become a headless platform (Symfony version, New UI, Database Support, and what not, even if PHP or Symfony was or was not the way to go, node was considerd) then you guys discussed it again after the Symfony migration was pushed down to us.How could so many smart people be so wrong?How where those decisions made? By whom? In what basis? Was it stability, performance, traceability?Or maybe the main path taken was just easiness for the developers, shorter path to delivery, less effort/cost.I believe the root of many of those issues is that we do not have a product team, we have a developement team named as product team and no product team at all.I believe that having a proper PRODUCT team, would prevent these issues from repeating themselves as frequently as they are, and would give the product and the development, direction.a) This team would be better built with a mix of mktg, dev, ops and biz people on it. A group made of almost exclusively developers is perfect to manage the DEVELOPMENT of Mautic, but lacks the perspective to lead Mautic as a PRODUCT.b) A a mix of people from almost every aspect touching Mautic would be needed in order to:Evaluate the current status of both Mautic and the external market/competitionDecide what is important, what is urgent and in which order things will have to take place (the current issue at hand).Study the competition, Gather feedback from both the existing and potential user base.Define new features.At this point in time, as a product, the danger of becoming feature obsolete is far greater than the danger of becoming technically obsolete.We got here, we got into this sad possition, because there is no product team, no product leadership, no product direction.The solution is simple, create a real product team! And give the Development team it’s right name too! (edited) |
| Yosu Cadilla | Product management - Wikipedia [NOTE: the last two messages were re-posted outside the thread and the discussion continued there - so the messages below are related to this, but appeared in a separate thread.] |
| Norman Pracht | It could. The Product team i manage with Dennis and Ruth has the door opened to welcome more people with several and different skills as always.But you see Yosu, on our side and with our skills and with the time we have, we do something. We do many things to guarantee you versions every month, security fixes on every minor version, and something adding new feature in the product.So you think that we don’t do good, or we don’t do enough, this is your right. But at least we do something and it is not small.And as said Alan or Nick, if you expect more nice feature, we need to have a better adapted codebase. Today the one we have makes us very slow the work on new features.In the meanwhile you do what ? Complain ? What are the solutions you propose ? What is the time you offer ? In which team are you okay to give some of your working day to make the product better ?So next time you come in this team product channel. Come with new idea, constructive and positive idea, you go and knock on another Open Source Project. |
| ekke | Hmm @Norman Pracht I understand where you’re coming from, but this sounds like a complete overreaction to me. In the text that you are replying to, I can’t see any negativism, just a “let’s rethink the way we create product strategy”.If you say our ways are already there and we are open to pile more people on the stack, that may caused by a different level of insights, or of expectations, or whatever, but I have no issues if those perspectives are discussed.My only questions would bewhen is a good time and where is the right placebut that goes more in the direction of the “future of Mautic” paper, since Yosu’s thoughts were merely an outcome of that. |
| Norman Pracht | I disagree and that goes with the rest.As i said, The Product team i manage with Dennis and Ruth has the door opened to welcome more people with several and different skills as always. |
| Nick Veenhof | I really don’t think it is that dramatic as projected. Mautic is still a very feature complete suite that is fully open source. Change is tricky, and change will go slower overtime if technical debt is not tackled. Open source is being driven by technical people that need to scratch an itch and solve business problems. A product team or core team as said could define initiativea but ultimately everybody chooses what he or she wants to work on in the form of a contribution. This could be a big one or not. Creating initiatives would help make suggestions for these directions imho |
Participants:
Norman Pracht, Red, Yosu Cadilla, Dennis Ameling (he/him), ekke