Product Team Meeting notes Tuesday 3rd November

Hello @here! It’s time for a new product team meeting :computer:

If you haven’t done an asynchronous meeting before, please respond in threads :slightly_smiling_face:

Also note you can start a reply with :bust_in_silhouette: to be anon, or :no_entry_sign: to go off the record and not be included in the notes, which will be exported and saved to Google Docs, and posted on the Community Forums.

The meeting will be open for 24 hours, after which the notes will be exported. People may comment thereafter but these won’t be included in the notes.

Let’s get going! :arrow_down:

:zero: Introductions - Who are you, where are you based, and how are you involved with this team?

Nick Veenhof Nick, CTO at Dropsolid. Based in Belgium and involved because Dropsolid is offering an Open DXP to its customers including Mautic as the Marketing Automation piece
Ruth Cheesley (she/her) Hello all, Ruth based in the UK, Project Lead for Mautic :mautibot:
Dennis Ameling (he/him) Dennis, based in The Netherlands and have been supporting Mautic since the beginning of this year. Excited to be here! :tada:
Norman Pracht Hi Nick, happy to see you there ! I’m Norman working in product team as much as i can :slightly_smiling_face: (edited)

:one: Review last meeting’s notes and actions (Share a link to the notes in a threaded reply, and call out any actions as sub tasks)

Dennis Ameling (he/him) @Ruth Cheesley (she/her) do we have the notes from August 25 somewhere? Only came across this one from July: Meeting notes Tuesday 28th July
Ruth Cheesley (she/her) Guessing they were not scraped over to the forums
Ruth Cheesley (she/her) I can do that
Ruth Cheesley (she/her) Meeting notes 25th August 2020 It needs some fixing, some parts haven’t come over from the Slack threads

:two: Mautic 3.2 Release Lead: who wants to step up for this?

Ruth Cheesley (she/her) I believe that we need both a release lead and a deputy unless we have someone already signed on for the deputy?
Dennis Ameling (he/him) I’m happy to be deputy - will be a great opportunity to test the (semi-)automated release process thanks to GitHub Actions, we can then drastically simplify the release lead doc as well :rocket:
Ruth Cheesley (she/her) Ok super, so long as you have the time in your schedule!
Norman Pracht I’d be happy to do it but i’m not sure @Dennis Ameling (he/him) i’m skilled enough for a version that could include migration files ? Tell me.
Norman Pracht Maybe we can do it together ?
Dennis Ameling (he/him) @Norman Pracht whenever you come across a PR that adds a file to the app/migrations/ folder, feel free to ping me or anyone else with some developer experience. Otherwise you should be able to do this :rocket:
Norman Pracht :heart:

:three: Mautic’s future direction - open discussion

Dennis Ameling (he/him) We’d like to publicly discuss Mautic’s future direction and get input from the community. We have tried to identify two possible ways forward with Mautic and put them together in this document: https://docs.google.com/document/d/1wv_qIim-IS5hxif0TX61cgSujvh7795-Hlng_-S9WBE/edit?usp=sharingPlease join us in the discussion either in the Google Doc (preferred) or in this thread on Slack!
Nick Veenhof I like what’s written here! :slightly_smiling_face:
Ruth Cheesley (she/her) That’s good to hear! Please do add any comments/thoughts you might have!

image Installing Mautic on a production server through Composer

Nick Veenhof https://drive.google.com/file/d/1leLp74ZfN_Cl3do3-qnIPxCJdmucc6W1/view But I would love to discuss this in more detail as the change required for this is quite substantial and probably needs to be pushed to 4.x or 3.2/3.3
john This is the point where I hit the wall with the Mautic Marketplace. The dir structure is not suited well for composer builds and it needs a lot of hacks to do so today. But moving directories around would be a BC break.Having Mautic installable via Composer would make the marketplace much simpler.
Nick Veenhof Yes
Nick Veenhof So if we can decide we do this for Mautic 4.0 this would be great. Any other suggestions how to handle this change is welcome. The actual patch is most likely trivial though
Nick Veenhof well, not that trivial if we want to support multiple folders for plugins, but still it won’t be that massive
john I would prefer to get rid of all core plugins for M4 and make them “easily” installable with Marketplace/Composer. (edited)
Nick Veenhof As in, make them part of the default mautic project within composer next to mautic core?
Nick Veenhof That’s not quite how Drupal handles this, but sure - we could go this way
Nick Veenhof the question then usually arises, what is part of Mautic core and how far do your integration tests go to make sure it all works
Nick Veenhof and where does that line stop
Ruth Cheesley (she/her) Definitely a topic we need to come to a consensus on but I agree @john it would be good to slim down Mautic’s core, and make it super easy for people to then selectively bring in the things that they need, rather than dump them with everything just in case. But it relies on both the underlying tooling and the marketplace to make that happen
Nick Veenhof @Ruth Cheesley (she/her) yes, however - you do need to decide which of those plugins are critical for the end-user-experience of Mautic and which are not. This heavily impacts the decision what to keep in the core product, even as a plugin, and what not.
Ruth Cheesley (she/her) Yes, indeed I see your point
Ruth Cheesley (she/her) I think that focus items is maybe the only plugin I see in the UI that is required for Mautic to function, if you want to use focus items.
Ruth Cheesley (she/her) But getting back to the original topic - I do think that from an infrastructure perspective it would make sense if we we were able to support a composer-based installation process. It would by the sounds of it be a BC break so it would have to be Mautic 4 at the earliest (May 2021) which could give us a decent amount of time to also work up the Marketplace proposition in parallel / depending on that feature
Nick Veenhof @Ruth Cheesley (she/her) you most likely could speak to Dries as well or internally in Acquia about the auto update initiative based on https://www.drupal.org/docs/8/update/automatic-updates & libsodium. How this then works with Composer -> No idea
Nick Veenhof but basically auto updating Mautic will be heavily impacted by this
Nick Veenhof either it’s composer flow, or you go the way mautic has today
Nick Veenhof they are not compatible with eachother and cause major data issues if used in combination
Nick Veenhof [#3104116] brings you deeped in the rabbit hole
Nick Veenhof deeper*
Nick Veenhof So to keep things simple it somehow needs to be clear how one can update Mautic. In my experience nearly all Drupal installations for serious installs go through composer, but the reason not to exclusively go with this is the initial install experience.
Ruth Cheesley (she/her) The main audience for Drupal is (afaik) developers. That’s not the case with Mautic. Most of our installers will be marketers. So I’m not sure it’s the same kind of user we are thinking of in terms of competence at command line, using Composer etc
Ruth Cheesley (she/her) However I do see the value that, if you use Composer in the rest of your stack, it’s super helpful to be able to use it here too.
Nick Veenhof @Ruth Cheesley (she/her) That doesn’t mean Marketeers need to have buy in from their developers in order to get it up and running or buy it from a technical party that wants to get control of the update path (hint, Acquia or Dropsolid) :slightly_smiling_face:
john We can use composer with attached UI to it. That’s what I wanted to do with the Marketplace. Composer is a PHP library and so it can be Mautic’s dependency. Composer 2 should be fast enough for HTTP requests.
Ruth Cheesley (she/her) Yes absolutely, but I am just making sure that we do remember, we are not the user in the majority of cases.
Ruth Cheesley (she/her) UI on top would be a great option
Nick Veenhof @john true, that is also what the Drupal community is doing now, and using encryption libraries to allow safe and verified updates
Nick Veenhof so that in the UI you are eating your own cake internally - would be stoked if we can use this. That said, we should allow an option to not allow auto updates so that ci/cd pipelines can take over. Or even more into the dreamland, that the auto update triggers a webhook instead of the update code so that a ci/cd pipeline can be executed and prepare a “release” for you
Ruth Cheesley (she/her) It sounds to me like this could be a really good initiative to take into Mautic 4 which would in itself have a lot of benefits (one of the areas I want us to improve is the installation process!) but also enable other innovations like the Marketplace. Do you feel like this is something you’d be up for leading @Nick Veenhof if we did make it a Mautic 4 initiative?
Nick Veenhof I need to be careful taking anything extra on my plate to be honest - certainly in these times I feel my limit has been reached. If we can get financial support somehow together I can delegate that to someone internally here or I am happy to be a soundboard for feedback. I feel honored with the question but also scared I will not be able to dedicate enough time for it
Ruth Cheesley (she/her) That’s a whole other question re. the financial support - as to how we decide to use the funds that the community has available to it
Nico Grienauer (he/him) Drupal/Analytics/Styleguides I talked also with some friends/partners, and they struggle with the current updatesystem etc too.

image Move from Travis to GitHub Actions

Dennis Ameling (he/him) Travis has recently been slowing us down in reviewing/merging PRs, due to the low amount of concurrent jobs and the large backlog they have. This PR switches Mautic to GitHub Actions and introduces (semi-)automated releases: https://github.com/mautic/mautic/pull/9355Since we already have a review from one core member, this PR is good to merge according to the Code Goverance model (https://contribute.mautic.org/community-structure/governance/code-governance).However I’d like to have a confirmation from @kuzmany and @Abu Musa before merging this. Are you OK with this? (edited)
Ruth Cheesley (she/her) I wholeheartedly support this, I’ve been talking with other Open Source project maintainers and they too are being forced away from Travis both due to the backlog, and the pricing model
Ruth Cheesley (she/her) and have seen big benefits from switching to Github Actions
Abu Musa @Dennis Ameling (he/him) I will review this PR today
Abu Musa @Dennis Ameling (he/him) I found this issue while working, seems like composer fails if you add a new composer.lockhttps://github.com/mautic/mautic/pull/9355#issuecomment-722002985
Dennis Ameling (he/him) @Abu Musa just commented in the GH PR

:six: Special characters in email addresses while importing from Hubspot - support it, or mark as incorrect?

Dennis Ameling (he/him) As reported by @kuzmany in https://mautic.slack.com/archives/CQMKV0RU1/p1604404606126800
Ruth Cheesley (she/her) My thoughts are we should look into the official docs on what is and isn’t allowed in an email address - there’s some useful links from the references on this Wikipedia article: https://en.wikipedia.org/wiki/Email_address
Ruth Cheesley (she/her) In addition to the above ASCII characters, international characters above U+007F, encoded as UTF-8, are permitted by RFC 6531, though even mail systems that support SMTPUTF8 and 8BITMIME may restrict which characters to use when assigning local-parts. (edited)

:seven: Any other business - feel free to add below! :arrow_down:

Red May we have a sample Nginx conf in the installation package or installation docunent?
Red Hi Dennis,may you also share what features will be in M4? I’d also like to know what’s the challenges to upgrade to Symfony 5?

Participants: Nick Veenhof, Ruth Cheesley (she/her), Dennis Ameling (he/him), Norman Pracht, Nico Grienauer (he/him) Drupal/Analytics/Styleguides, john, Abu Musa, Red

As the conversations in Slack spun out into threads that weren’t part of the meeting I have added separate topics for these which collate all the discussions:

  1. Future direction of Mautic
  2. Composer support