GSoC 2024 Project Report: End to End Test Suite for Mautic

Community Blog
1

Project Overview:

Mautic currently has around 60% code coverage with unit and functional tests, but lacks comprehensive end-to-end testing. As a result, we mostly rely on users manually testing bug fixes and new features. This reliance creates challenges in ensuring the reliability and stability of new changes introduced through pull requests (PRs). Dependence on user testing often results in bugs being discovered post-release, impacting user experience and trust in the platform.

This project aimed to address these issues by focusing on four main goals:

  • Develop an End-to-End Test Suite
  • Integrate the test suite with GitHub Actions for automated testing. 
  • Write detailed documentation to support contributors in writing and executing tests effectively.
  • Educate Marketers on Contribution

Pull Requests and Commits:

S.No.

Name

Type

Status

1.

Setup Codeception and add e2e tests for contacts

Code

Merged

2.

Improve e2e test suite

Code

Merged

3.

Add Documentation for End-to-End Test Suite

Docs

Merged

4.

GitHub Action Integration for e2e test suite

CI/CD

In QA

5.

Add e2e test suite command

Code

Merged

Documentation:

https://devdocs.mautic.org/en/5.x/testing/e2e_test_suite.html

Blogs:

S.No.

Name

1.

Week 0

2.

Week 1-2

3.

Week 3-4

4.

Week 5-6

5.

Week 7-8

6.

Week 9-10

7.

Week 11-12

Proposal vs. Actual Work:

During the community bonding period, we discussed that writing E2E tests for multiple bundles might not be feasible within the project's timeframe. Therefore, we focused on completing the tests for the contacts bundle.

Future Work:

The next steps involve expanding the test suite to cover more bundles and increase code coverage.

Acknowledgement:

I want to thank my mentors—Mohit Aghera, Ruth Cheesley, and John Linhart—for always being available, giving valuable advice, and reviewing my work. Special thanks to Rahul Shinde for his help as well. My mentors have been incredibly supportive throughout the GSoC period. I’m also grateful to Google for this opportunity. The community has been very welcoming. The last three months have been an amazing learning experience, and I’m grateful for everything I’ve learned. I look forward to continuing my contributions to Mautic.

This is a companion discussion topic for the original entry at https://www.mautic.org/blog/community/gsoc-2024-project-report-end-end-test-suite-mautic
1 Like