Learnings from the Trial

As I mentioned earlier, recently we conducted a trial with one of our clients to learn about how MTA could improve our processes here in Australia and reinforce our standards of user experience, application maintainability, and innovative practises. Over the course of roughly 6 weeks, a member of the client’s team and I learnt how to utilise and leverage MTA, discovered how to integrate it into our client’s existing Mendix application, and discussed MTA’s viability; its benefits as well as its shortcomings.
I’d like to share some of the key things we learnt.

First, the positives:

• Back-end data variations allow for all user journey paths to be tested during a testing cycle – if you can think of an edge case, MTA can test for it.
• Application user experience is smoother as a greater number of issues are found before reaching Live environments than with traditional QA testing.
• Once some core test cases are built, they can be rolled out and used in similar test cases, further shortening development time.
• MTA allows for front-end UI testing which can test a user’s direct UX journey through the application. Comparatively to a back-end or traditional test, there is almost no “faking” of certain test components needed, leading to an accurate, clean test that can be confidently relied on.
• MTA can also test for cases where you expect something to go wrong, such as ensuring a user does not have access to admin parts of your application.

What about some of the shortcomings?

• MTA is not a “set and forget” tool. A consistent, steady hand on the wheel is needed to continue to see positive results emerging from tests.
• Tests need to wait to be built until the application’s functionality is semi-stable. New functionalities are always in flux near the beginning, so tests cannot be reliable until the design or code of something is settled.
• MTA does have a steep learning curve, especially when one tries to apply a traditional QA mindset to building its tests. Once over that hump, however, it becomes an effective and easy-to-use tool.
• Productively can potentially see a dip during the initial set-up of MTA’s integration, due to the time needed to implement the plugins needed.