Skip to content

How e-Quallity – A Leader Testing Services Company – Automated 20X Faster with MuukTest

Overview

e-Quallity is a leading provider of testing services in Latin America with expertise in test design, execution, and automation. e-Quallity was selected to compare the performance of their current testing processes versus those of MuukTest during an actual customer’s testing engagement. MuukTest automated tests 20x faster and found 20% more defects than e-Quallity’s traditional testing methods, not to mention that the automation engineering cost was 95% lower.

Product Used for Benchmarking

e-Quallity ran the comparison test on a typical customer project. The specific application tested was for a SaaS web application written in PHP that targeted school systems. The application captures and shares students’ grades while enabling messaging between stakeholders, including teachers, administrators, tutors, and students.

How MuukTest Performed – Benchmarking

In summary, MuukTest vastly outperformed today’s methods:

e-QuallityMuukTestMuukTest Value Added
# Test Cases Designed288841+ 3X more tests
Test Design80 hours6.85 hours+ 12X faster
Test Automation136 hours6.85 hours+ 20X faster
Automation
Engineering Cost (US)1
$7,072 USD$356 USD95% lower
1 average cost of a software tester in the US is $99,082 USD ($52/hr) https://www.indeed.com/salaries/Software-Test-Engineer-Salaries

In the course of benchmarking, e-Quallity identified an opportunity for improvement. During the initial execution of MuukTest automated tests, the test engineer needed to review the results to confirm validity. This review slowed the process down. As a result, for initial execution, the performance time was slowed down to 2.4X. However, following initial execution, performance time improved up to 2.6X faster than manual execution. Given this result, we have prioritized eliminating the need to review and confirm automated test validity by creating a functionality that will perform the test engineer assessment.

Benefits & Outcomes

As noted in the benchmarking exercise, MuukTest improved test engineers’ performance in every metric.

MuukTest auto-generated 738 test cases with variations after the user created six journey scenarios with MuukTest extension. The remaining 103 scenarios were created with the Extension tool. As a result, all 738 test cases can be executed at any time, based on the user’s needs —for example, when they send a new build to production.

In addition, it is worth mentioning that the test engineer in e-Quallity was able to use MuukTest from day one, after a 1-hour onboarding. Industry competitors need more than three months of models training to provide any value.

Note: e-Quallity is an expert software testing company. Their productivity using traditional testing methods is likely much higher than that of the average software company; this means that MuukTest will perform even better than the average software company.

What’s Next for MuukTest

The benchmark also provided information to design our roadmap:
The 103 test cases created with the Extension tool will be examined to determine how to automate them with fewer inputs — improving the automation metrics.

We have shifted our focus in prioritizing our roadmap user stories related to automating test results review. We want to help the user automatically analyze the expected results of a test, which in this benchmark, required a test engineer analysis time.

(Visited 10 times, 1 visits today)