Managing Test Automation in a Growing QA Team

Test automation is one discipline every QA team should practice and use productively. While automated checks are the tester’s companion, they extend support in reducing time spent on cumbersome, redundant tasks. QA teams are supporting businesses in achieving faster time to market by driving the left shift in their test strategy. A growing team is a clear indication of a growing business, and it’s a problem good to have. However, as the QA team gets into the growth spurt, managers have a hard time keeping up with the status quo of test automation.

Growing pains are real, and managers must lead the teams and juggle the automation while keeping the deviations at bay as much as possible. When one has to scale the tests within a scaling environment, it is like walking on a wobbly ship while keeping your balance while reaching the other point. So how can managers and team leads bring stability within chaos and drive the automation tasks without compromise or delay?

Well, the answer always lies within the process. 

Unambiguous structured test processes and guidelines provide stable guardrails to keep team members focused on the goal. In this article, we’ll walk through important aspects that can be put in place while leading a growing test automation team.

The holistic view of any implementable process has three main activities:

• Plan
• Prepare
• Implement. 

Driving each stage with effective tools and techniques guarantees an impactful result. So let us dive into each stage to understand the process in detail.

Plan 

The importance of planning cannot be ignored. All successful implementations are driven by well-planned activities/tasks. Therefore, QA teams have to work towards preparing a QA charter and making it available to the entire test automation team. Let us understand more about the QA charter with the FAQs below.

What is a QA Charter’? 

A ‘QA charter’ lays out the team’s automation objectives, scope, and people responsible for achieving the objectives. The charter should provide additional information about the application’s critical features, customer demographics, technical constraints and risks, a timeline for a specific activity, milestones, success criteria, coverage metrics, etc. 

Why do we Need a QA Charter? 

QA charters give a clear starting point and lay out a stable guard rail for all team members. The charter validates what is expected from the team and the application. It also helps teams figure out all the corners through which they have to maneuver to achieve specific objectives. 

Who Should Prepare a QA Charter? 

Usually, senior roles in the teams, like Lead or Manager, should take up the responsibility of driving the QA charter. 

Can the QA Charter Change? 

Ideally, once it is identified and mapped out, it is highly discouraged to make any changes. However, having said that, if the application is going through massive change, then naturally, the QA charter may need little tweaks to match the changing need. 

Who Should Use the QA Charter? 

Any test automation team can utilize a QA charter. However, this is most effective for teams under constant change or growth spurt.

Is There a Specific Template for the QA Charter? 

No, this is a flexible structure that can be prepared as per the needs of each team. A sample QA charter is below, but the details are not restricted to the below template. 

Prepare & Implement 

After planning and before implementing, teams have to prepare the prioritized list of tasks to be put into action. Preparation of this implementable backlog needs a disciplined approach. Planning the automation tasks aids the QA team in regulating the execution of the backlog according to the timelines planned. 

Let’s address the next obvious question, “How to prepare a prioritized test automation backlog?” 

It’s quite hard to beat the test team’s perception of “everything is important, everything needs attention.” However, the day-to-day scenario of implementing everything into test automation is neither a sustainable nor scalable solution. Higher-value features should be the ones automated first. The Value-Complexity Matrix assists the test automation team in identifying the most critical scenarios.

Every application feature should be passed through the value-complexity filter to validate its importance. 

Low Value – High Complexity: Features identified as high effort with low value should be discarded from the test automation backlog. These tests turn out to be flaky in the regular test runs; they are better off being tested manually. 

Example:

• Sorting functionality 

It takes a max of 2 minutes to check if the sorting is working as expected on UI. However, if we try to implement the test through automation, the complexity of the code grows exponentially. 

Low Value – Low Complexity: Features identified to deliver low value but also needs lesser effort to implement and add to the backlog as “priority 3”. When the test team has exhausted all the implementations and has an additional effort to spend, the team can pull these priority three tasks and implement them. 

Example:

• Content elements 

Guides, FAQ section, Comment section, etc., are the Document Object Model (DOM) elements that are read-only. It is important to check if these elements are rendered onto the UI as expected. However, these are not features that can break applications. 

High Value – High Complexity: Features high in effort and value need special attention. Not everything should be picked up to automate; it is recommended to identify the most crucial features and automate only those. These tests often pose challenges in maintenance, specifically when the business functions get enhanced.

Examples: 

• Business edge cases that need mocking data 

• Authorizations/ACL 

• Multiple external application dependencies before executing the actual feature 

High Value- Low Complexity: Features delivering high value but requiring less effort to implement should be done first. They help in achieving better test confidence. 

Examples: 

• Approval scenarios 

• Billing and payments (on dummy products) 
• Form elements (CRUD operations), etc.
  
  

Measure 

The automation strategy you choose should be regularly checked for effectiveness to ensure the team is constantly progressing toward a defined goal. Metrics give a lot of information about the process and solution. It also helps tests build historical data to keep track of performance or deviations over the long run. 

How do you measure the effectiveness of the automation test strategy? 

Identify a list of KPIs that boost the overall performance of an application. Then, scale up the measurement against these KPIs. 

Below is a list of common measurable KPIs:

1. Ease of test code maintenance 
2. Stability of the test builds (frequent failures display non-reliable tests) 
3. Cycle time (overall time to execute) 
4. Test Recovery rate (How quickly the failing tests can be fixed?) 5. Reusability of test code 
5. Reusability of test code 
6. Code Understandability (Can new team members understand it easily?) 
7. Ease of executing in extended environments (other than test environments)

Conclusion 

Stabilizing automation test activities can be an exhaustive process for test managers. Well-defined processes are like guardrails to the automation teams, which assist them in running at high speeds without any disruption by keeping deviations at bay. It also prevents any assumptions and overlooking of automation goals. While it is important to spend a good amount of time charting the automation strategy, it is equally important to measure the implemented strategy’s effectiveness.