Regression Testing: Definition and Implementation

In this article, we’ll be discussing what regression testing is, why it’s helpful, and how to do it. Regression testing tells us how unchanged or old features will be affected when adding a feature, deleting a feature, modifying a feature, and fixing defects. In regression testing, we re-execute the same test cases in different releases or builds or test cycles to make sure that changes like adding, deleting, modifying feature, and fixing defects is not affecting the other features.

How Does Regression Testing Work?

Let’s use an example from a banking app with the following features: login, bank transfer, loans, and transaction history.

In Release 1, let us assume that only the login and bank transfer modules are released to production.

In Release 2, only the loans module is released to production.

In Release 3, transaction history is released to production.

In this scenario, the tester has to test all the features in all the releases since adding the modules may impact old/existing code.

More specifically, the purpose of this technique is to verify that modifications in the software or the environment have not caused unintended adverse side effects and that the system still meets its requirements. It is common for organizations to have a regression test suite or regression test pack. This is a set of test cases that are specifically used for regression testing. They are designed to collectively exercise most functions (the most important ones) in a system but not test anyone in detail. It is appropriate to have a regression test suite at every level of testing (component testing, integration testing, system testing, etc.). 

Do We Have to Test All the Features After Regression Testing?

Every time we test, we should follow the below procedure to ensure the system/software is working properly:

1. Always test newly added features first.
2. Test for removed or deleted features.
3. Test for the modifications.
4. Test for the defects.
5. Test for the old features.

Another question testers might ask is “How exactly we’ll know what sort of testing is required or what level of regression testing is needed?.”

Types of Regression Testing

1. Unit Regression Testing:

Unit Regression Testing includes only the changes or modifications done by developers and when there is no dependency between the modules.

2. Regional Regression Testing:

Regional Regression Testing includes the changes and impacted areas or affected areas.

Here, testers may interact with customers, business analysts, the development team, and fellow testers, along with a person with strong product knowledge. Testers will gather information about the dependency of impacted areas, consolidate and document all the information, and present this information in an impact meeting. This method tells us which areas need to be thoroughly tested.

3. Full Regression Testing:

Full Regression Testing includes the new changes and all the remaining features of an application.

When a lot of changes are made by developers, spending time doing impact analysis is time-consuming, so in this scenario, we test the entire software. Also, when the changes are made in the root of the product, we should test the entire product with Full Regression Testing.

All the test cases in a regression test suite would be executed every time a new version of the software is produced. This makes them eligible for test automation. If the regression test suite is large it may be more appropriate to select a subset for execution. Regression tests are executed whenever the software changes, either because of fixes or new or changed functionality. It is also a good idea to execute them when some aspect of the environment changes. For example, when a new version of a database management system is introduced or when a new version of a source code compiler is used.

Regular Maintenance of a Regression Test Suite Is Important Too

Regression test suites evolve over time in line with the software. As new functionality is added to a system or as old functionality is changed or removed, new regression tests should be added to the system.

As new tests are added a regression test suite may become very large. If all the tests must be executed manually, it may not be possible to execute them all every time the regression suite is used. In this case, a subset of the test cases must be chosen. This selection should be made considering the latest changes that have been made to the software. Sometimes a regression test suite of automated tests can become so large that it is impossible to execute them all. It may be possible and desirable to eliminate some test cases from a large regression test suite, especially if they are repetitive or can be combined. Another approach is to eliminate test cases that have not found a defect for a long time. 

As Linus Torvalds says, “Regression testing”? What’s that? If it compiles, it is good; if it boots up, it is perfect.” The more defects we find in this technique, the more “bug-free” the product will be later. We have discussed in this article that regression testing plays a vital role, hence we must be perfect in selecting the test cases and maintaining them. We must work on minimizing the execution time of regression testing by any means of automation.

Happy Testing!