CI/CD Test Automation Tools to Automate your Test Suite

Functional testing has been around for so long that most testers are unaware that continuous testing is critical for better and faster results. In addition, this type of testing leaves more time for the QA teams to work on other tasks like exploratory testing. This article will discuss automating your test suite with a continuous integration and continuous delivery pipeline (CI/CD for short). It will introduce four popular CI/CD test automation tools that can integrate easily with some software testing automation tools. 

A Brief on Continuous Integration and Continuous Deployment

Continuous Integration and Continuous Deployment consist of a series of steps to automatically deliver a new code with all the testing done instantaneously. These steps, better known as “pipelines,” help build better software with high-quality assurance by ensuring that most of the testing is executed automatically, delivering the software to the final user once the testing is completed. By enabling these CI/CD pipelines, developers can focus on building software and simply wait to see their work go live just minutes after they’ve finished coding it.

Goal: Increase Testing Value with Fast Feedback

First, we need to have a complete test suite that covers the whole product or service. Remember, every test script needs to provide value (focus on repetitive tests and risk-based test scenarios). Developers should not run thousands of test scripts after every code check-in. This would cause long waiting times for builds to execute. So, we must carefully select a suitable test suite that will provide fast and reliable quality feedback while running our CI.

As a QA team, we need to work together to plan the CI implementation, not just jump headfirst into CI. A proper testing strategy is required to ensure we can realize the benefits of CI, namely fast and reliable feedback. Don’t be afraid of failure as part of the journey; after experiencing bumps and bruises and learning from them, the team will be ready.

Let’s Explore Four Popular CI/CD Test Automation Tools

– Jenkins 

The Jenkins CI/CD tool is an open-source project that automatically allows developers to build, test, and deploy code. It pulls the new code and starts the CI/CD cycle. It has many functions and plug-ins to add via the web browser, and it can be molded to work in the way we want it. 

While Jenkins is open source, its learning curve is high, especially for newcomers. This tool is unique in that it can work like any hosted CI, but you can configure it any way you want to accommodate your needs. It’s the most used CI/CD tool, and I recommend trying it out at least once.

Pros

• It is Open Source
• Easy configuration
• Large community

Cons

• High learning curve
• Hard to implement it correctly
• Infrastructure maintenance

Example of the Jenkins interface:

– Jenkins X

Jenkins X is another version of Jenkins that runs CI/CD pipelines integrated with Kubernetes. It reduces the time it takes to configure all Kubernetes environments, which Jenkins X incorporates natively. This Jenkins variant helps development teams to build/test/deploy much faster with the appropriate DevOps best practices.

– Travis-CI

Travis CI is another CI/CD tool. It’s free to use for open-source projects and offers a paid version for private projects. It works like a charm with GitHub, enabling you to build and test whenever a push or pull is made. In addition, it is easier to configure than Jenkins, and it has no maintenance cost since it is a hosted CI/CD.

This is a great tool to start with if you want to develop a CI/CD pipeline since it is easy to integrate with GitHub, and the configuration is outlined in an easy step-by-step guide. 

Pros

• GitHub integration
• Documentation available
• Simple to configure

Cons

• Pay to use
• Not much customization

Example of Travis-CI Interface:

 

– CircleCI 

CircleCI is a CI/CD tool that is simple to use. Its configuration must be written in a YAML file but has various features that make it a viable option. For instance, we can use environment variables for security issues; keep in mind that sensitive data must not be in the config file. It has other features like orbs, branch filtering, parallel tasks, and more.

In most cases, development teams will have more than one branch that they will be working on. To assist this process, CircleCI offers an excellent branch filtering tool. This feature lets you filter the branches that pushed new code and run the tasks that are needed. For instance, if I have two branches: master and development, I can run tests depending on the branch that was pushed or merged code. This way, we have control over what is needed to be tested and built, and are aware of any potential bugs.

CircleCI can integrate with Slack to keep track of the builds and tests. In real-time, it notifies if a build passed or failed in a Slack message, allowing developers to investigate what happened at that time and fix the reported bug immediately. 

Pros

• Easy to configure
• Cool features to use
• Gives instant feedback
• Information about build/tests in logs
• Easy GitHub integration

Cons

• Poor documentation
• Small community
• Pay to use

Here you can see how the CircleCI pipelines are laid out:

I recommend using CircleCI; it is just so straightforward to set up and maintain.

Final Thoughts

Continuous feedback is vital in reducing business risk and potential outages. Therefore, developers should visualize continuous testing as an uninterrupted process flow that embraces all testing forms, including exploratory testing and performance & load testing.

CI/CD pipelines can help us deliver new code changes to our users, using the proper continuous testing strategy to release multiple times per day. Quality assurance is crucial to have confidence in our product and make our customers satisfied and happy. 

Disclaimer: There are more CI/CD tools, such as Github Actions, Shippable, TeamCity, GitLab, CodeShip, Semaphore, GoCD, BuildKite, Bitrise; still we focused on the most popular ones.