What Happens When Testing Goes Wrong? The Impact of Poor Software Quality

Any testing process is fundamentally an information-gathering and learning process. As a stakeholder in the product, testing permits you to obtain early feedback about the system to make informed decisions, even though testing may appear to be a simple process at first glance. This is one of the software development life cycle’s most creative and complex processes. In many ways, it is similar to healthcare. The majority of people ignore the testing process when things are going well, but as things scale up, one must make sure the testing process is in order. Many severe social and financial consequences can result from poor testing.

Throughout this article, I will discuss some unforeseen risks that may occur when testing goes wrong.

Loss of Reputation

It is difficult to build a reputation and even more difficult to maintain one. In today’s competitive world, customers are highly unforgiving of any negative experiences. If your products do not deliver the value you claim in your marketing, it is very easy for your customers to lose trust in your products. The loss of reputation damages the current product and introduces a negative bias to any potential future products. This is equivalent to indirect help to your competitor.

The following are some examples of such situations:

• Windows Blue Screen Errors & Abrupt Crash
• A 404 error occurs after a large form is filled out and submitted by the customer.
• Issues related to usability and user experience in tech products.
• Apps with accessibility issues/challenges.

In summary, customers are often unforgiving, and poor feedback may soon be found on web platforms. In today’s world, social media reviews are the new norm, and most new customers will always trust them more than your claims of improved products. If you want to see how it works, I suggest you search for the #ProductName on online platforms and see what people say about the product. Here are a few prominent platforms to check out:

• Twitter
• LinkedIn
• Facebook
• App Store
• Play Store

Unpleasant Surprises

Good testing aims to mitigate high potential risks by designing a strategy that considers your situation’s project, product, and quality context. One project’s generic strategies cannot be directly copied and pasted into another. It is common for teams to hit a blind spot regarding the quality they deliver and then experience unpleasant surprises in the production.

In most cases, production bugs are caused by one or more of the following factors:

• Unreliable assessment
• Shallow testing
• Ignorance of customer personas
• Poor understanding of product
• Poor bug advocacy or reporting

For example, I encountered two production issues while using some very basic yet important features of popular mobile applications. 

Loss of Revenue

As teams race to deliver new features and gain market share, they tend to take on technical debt by avoiding unit tests or neglecting to test. A technical project’s health eventually suffers as a result of these technical debts. After this debt has grown beyond a certain point, bugs caused by side effects become commonplace. As a result, a long-term strategy that seeks to gain market share at the expense of neglected testing can directly result in a loss of revenue.

Fixing a bug in production is extremely costly and risky. Good testing can help you save most of this cost by helping you identify issues before it reaches the customer. This way, you can focus on your business more proactively than reactively.

The severity and impact of any bug in production are inversely proportional to the likelihood that your customer will retain your subscription or service.

If revenues don’t meet initial product expectations, even large tech giants have had to kill or replace the product with a new one. Here is a compilation of tech products discontinued by tech giants like Google & Microsoft:

• Google Graveyard – Killed by Google
• Microsoft Graveyard – Killed by Microsoft

Trivia: According to the statistics, software failures cost the US economy $ 1.7 trillion.

Slow or No Feedback Loop

The information testing serves as a direct feedback loop for the product development team. An effective testing process aims to make the information-gathering process more efficient and cheaper if performed well. However, if your testing process does not adequately align with your context and needs, it can quickly slow down your feedback cycle. As a result of slow feedback, the gap between the product plan and the actual market needs eventually widens with time and leaves you in a disillusioned state. 

Lawsuits

The omnipresence of software cannot be ignored in today’s connected world. Our lives are heavily influenced and reliant on software products. The close interaction with the social world provides countless opportunities for growth as well as several unnoticed traps of potential trouble.

Over the past decade, software products have been the subject of increasing legal action and lawsuits. If you are interested in knowing some interesting case studies about how some of the most popular software products too landed up in legal troubles, I would recommend the book titled “Software Bug Stories” by Ajay Balamurugadas & Saranya Easwaran. This book includes a collection of common software bugs and lessons about your testing process and approach.

Legal issues or penalties associated with software products are commonly caused by the following issues:

• Accessibility Issues
• Security Loopholes
• User Privacy Concerns
• Data Loss
• Product Downtime

Cost of Poor Software Quality (CPSQ) in the US: A 2020 report estimates the total cost of poor software quality in the US for 2020 at $2.08 trillion. Among the biggest contributors to CPSQ are operational failures, including security and downtime issues. However, the failures and losses described above may only be the tip of the iceberg, as most such incidents go unreported.

For all our visual readers, I have summarized this entire article in the form of this mind map.

Thank you for reading this article. I hope that you gained a better understanding of the importance of having an efficient testing process and team. Our next article will discuss factors to consider when reviewing a testing strategy.