Regression Testing in Agile - Importance, Strategies and Challenges

One of the biggest changes faced during this pandemic was switching from physical services to digital services, be it buying essentials, online education, entertainment, and even doctor visits. Users have learned that pretty much any industry they can think of that once needed a physical presence, can now be accessed using an app or website. This shift resulted in the need for more robust applications/websites, and subsequently requires software development teams make sure the software is built with care and tested thoroughly.

I recently came across this article by Cigniti that lists the Top Software Failures and another by Design News that goes over the 2020 Software Failures Linked to COVID-19. In order to avoid issues like those discussed in these articles, it is important to make sure the software is developed and tested well. QA analysts need to ensure code quality is maintained even in a fast-paced environment like Agile by testing and regression testing the software thoroughly. In this article, I will be sharing some strategies to regression test in an Agile environment despite the challenges.

Let us first understand what Regression testing is.

According to the ISTQB glossary, Regression Testing is “a type of change-related testing to detect whether defects have been introduced or uncovered in unchanged areas of the software.” Essentially, whenever the existing code has been changed or updated, we need to ensure that the unchanged and  previously working code is still working as expected.

Importance of Regression testing.

Before we dive into the strategies and challenges of performing regression tests, it’s very important to understand why you should do it. Regression testing is essential in an Agile environment because, unlike in Waterfall environments, the testing is done in each sprint. When we have code being deployed to the test environment after each iteration it makes the need to regression test even more necessary. Here are some reasons why it is of utmost importance to regression test:

• Identifies bugs earlier in the cycle / Stop bugs and issues before being deployed to production.
• It also helps in making sure the same bugs or defects should not occur again.
• Allows for a complete check of functionality after any change has been made to the code.
• Helps maintain the quality of the product.
• Prevents rework and saves time, effort, and budget of the project.
• Can be automated to save time, money, and resources needed.
• Maintains test coverage of previous test cases.
• Accelerates the time-to-market of software product.

Reasons when Regression testing must be performed.

A smart tester should be able to decide when it is critical to regression test and when it would add no value. It is essential for an organization to keep updating their software for various reasons, and to maintain the quality of the software, regression testing must be done in the following cases:

• New functionality added to the application
• New requirements added
• Bug fixes
• Patch fixes
• Changes in the source code
• Performance issue fix
• Changes in configuration

Methods to perform Regression testing.

Regression testing practices vary among organizations. It can be achieved by adopting either Manual Regression Testing or Automated Regression Testing. I prefer a combination of both. At AWH, we use Cross Browser Testing for automated test scripts where I record all the scripts and run them when needed. And when I want to do targeted testing, I always prefer manual. Whatever mode you pick you must know how to manage the extent of your regression. This can be done by selecting the right technique based on the scope of your project or scope of the change. Some common types of regression testing methods are:

• Corrective
• Progressive
• Partial
• Complete
• Selective
• Unit

Strategies for Regression Testing

In Agile, even though development is done on one new feature, testing must be performed on all new and existing features. The best way to cope with a large amount of regression testing is to design an efficient testing strategy. Here are some strategies used:

• Create a regression test suite and include all critical test cases in it.
• Prioritize the test cases in your regression test suite. High priority and critical test cases should be executed first.
• PLAN your regression- extent, time, resources, test cases.
• Always manual regression test the core business features and functionalities first. Do not fully rely on automated test scripts as you can have false positives, timeouts or other technical issues.
• Automate as many test cases as possible. This will reduce the effort and time spent doing manual regression testing especially across multiple browsers and devices.
• Create an automated regression test suite that runs periodically.
• Understand the scope and extent of your regression testing and create a plan accordingly.
• If you are not accustomed to the business or have been newly hired, invest some time in understanding the core business functionalities.
• Update the regression test suite when needed. Archive obsolete test cases and add new important ones.
• Have developers create unit test cases for every new code and execute unit-level regression testing before a test deployment.
• Record and report all bugs found during regression testing.

Challenges faced while performing regression testing in an Agile environment.

Let’s face it, most companies fail at performing regression testing effectively because the testing teams face some known and unknown challenges along the way. Some of the main challenges that are seen more frequently are:

• Expense - It’s expensive to continually test an applications functionality in every sprint.
• Time – It’s very time consuming to test large applications. There is also time is invested in creating a regression test suite and maintaining it.
• Availability – Many times testers are spread over multiple project and clients, so they have a limited amount of time available to work on each. This may lead to a miss in executing important test cases or rushing through them.
• Maintenance - From time to time maintenance is needed from testers to update the regression test suite to include new test cases or update existing ones.
• Skilled resources are needed for automating test cases.
• Lack of Knowledge - Newly hired or untrained resources do not have knowledge of the application.
• Complexity – Many times the application is too large/complex and so some bugs are not caught before releasing it to the public.
• Mundane - It can become mundane for QA teams to re-test already tested code to ensure it still works.

Organizations can overcome these challenges with the help of proper estimates, training, planning and implementation. It is never advisable to skip the regression testing phase as there are more chances that you will find a bug during this phase than anything other.

Conclusion

Change is the only constant in web and mobile software development - and regression testing is key to keep up with the change by improving the overall quality of the product. It’s rightly said that as quoted by Stf, “Software testers do not make software; they only make them better.” Do what a good tester does and work towards making the software better for the users.

Happy Testing!

Citations

ISTQB Glossary, glossary.istqb.org/en/search/regression%20testing.

15, John Blyler | Dec. “2020 Software Failures Linked to COVID-19.” Designnews.com, 15 Dec. 2020, www.designnews.com/design-software/2020-software-failures-linked-covid-19.

Cigniti Technologies Cigniti is a Global Leader in Independent Quality Engineering & Software Testing Services with offices in US. “37 Epic Software Failures That Mandate the Need for Adequate Software Testing.” Software Testing Blog by Cigniti Technologies, 3 Dec. 2020, www.cigniti.com/blog/37-software-failures-inadequate-software-testing/.

Stf. “Software Testing Quotes.” SOFTWARE TESTING Fundamentals, 6 Sept. 2020, softwaretestingfundamentals.com/software-testing-quotes/.