01 Apr Give Your Regression Test Suite the Love It Deserves!
The following is a blog based on the Quality in a Quick video episode “Don’t Overlook Improving Your Regression Testing” which aired on YouTube on February 12th, 2018. Watch the video here.
Regression testing is a topic so common people tend to believe there’s not much more they can learn about it. Yet, when it comes to strategic test planning, it’s an area I find many organizations are challenged.
IEEE, the Institute of Electrical and Electronics Engineers, defines regression testing as “The selective testing of software to detect defaults introduced during modification of a system or system component, to verify that modifications have not caused unintended adverse effects or to verify that a modified system or system component still meets the specified requirements.”
More simply, regression testing is the strategic process to retest something that has already been tested looking for inadvertently introduced defects. Regression testing should always be performed when: new releases of the packaged software have been put out to production; the application is enhanced or changed for any reason; and if either side of a system has changed. If there are changes to the configuration, be sure to perform regression tests and whenever changes are made after a testing stage is completed.
The two primary objectives of regression testing are to ensure:
- What has been fixed is truly fixed. Does it work as intended? Does it meet specifications and is it now fit for use? And remember, fit for use is based on what the end user, the customer, determines.
- No other defects have been introduced.
It’s very important to have a well-defined, strategic Regression Test Suite (RTS)…a collection of tests. Some quick design tips for a more effective RTS:
- Keep the tests simple and independent.
- Group linked tests into an easily identifiable test suite so those tests are executed, deleted, and moved as a single unit.
- Utilize an intuitive naming convention so it’s obvious what that particular RTS validates (i.e. What’s it’s testing objective?)
- Modularize test scripts to facilitate change handling and also to decrease maintenance.
- Lock critical tests into the fundamental test suite, so those are the core tests you’re always going to execute.
The five most common problems organizations have in creating a strategic RTS are:
- The first and most obvious is they have no regression test suite…nothing exists.
- The validity of existing RTS is unknown. Do you know whether or not any of the tests in your RTS are obsolete? Are they still valid?
- The original conditions in the RTS cannot be recreated.
- The RTS is too large to run after each change. Your Regression Test Suite is too massive. There are far too many test cases. That’s typically an indication (and we come to the fifth most common challenge)…
- The organization does not have any regression test tools. They have nothing for test automation.
But, remember, the key is to regularly ask “is our regression test suite still okay? Are all tests valid? Do any need to be modified or deleted (since they’re now obsolete)? Do they still test what we need to validate and are they finding defects? Ask yourself that. Keep your RTS up to date and give it the love it deserves! Your regression testing will go much smoother.
If you have any feedback, any questions, or any topics you’d like to see in future blogs or “Quality in a Quick” videos please email me at bcrews@checkpointech.com. I’m Bob Crews, President and Co-Founder of Checkpoint Technologies.
Thank you so much. Make it a great day!
Bob Crews
Email: bcrews@checkpointech.com
LinkedIn: www.linkedin.com/in/bob-crews-checkpointech
