20 Sep Two Types of Heuristics to Improve Your Testing Process!
The following is a blog based upon the Quality in a Quick video episode “Learn to Test What Needs to Be Tested Using Heuristics!” which aired on YouTube on 6/4/2018
Before I dive into today’s topic, I’d like to share with you the challenge, the problem the topic is addressing. I frequently hear managers and other leaders at many organizations state, “We don’t test enough, too many defects are getting into production.” Have you heard that?
The truth is – you’re probably testing too much! You’re probably designing and maintaining too many test cases. Perhaps you’re executing the same test for the same conditions over and over again. You’re busy (wow, you’re busy!), but you’re not productive, and you’re failing to test what truly needs to be tested. You’re not maximizing the coverage…coverage of requirements, use cases, et cetera. The solution is to design, develop, and create strategies for test planning, test design, and test data generation.
One excellent strategy is the use of heuristics. Heuristics is defined as “involving or serving as an aide to learning, discovery, or problem-solving by experimental and especially trial and error methods”. Heuristic thinking involves mental shortcuts that ease the cognitive load of decision making.
Some examples of this method include using a rule of thumb, an educated guess, perhaps an intuitive judgment, stereotyping or profiling, common sense. In other words, paying attention to your gut. Think about whenever you’ve played 20 Questions. In 20 Questions, if I state, “I’m thinking of a famous person,” in all likelihood, what are some of the first questions you’re going to ask?
Probably “is the person a female (or male)? Is he/she alive? Is the person fictitious (or real)? You might try to ask a yes/no question to focus on the occupation. Why are you asking these specific questions in this specific order? Because you’re trying to eliminate or “narrow down” the possibilities as much as possible. And each question you ask influences the next question you ask.
Testing should be the same way! With testing, it’s beneficial, to address the greatest number of logical criteria to detect a defect. From 20 Questions, what can we apply to testing? Well, identifying relevant questions addresses the larger pool of requirements, defects, or test. Using the knowledge of the subject and what we learn as we test adds value. Utilizing heuristics and exploratory methods adds value.
The two types of heuristics I’d like to emphasize are code heuristics and test case heuristics.
Code heuristics is used during test case design, perhaps even unit testing. It’s especially effective when designing a unit test in a shift-left environment when testers have strong technical skills.
The idea is that you want to focus on the high-risk code, perhaps large modules, because they tend to have more defects. Modules that have changed frequently are defect prone. Modules that have failed frequently in the past will likely fail again in the future. Modules with very high coupling with other modules or units tend to have more defects.
For object-oriented programs, classes deeper in the inheritance hierarchy tend to have more defects. These are all aspects you need to think about when utilizing the strategy of heuristics.
Test case heuristics, obviously, is focused on the test cases. Ask “which tests are uncovering the greatest number of and most significant defects?” Prioritize the test cases and only run the top few.
Tests and data that have found defects in the past will probably find them again! Look at those tests that exercise commonly used and core features. Ensure you know which tests maximize coverage of requirements and the source code. When utilizing test case heuristics, keep all of these in mind. Respect your knowledge and understand what you know.
You probably know a lot more than you think, and believe me, utilizing heuristics will truly improve your software testing processes!
If you have any feedback, any questions, or any topics you’d like address in future blogs or “Quality in a Quick” videos, please email me at bcrews@checkpointech.com. I’m Bob Crews, President and Co-Founder of Checkpoint Technologies.
Thank you so much. Make it a great day!
Bob Crews
Email: bcrews@checkpointech.com
LinkedIn: www.linkedin.com/in/bob-crews-checkpointech
