The Benefit of Exploratory Testing in Actual Cases:
In the past, software testing was seen as a strict process, but there has been a movement towards a less rigid testing method not dependent on predetermined scripts. Context-driven testing, otherwise known as exploratory testing, is becoming increasingly popular. This strategy gives testers more freedom to use their abilities and insights while also requiring them to optimize the performance of their tasks.
Recommended IPTV Service Providers
- IPTVGREAT – Rating 4.8/5 ( 600+ Reviews )
- IPTVRESALE – Rating 5/5 ( 200+ Reviews )
- IPTVGANG – Rating 4.7/5 ( 1200+ Reviews )
- IPTVUNLOCK – Rating 5/5 ( 65 Reviews )
- IPTVFOLLOW -Rating 5/5 ( 48 Reviews )
- IPTVTOPS – Rating 5/5 ( 43 Reviews )
However, not everyone is sold on the benefits of exploratory testing. Some people express concerns over its seemingly unstructured nature and the emphasis it places on individual responsibility. But such concerns mostly stem from a misunderstanding of what exploratory testing actually is. Rather than testing haphazardly without any guidelines, it is an organized and systematic method that has proven to be highly effective.
Doubters seek solid proof that exploratory testing offers advantages beyond improving tester morale. In response to this, we set up a study where we directly compared the outcomes of exploratory testing versus script-based testing. The findings are rather intriguing, as you will see.
Takeaways from this article:
Contrasting Context-driven (Exploratory Testing) with Scripted Testing Teams
A Tale of Two Teams, Two Techniques:
We split the testers into two teams with three members each. Both teams had a similar understanding of the application. The same definitions for defect severity (major, minor) were established for both teams. The same application build was given to both teams. One team, referred to as the “scripted” team, adhered to the conventional, script-based testing approach, while the “exploratory” team utilized a context-driven testing method. The testing was carried out over two rounds, each lasting three days.
The script-based team constructed 15 limited-range test cases to scrutinize five business workflows. Within this framework, the testers were not allowed to venture beyond what the script dictated.
In contrast, the exploratory team developed two visual mind maps — one for pinpointing test coverage and test charters, and another to cover product components or modules. This process generated 24 high-level test charters, which enabled contextual interpretation, thus broadening the testing range for testers.
The Initial Phase:
During the first three days, the scripted team completed six test cases and identified six major defects.
The exploratory team, on the other hand, carried out 13 test sessions ranging from 30 to 180 minutes each and discovered 10 major defects as well as five minor defects.
Interesting to note is that every defect the scripted team found was also identified by the exploratory team.
The Second Phase:
In this phase, the scripted team completed nine test cases, reporting 10 major defects along with eight minor defects.
The exploratory team ran 18 sessions and reported 14 major defects and five minor ones.
During the second phase, the scripted team found two major defects and one minor defect that the exploratory team missed. Conversely, the exploratory team found three major defects and one minor defect not reported by the scripted team.
Even though the intricacies of the selected workflows and test cases were not factored into this analysis, noteworthy conclusions can still be drawn.
What are the Implications?
Judging from the results, the exploratory method, with its emphasis on individual responsibility and flexibility, leads to more productive testing. By creating and adjusting test charters as the test sessions proceed, testers are able to cover more areas and detect defects within context. This form of liberty is lacking in script-based testing, posing a potential obstacle to defect detection.
Strict adherence to scripts results in overused paths, but it is only through diverging from these paths that all defects can be found. As some industry insiders have posited: “If you visualize a product as a field littered with landmines, with each landmine symbolizing a defect, it becomes evident that walking the same route repeatedly will not detect all of them.”
In hindsight, none of the methods was impeccable as each team reported defects that the other team overlooked, albeit more were reported overall by the exploratory team.
Ideally, the most effective way to minimize defects could be a combination of both methods. However, numerous benefits that are key selling points of the context-driven method cannot be ignored. It requires less prep time and documentation, flags issues earlier, and encourages testers to employ analytical skills and logical reasoning. Testers gain a more comprehensive understanding of the product and consequently become advocates for the end user.
Conclusion
The results of our study show that exploratory testing contributes to detecting a higher number of defects before the product is released, ensuring a better-quality product and ultimately more content and fulfilled testers. These are desirable results from any perspective.
About the Author
Mush Honda is the Quality Assurance Director at KMS Technology, a company dedicated to providing IT services throughout all stages of software development with offices in Atlanta, GA, and Ho Chi Minh City, Vietnam. He has previously been a tester at organizations such as Ernst & Young, Nexidia, Colibrium Partners, and Connecture. Among KMS services are application management, testing, support, professional services, and staff augmentation.
Do you concur with our review? Please feel free to share your thoughts and queries below.
PREVIOUS Tutorial | NEXT Tutorial #4: Exploratory Testing Using HP Sprinter